Method and apparatus for previewing device-cloud collaborative content

Abstract

The present application provides a method and apparatus for previewing device-cloud collaborative content. The method for previewing device-cloud collaborative content provided in the present application includes the steps of: displaying an editing interface, the editing interface having a preview control and a plurality of setting controls, the plurality of setting controls being used to set parameters associated with the device-cloud collaborative content, the parameters associated with the device-cloud collaborative content including parameters of a receiving device of the device-cloud collaborative content; and displaying the preview interface based on the parameters of the receiving device in response to a trigger operation on the preview control, the preview interface being used to simulate an interface for displaying the device-cloud collaborative content on the receiving device. The solution provided in the present application can simplify the editing process and help users improve editing efficiency.

Description

[Technical Field]

[0001] This application claims priority to Chinese Patent Application No. 202211576593.7, filed with the State Intellectual Property Office of China on December 9, 2022, entitled "Method and Apparatus for Content Editing and Preview Based on Device-Cloud Collaboration," and Chinese Patent Application No. 202310780902.0, filed with the State Intellectual Property Office of China on June 28, 2023, entitled "Method and Apparatus for Previewing Device-Cloud Collaborative Content," both of which are incorporated herein by reference in their entireties.

[0002] The present application relates to device-cloud collaboration technology, and in particular to a method and apparatus for previewing device-cloud collaborative content. [Background technology]

[0003] Device-cloud collaborative content is typically content that is primarily calculated or computed using high computing power on the cloud side and presented after the remaining calculations are completed on the device side. Device-cloud collaborative games are an implementation of the aforementioned device-cloud collaborative content. The device on the cloud side completes game data calculations that require high computing power and transmits intermediate data to the device side, and the device on the device side completes the remaining calculations and presents the results.

[0004] Game content editing is one of the most important parts in game development. Typically, a user (e.g., an editor) edits game content through a game engine editor or a game editor. A device-cloud collaborative game exists on both the device side and the cloud side of the game, but has different attributes on the two sides. Therefore, during game content editing, a user uses the game engine editor to divide a game project with device-cloud collaborative content into a cloud-side version and a device-side version, synchronizes and edits the device-cloud collaborative content in the two versions, configures the attributes of the device-cloud collaborative game for each version, and then runs the cloud-side version and the device-side version on the cloud side and the device side, respectively, to obtain feedback on the final presentation result on the device side under the current configuration.

[0005] However, the aforementioned device-cloud collaborative content editing process needs to be repeated multiple times, which is tedious and labor-intensive. Summary of the Invention [Means for solving the problem]

[0006] The present application provides a method and apparatus for previewing device-cloud collaborative content to simplify the editing process, help users improve editing efficiency, and reduce labor intensity.

[0007] According to a first aspect, the present application provides a method for previewing device-cloud collaborative content, the method being executed by a generating device of the device-cloud collaborative content and including the steps of: displaying an editing interface, the editing interface including a preview control and a plurality of setting controls, the plurality of setting controls being used to set parameters associated with the device-cloud collaborative content, the parameters associated with the device-cloud collaborative content including parameters of a receiving device of the device-cloud collaborative content; and displaying the preview interface based on parameters of the receiving device in response to a trigger operation on the preview control, the preview interface being used to simulate an interface for displaying the device-cloud collaborative content on the receiving device.

[0008] In this application, the cloud side completes the entire editing process of device-cloud collaborative content by editing and simulating the device side, which allows users to autonomously modify parameter configurations and quickly obtain feedback, thereby simplifying the editing process, helping users improve editing efficiency, and reducing labor intensity.

[0009] A number of setting controls are used to set parameters related to device-cloud collaborative content. In an embodiment of the present application, the parameters related to device-cloud collaborative content include:

[0010] (1) Device-cloud collaborative content receiving device parameters The parameters are used to simulate the execution environment of the receiving device, and may include, for example, but are not limited to, the type of receiving device, the model of the receiving device, the chip model of the receiving device, the battery level of the receiving device, the resolution of the receiving device, the volume of the receiving device, the ambient temperature of the receiving device, etc. The receiving device may be simulated on the generating device based on the aforementioned parameters, so that even if an actual receiving device is not used, a virtual receiving device may be built on the generating device based on the attributes of the receiving device. In this way, the generating device may edit and generate the device-cloud collaborative content and further simulate the receiving device to present the edited and generated device-cloud collaborative content to the user, so that the user can quickly and intuitively preview the editing effect.

[0011] (2) Coding parameters The coding parameters are used to perform an encoding process on the device-cloud collaborative content to obtain a bitstream, or to perform a decoding process on the bitstream to reconstruct the device-cloud collaborative content. For example, the coding parameters may include, but are not limited to, video encoder / decoder parameters, audio encoder / decoder parameters, etc. For device-cloud collaborative content, see video / audio encoding / decoding methods. Therefore, multiple encoding / decoding parameters are involved. The generating device may encode the device-cloud collaborative content based on the encoding parameters to obtain a bitstream, and the receiving device may decode the bitstream based on the coding parameters to obtain the reconstructed device-cloud collaborative content.

[0012] (3) Transmission parameters The transmission parameters are used to simulate the conditions and environments of bitstream transmission, and may include, but are not limited to, transmit / receive frame rates, maximum transmit / receive bit rates, network delay, network instability, physical distance, physical location, transmission channel, etc. The bitstream obtained by encoding the device-cloud collaborative content needs to be transmitted from the generating device to the receiving device. To adapt to the conditions of the network or another transmission medium, the parameters transmitted by the generating device and received by the receiving device may be configured based on the aforementioned parameters. In addition, the conditions encountered when the bitstream is transmitted from the generating device to the receiving device, or within the network or on another transmission medium, are simulated to account for the changes caused by the aforementioned conditions when the bitstream arrives at the receiving device.

[0013] (4) Instance attribute parameters of device-cloud collaborative content The device-cloud collaborative content instance attribute parameters are used to generate the device-cloud collaborative content and may include, for example, but are not limited to, the position, ratio, rotation of the device-cloud collaborative content instance (e.g., an object, moving target, or person in a picture of the device-cloud collaborative content), the intensity, weight of special effects, etc. These parameters are associated with the rendering, image quality, etc. of the device-cloud collaborative content and may have the most direct impact on the presentation effect of the receiving device. Changing one or more parameter values ​​may result in a change in the presentation effect of the receiving device.

[0014] In an embodiment of the present application, a plurality of setting controls included in the editing interface correspond to a plurality of parameters, and a user may change the parameter values ​​corresponding to the setting controls by operating the setting controls.

[0015] In a possible implementation, the editing interface may include multiple windows.

[0016] The first window may correspond to a parameter of a receiving device of the device-cloud collaborative content. The first window may include a plurality of controls, the plurality of controls corresponding to a parameter of the receiving device. For example, the plurality of controls may be used to set a type of the receiving device, a model of the receiving device, a chip model of the receiving device, a battery level of the receiving device, a resolution of the receiving device, a volume of the receiving device, an ambient temperature of the receiving device, etc.

[0017] The second window may correspond to a coding parameter. The second window may include a plurality of controls, where the plurality of controls correspond to the coding parameters. For example, the plurality of controls may be used to set a video encoder / decoder parameter, an audio encoder / decoder parameter, etc.

[0018] The third window may correspond to a transmission parameter. The third window may include a plurality of controls, and the plurality of controls correspond to a transmission parameter. For example, the plurality of controls may be used to set a transmission / reception frame rate, a maximum transmission / reception bit rate, a network delay, a network instability, a physical distance, a physical location, a transmission channel, etc.

[0019] The fourth window may correspond to instance attribute parameters of the device-cloud collaborative content. The fourth window may include a plurality of controls, and the plurality of controls correspond to instance attribute parameters of the device-cloud collaborative content. For example, the plurality of controls may be used to set the position, ratio, rotation, special effect intensity, weight, etc. of the instance of the device-cloud collaborative content.

[0020] It should be noted that the four aforementioned windows do not constitute a limitation on the editing interface. The editing interface in this application may include fewer or more windows. This is related to the editing requirements of device-cloud collaborative content. The number of windows and content is not particularly limited in this application, nor are the controls included in each window particularly limited.

[0021] In a possible implementation, the editing interface may include multiple controls.

[0022] The first control may correspond to a parameter of a receiving device of the device-cloud collaborative content. The first control may include multiple sub-controls, where the multiple sub-controls correspond to parameters of the receiving device. For example, each of the multiple sub-controls may be used to set a type of the receiving device, a model of the receiving device, a chip model of the receiving device, a battery level of the receiving device, a resolution of the receiving device, a volume of the receiving device, an ambient temperature of the receiving device, etc.

[0023] The second control may correspond to a coding parameter. The second control may include multiple sub-controls, each of which corresponds to a coding parameter. For example, each of the multiple sub-controls may be used to set a video encoder / decoder parameter, an audio encoder / decoder parameter, etc.

[0024] The third control may correspond to a transmission parameter. The third control may include multiple sub-controls, and the multiple sub-controls correspond to transmission parameters. For example, each of the multiple sub-controls is used to set a transmission / reception frame rate, a maximum transmission / reception bit rate, a network delay, a network instability, a physical distance, a physical location, a transmission channel, etc.

[0025] The fourth control may correspond to an instance attribute parameter of the device-cloud collaborative content. The fourth control may include multiple sub-controls, and the multiple sub-controls correspond to instance attribute parameters of the device-cloud collaborative content. For example, each of the multiple sub-controls is used to set the position, ratio, rotation, special effect intensity, weight, etc. of the instance of the device-cloud collaborative content.

[0026] It should be noted that the above four controls do not constitute limitations on the editing interface. The editing interface in this application may include fewer or more controls, depending on the editing requirements of device-cloud collaborative content. The number and content of the controls are not particularly limited in this application, nor are the sub-controls included in each control particularly limited.

[0027] The preview control may be used to trigger the presentation of the edited device-cloud collaborative content. The preview control may be presented in each window or control, although this is not specifically limited.

[0028] The preview interface is used to simulate the interface for displaying the device-cloud collaborative content on a receiving device.

[0029] As described above, the editing interface displays multiple setting controls. Based on this, a user may perform actions on the multiple setting controls in the editing interface, such as tapping a control, entering text / numbers, or dragging a slider, to modify the values ​​of parameters corresponding to the setting controls. After the user modifies the values ​​of one or more parameters, the user may timely trigger a preview control in the editing interface to preview the impact of the modification on the device-cloud collaborative content and the effects generated therefrom.

[0030] In an embodiment of the present application, the generating device may first obtain the device-cloud collaborative content, simulate transmission conditions based on transmission parameters to simulate the device-cloud collaborative content received at the receiving device, simulate the display of the receiving device based on the parameters of the receiving device of the device-cloud collaborative content, and further display the device-cloud collaborative content on a preview interface.

[0031] Obtaining the device-cloud collaborative content may include pre-editing the device-cloud collaborative content based on the requirements of the device-cloud collaborative content. This operation does not involve transmission or presentation. Therefore, the device-cloud collaborative content may be pre-edited and then stored. Subsequent testing may be performed only on the performance of the transmitting and receiving devices. Alternatively, the device-cloud collaborative content may be obtained from a pre-set file. That is, the user pre-edits the device-cloud collaborative content based on the requirements of the device-cloud collaborative content and stores the content in the form of a file, or generates the device-cloud collaborative content based on the instance attribute parameters of the device-cloud collaborative content. In other words, in the editing process, the user may edit the content while testing the performance of the transmitting and receiving devices.

[0032] As described above, based on the transmission parameters, the conditions encountered when the bitstream is transmitted from the generating device to the receiving device, or within a network or over another medium, may be simulated to simulate the changes caused by the influence of the aforementioned conditions when the bitstream reaches the receiving device. Thus, even if the generating device does not actually transmit the bitstream, the generating device may further construct a virtual transmission path to simulate the data received by the receiving device after the bitstream is transmitted over the virtual transmission path.

[0033] The generating device may obtain a receiving device through simulation based on parameters of the receiving device of the device-cloud collaborative content, such that even if an actual receiving device is not used, a virtual receiving device may be built on the generating device based on attributes of the receiving device. Thus, the generating device may simulate a receiving device that processes the received data to obtain the device-cloud collaborative content on the receiving device.

[0034] In a possible implementation, the generating device obtains the device-cloud collaborative content, encodes the device-cloud collaborative content based on coding parameters to obtain a bitstream, transmits the bitstream based on transmission parameters, simulates a receiving device receiving the bitstream based on parameters of the receiving device of the device-cloud collaborative content, decodes the bitstream based on the coding parameters to simulate the device-cloud collaborative content received on the receiving device, and obtains a display of the receiving device obtained through the simulation, whereby the device-cloud collaborative content is displayed in a preview interface.

[0035] In the above process, the complete data processing process from the generating device to the receiving device can be simulated in detail. Specifically, the generating device obtains the device-cloud collaborative content based on the instance attribute parameters of the device-cloud collaborative content, encodes the device-cloud collaborative content based on coding parameters to obtain a bitstream, and then transmits the bitstream based on transmission parameters. In the bitstream transmission process, the generating device simulates a transmission environment as if the bitstream were being transmitted in a real environment. The receiving device (obtained by the generating device through simulation based on the parameters of the receiving device) receives the bitstream, decodes it based on the coding parameters, the attributes of the receiving device, and the capabilities of the receiving device, etc., reconstructs the device-cloud collaborative content based on the instance attribute parameters of the device-cloud collaborative content, and displays the device-cloud collaborative content on a preview interface.

[0036] The generating device simulates a virtual receiving device based on the parameters of the receiving device, and displays the content pictures acquired under the aforementioned parameter configurations to the user in the form of a preview interface, so that the user can quickly and intuitively see the effects that may be presented on the receiving device under the aforementioned configured parameters, in order to timely modify the corresponding parameters based on requirements.

[0037] According to a second aspect, the present application provides an apparatus for previewing device-cloud collaborative content, including a display module. The display module is configured to display an editing interface, the editing interface including a preview control and a plurality of setting controls, the plurality of setting controls being used to set parameters associated with the device-cloud collaborative content, the parameters associated with the device-cloud collaborative content including parameters of a receiving device of the device-cloud collaborative content. The display module is further configured to display the preview interface based on the parameters of the receiving device in response to a trigger operation on the preview control, the preview interface being used to simulate an interface for displaying the device-cloud collaborative content on the receiving device.

[0038] In a possible implementation, the parameters of the receiving device include at least one of the following: a type of the receiving device, a model of the receiving device, a chip model of the receiving device, a battery level of the receiving device, a resolution of the receiving device, a volume of the receiving device, and an ambient temperature of the receiving device.

[0039] In a possible implementation, the parameters associated with the device-cloud collaborative content further include coding parameters and / or transmission parameters, where the coding parameters are used to perform an encoding process on the device-cloud collaborative content to obtain a bitstream or a decoding process on the bitstream to reconstruct the device-cloud collaborative content, and the transmission parameters are used to simulate a situation and environment for transmitting the bitstream.

[0040] In a possible implementation, the display module is further configured to display a coding parameter setting window in response to a trigger operation on the first setting control, the coding parameter setting window including a plurality of first setting sub-controls, each of the plurality of first setting sub-controls being used to set a video encoder / decoder parameter and an audio encoder / decoder parameter, and the first setting control being one of the plurality of setting controls.

[0041] In a possible implementation, the display module is further configured to display a transmission parameter setting window in response to a trigger operation on the second setting control, the transmission parameter setting window including a plurality of second setting sub-controls, each of which is used to set a transmit / receive frame rate, a maximum transmit / receive bit rate, a network delay, a network instability, a physical distance, a physical location, and a transmission channel, and the second setting control is one of the plurality of setting controls.

[0042] In a possible implementation, the parameters associated with the device-cloud collaborative content further include instance attribute parameters of the device-cloud collaborative content, which are used to generate the device-cloud collaborative content.

[0043] In a possible implementation, the display module is further configured to display an instance attribute parameter setting window in response to a trigger operation on the third setting control, the instance attribute parameter setting window including a plurality of third setting sub-controls, each of the plurality of third setting sub-controls being used to set the position, ratio, rotation, special effect intensity, and weight of an instance of the device-cloud collaborative content, and the third setting control being one of the plurality of setting controls.

[0044] In possible implementations, the device-cloud collaborative content includes content of a device-cloud collaborative game, content of a device-cloud collaborative document, or content of a device-cloud collaborative live broadcast.

[0045] According to a third aspect, the present application provides an electronic device including one or more processors and a memory configured to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors are enabled to perform a method according to any one of the implementation forms of the first aspect.

[0046] According to a fourth aspect, the present application provides a computer-readable storage medium containing a computer program, which, when run on a computer, enables the computer to perform a method according to any one of the possible implementations of the first aspect.

[0047] According to a fifth aspect, the present application provides a computer program product, the computer program product including computer program code, which, when executed on a computer, enables the computer to perform a method according to any one of the implementation forms of the first aspect. [Brief explanation of the drawings]

[0048] [Figure 1] FIG. 1 is a diagram of a device-cloud collaborative content application scenario according to the present application. [Figure 2] 2 is a diagram of the structure of a device-cloud collaborative content generation device 200 according to the present application. [Figure 3] 3 is a diagram of the structure of a device-cloud collaborative content receiving device 300 according to the present application. [Figure 4] 4 is a flow diagram of a process 400 of a method for previewing device-cloud collaborative content according to the present application. [Figure 5a] FIG. 1 is a diagram of a device-cloud collaborative content editing and preview interface according to the present application. [Figure 5b] FIG. 1 is a diagram of a device-cloud collaborative content editing and preview interface according to the present application. [Figure 6a] FIG. 1 is a diagram of the game editor interface. [Figure 6b] FIG. 10 is a diagram of a device-cloud collaborative content editing and preview user interface. [Figure 6c] FIG. 10 is a diagram of a device-cloud collaborative content editing and preview user interface. [Figure 6d] FIG. 10 is a diagram of an editing interface in which pre-set parameters are mapped to user controls. [Figure 6e] FIG. 1 is a diagram of the game editor interface. [Figure 6f] FIG. 1 is a diagram of the game editor interface. [Figure 6g] FIG. 10 is a diagram of a device-cloud collaborative content preview interface. [Figure 6h] FIG. 10 is a diagram of a preview interface. [Figure 7] FIG. 1 is a diagram of a game editor. [Figure 8] A diagram of two game editors. [Figure 9] FIG. 10 is a diagram illustrating a simulation of device-cloud transmission in a storage / readout manner. [Figure 10] 1 is a diagram of the structure of an apparatus 1000 for previewing device-cloud collaborative content according to the present application. DETAILED DESCRIPTION OF THE INVENTION

[0049] In order to make the objectives, technical solutions and advantages of the present application clearer, the following will clearly and completely describe the technical solutions of the present application with reference to the accompanying drawings of the present application. It is obvious that the described embodiments are only a part, not all, of the embodiments of the present application. Any other embodiments obtained by those skilled in the art based on the embodiments of the present application without creative efforts shall fall within the protection scope of the present application.

[0050] In the specification, embodiments, claims, and accompanying drawings of this application, terms such as "first" and "second" are intended merely to distinguish and describe, and should not be understood as indicating or implying relative importance or an order. Also, the terms "include," "have," and any variations thereof are intended to cover non-exclusive inclusions, for example, including a series of steps or units. A method, system, product, or device is not necessarily limited to the steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such process, method, product, or device.

[0051] In this application, it should be understood that "at least one" refers to one or more, and "multiple" refers to two or more. The term "and / or" is used to describe an association relationship between related objects and indicates that three relationships may exist. For example, "A and / or B" can represent three cases: only A is present, only B is present, and both A and B are present, where A and B may be singular or plural. The character " / " typically indicates an "or" relationship between related objects. "At least one of the following" or similar expressions means any combination of these, including any combination of one or more of the following. For example, "at least one of a, b, or c" can refer to a, b, c, a and b, a and c, b and c, or a, b, and c, where a, b, and c may be singular or plural.

[0052] Terminology in this application: Device-Cloud Collaborative Content: Device-Cloud collaborative content is content that is primarily computed or calculated using high computing power on the cloud side and presented after the remaining calculations are completed on the device side, for example, device-cloud collaborative games.

[0053] Device-cloud collaboration data: Device-cloud collaboration data may also be referred to as device-cloud collaboration intermediate data, intermediate data, etc. The data is generated by the cloud side, transmitted to the device side for use, and used by the device side to complete content presentation.

[0054] Device-Cloud Transmission: After the device-cloud collaborative data is generated on the cloud side, the device-cloud transmission is the process of encoding on the cloud side, transmitting through a network or another medium, and decoding on the device side.

[0055] Device-Cloud Transmission Conditions: Device-Cloud transmission conditions are the conditions of a network or another medium used for device-cloud transmission, which affect the quality of the device-cloud transmission.

[0056] Game Content: Game Content is any templates, modules, components, functions, features, images, audio and video data, or other content used or operating in the creation and running of a game.

[0057] Presentation: Presentation includes actions by which a gaming device communicates game content to the user, such as image rendering, audio and video playback, mobile phone vibration, flashing LED indicators, and vibration of controls.

[0058] FIG. 1 is a diagram of an application scenario of device-cloud collaborative content according to the present application. As shown in FIG. 1, the device-cloud collaborative content executed in this scenario is, for example, the content of a device-cloud collaborative game, the content of a device-cloud collaborative document, or the content of a device-cloud collaborative live broadcast. The generating device of the device-cloud collaborative content (for short, the generating device, i.e., the cloud-side device (for short, the cloud side), including, for example, a computer or a server) uses the high computing power of the device-cloud collaborative content to complete the main calculation or operation and send device-cloud collaborative data (also referred to as intermediate data). The receiving device of the device-cloud collaborative data (for short, the receiving device, i.e., the device-side device (for short, the device side), including, for example, a mobile phone or tablet) receives the device-cloud collaborative data, completes the remaining calculation, and then presents the device-cloud collaborative content based on the device-cloud collaborative data.

[0059] This embodiment of the present application may be implemented by a device-cloud collaborative content generation device.

[0060] 2 is a diagram of the structure of a device-cloud collaborative content generation device 200 according to the present application. Note that the device-cloud collaborative content generation device 200 shown in FIG. 2 is merely an example, and the device-cloud collaborative content generation device 200 may have more or fewer components than those shown, a combination of two or more components, or a different arrangement of components. The various components shown in FIG. 2 may be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing circuits and / or application-specific integrated circuits. The device may be, for example, an electronic device having high computing power, such as a computer or a server.

[0061] The device-cloud collaborative content generation device includes at least one processor 211, at least one memory 212, at least one transceiver 213, at least one network interface 214, and at least one antenna 215. The processor 211, the memory 212, the transceiver 213, and the network interface 214 are connected, for example, through a bus. In an embodiment of the present application, the connection may include various interfaces, transmission lines, buses, or the like. This is not limited in the embodiment. The antenna 215 is connected to the transceiver 213. The network interface 214 is configured to allow the device-cloud collaborative content generation device to be connected to another device through a communication link.

[0062] The processor 211 is primarily configured to process cloud-side data in the device-cloud collaborative content, control the generating device of the entire device-cloud collaborative content, execute software programs, process data of the software programs, and is configured to support the generating device of the device-cloud collaborative content in performing the actions described in the embodiments, for example.

[0063] The memory 212 is mainly configured to store software programs and data. The memory 212 may exist independently or be connected to the processor 211. Optionally, the memory 212 may be integrated with the processor 211, for example, integrated into a chip. The memory 212 can store program codes for implementing the technical solutions of the embodiments of the present application, and the processor 211 controls the execution thereof. Various types of computer program codes that are executed may also be considered as driver programs for the processor 211.

[0064] 2 shows only one memory and one processor. In an actual device-cloud collaborative content generation device, there may be multiple processors and multiple memories. The memory may also be referred to as a storage medium, a storage device, etc. The memory may be a storage element on the same chip as the processor, i.e., an on-chip storage element, or may be an independent storage element. This is not a limitation in the embodiments of the present application.

[0065] The transceiver 213 may be configured to support signal reception or transmission between a device generating device for device-cloud collaborative content and a device receiving device for device-cloud collaborative content, and the transceiver 213 may be connected to an antenna 215. The transceiver 213 includes a transmitter Tx and a receiver Rx. Specifically, the one or more antennas 215 may receive signals. The receiver Rx of the transceiver 213 is configured to receive signals from the antenna, convert the signals into data, and provide the data to the processor 211, so that the processor 211 further processes the data. The transmitter Tx in the transceiver 213 is further configured to receive modulated data from the processor 211, convert the modulated data into a signal, and transmit the signal through the one or more antennas 215.

[0066] The transceiver 213 may also be referred to as a transceiver unit, a transceiver machine, a transceiver device, etc. Optionally, a device within the transceiver unit configured to implement a receiving function may be considered a receiving unit, and a device within the transceiver unit configured to implement a transmitting function may be considered a transmitting unit. In other words, the transceiver unit includes a receiving unit and a transmitting unit. The receiving unit may also be referred to as a receiver, an input port, a receiving circuit, etc. The transmitting unit may also be referred to as a transmitter, a transmitting machine, a transmitter circuit, etc.

[0067] It will be understood that the structure shown in this embodiment of the present application does not constitute a specific limitation on the device-cloud collaborative content generation device. In some other embodiments of the present application, the device-cloud collaborative content generation device may include more or fewer components than those shown in the figure, or a combination of some of the components, or a division of some of the components, or a different arrangement of the components. The components shown in the figure may be implemented by hardware, software, or a combination of software and hardware.

[0068] 3 is a diagram of the structure of a device-cloud collaborative content receiving device 300 according to the present application. It should be understood that the device-cloud collaborative content receiving device 300 shown in FIG. 3 is merely an example, and that the device-cloud collaborative content receiving device 300 may have more or fewer components than those shown, a combination of two or more components, or a different arrangement of components. The various components shown in FIG. 3 may be implemented in hardware, including one or more signal processing circuits and / or application-specific integrated circuits, software, or a combination of hardware and software. The device may be, for example, a user device such as a mobile phone or tablet.

[0069] The device-cloud collaborative content receiving device 300 may include a processor 310, an external memory interface 320, an internal memory 321, a universal serial bus (USB) interface 330, a charging management module 340, a power management module 341, a battery 342, an antenna 1, an antenna 2, a mobile communication module 350, a wireless communication module 360, an audio module 370, a speaker 370A, a receiver 370B, a microphone 370C, a headset jack 370D, a sensor module 380, a button 390, a motor 391, an indicator 392, a camera 393, a display 394, and a subscriber identification module (SIM) card interface 395, among others. The sensor module 380 may include a pressure sensor 380A, a gyro sensor 380B, an air pressure sensor 380C, a magnetic sensor 380D, an acceleration sensor 380E, a distance sensor 380F, an optical proximity sensor 380G, a fingerprint sensor 380H, a temperature sensor 380J, a touch sensor 380K, an ambient light sensor 380L, a bone conduction sensor 380M, and the like.

[0070] The processor 310 may include one or more processing units. For example, the processor 310 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). The different processing units may be independent devices or may be integrated into one or more processors.

[0071] The control unit may be the central and command center of the device-cloud collaborative content receiving device 300. The control unit may generate operation control signals based on the instruction operation code and the time series signal to complete the control of instruction fetching and instruction execution.

[0072] A memory may also be located in the processor 310 and configured to store instructions and data. In some embodiments, the memory in the processor 310 is a cache. The memory may store instructions or data that have just been used or that are periodically used by the processor 310. When the processor 310 needs to use the instructions or data again, the processor can retrieve the instructions or data directly from the memory. This avoids repeated accesses and reduces the latency of the processor 310 to improve system efficiency.

[0073] In some embodiments, the processor 310 may include one or more interfaces, which may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, a universal serial bus (USB) interface, etc.

[0074] The I2C interface is a bidirectional synchronous serial bus and includes one serial data line (SDA) and one serial clock line (SCL). In some embodiments, the processor 310 may include multiple groups of I2C buses. The processor 310 may be coupled to the touch sensor 380K, a charger, a flash, the camera 393, etc. through different I2C bus interfaces. For example, the processor 310 may be coupled to the touch sensor 380K through an I2C interface, whereby the processor 310 communicates with the touch sensor 380K through the I2C bus interface to implement touch functionality of the device-cloud collaborative content recipient device 300.

[0075] The I2S interface may be configured to implement audio communication. In some embodiments, the processor 310 may include multiple groups of I2S buses. The processor 310 may be coupled to the audio module 370 via the I2S bus to implement communication between the processor 310 and the audio module 370. In some embodiments, the audio module 370 may transmit audio signals to the wireless communication module 360 ​​via the I2S interface to implement a function for answering a phone call through a Bluetooth headset.

[0076] The PCM interface may also be configured to perform audio communications and sample, quantize, and encode analog signals. In some embodiments, audio module 370 may be coupled to wireless communications module 360 ​​through a PCM bus interface. In some embodiments, audio module 370 may alternatively transmit audio signals to wireless communications module 360 ​​through the PCM interface to implement functionality for answering phone calls through a Bluetooth® headset. Both the I2S interface and the PCM interface may be used for audio communications.

[0077] The UART interface is a universal serial data bus configured to implement asynchronous communication. The bus may be a bidirectional communication bus. The bus converts data to be transmitted between serial and parallel communication. In some embodiments, the UART interface is configured to normally connect the processor 310 to the wireless communication module 360. For example, the processor 310 communicates with a Bluetooth module in the wireless communication module 360 ​​through the UART interface to implement Bluetooth functionality. In some embodiments, the audio module 370 may transmit audio signals to the wireless communication module 360 ​​through the UART interface to implement a function for playing music through a Bluetooth headset.

[0078] The MIPI interface may be configured to connect the processor 310 to a peripheral device such as a display 394 or a camera 393. The MIPI interface includes a camera serial interface (CSI) and a display serial interface (DSI), among others. In some embodiments, the processor 310 communicates with the camera 393 through the CSI interface to implement a photo-taking function of the device-cloud collaborative content receiving device 300. The processor 310 communicates with the display 394 through the DSI interface to implement a display function of the device-cloud collaborative content receiving device 300.

[0079] The GPIO interface may be configured by software. The GPIO interface may be configured as a control signal or a data signal. In some embodiments, the GPIO interface may be configured to connect the processor 310 to the camera 393, the display 394, the wireless communication module 360, the audio module 370, the sensor module 380, etc. The GPIO interface may alternatively be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0080] The USB interface 330 is an interface that complies with the USB standard specification, and may specifically be a Mini USB interface, a Micro USB interface, a USB Type-C interface, etc. The USB interface 330 may be configured to connect a charger to charge the device-cloud collaborative content receiving device 300, to transmit data between the device-cloud collaborative content receiving device 300 and a peripheral device, or to connect to a headset to play audio through the headset. Alternatively, the interface may be configured to connect to another user equipment, for example, an AR device.

[0081] It will be understood that the interface connection relationships between modules illustrated in the embodiments of the present application are merely examples for the purpose of explanation and do not constitute limitations on the structure of the device-cloud collaborative content receiving device 300. In some other embodiments of the present application, the device-cloud collaborative content receiving device 300 may alternatively use an interface connection method different from the aforementioned embodiments, or may use a combination of multiple interface connection methods.

[0082] The charging management module 340 is configured to receive charging input from a charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 340 may receive the charging input of the wired charger through the USB interface 330. In some wireless charging embodiments, the charging management module 340 may receive the wireless charging input through a wireless charging coil of the device-cloud collaborative content receiving device 300. The charging management module 340 may charge the battery 342 while also providing power to the user equipment through the power management module 341.

[0083] The power management module 341 is configured to connect the battery 342, the charging management module 340, and the processor 310. The power management module 341 receives input from the battery 342 and / or the charging management module 340 and provides power to the processor 310, the internal memory 321, the external memory, the display 394, the camera 393, the wireless communication module 360, etc. The power management module 341 may be further configured to monitor parameters such as battery capacity, battery cycle count, and battery health status (fault or impedance). In some other embodiments, the power management module 341 may alternatively be located in the processor 310. In some other embodiments, the power management module 341 and the charging management module 340 may instead be located in the same device.

[0084] The wireless communication function of the device-cloud collaborative content receiving device 300 may be implemented through antenna 1, antenna 2, mobile communication module 350, wireless communication module 360, modem processor, baseband processor, etc.

[0085] Antenna 1 and Antenna 2 are configured to transmit and receive electromagnetic signals. Each antenna in the device-cloud collaborative content receiving device 300 may be configured to cover one or more communication frequency bands. Different antennas may be further multiplexed to improve antenna utilization. For example, Antenna 1 may be multiplexed as a diversity antenna in a wireless local area network. In some other embodiments, the antennas may be used in combination with tuning switches.

[0086] The mobile communication module 350 may be applied to the device-cloud collaborative content receiving device 300 and may provide a wireless communication solution for use in 2G, 3G, 4G, 5G, etc. The mobile communication module 350 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 350 may receive electromagnetic waves through the antenna 1, perform processing such as filtering or amplification on the received electromagnetic waves, and transmit the electromagnetic waves to the modem processor for demodulation. The mobile communication module 350 may further amplify signals modulated by the modem processor and convert the signals into electromagnetic waves for emission through the antenna 1. In some embodiments, at least some functional modules in the mobile communication module 350 may be located within the processor 310. In some embodiments, at least some functional modules in the mobile communication module 350 may be located within the same device as at least some modules of the processor 310.

[0087] The modem processor may include a modulator and a demodulator. The modulator is configured to modulate a low-frequency baseband signal to be transmitted into a mid- to high-frequency signal. The demodulator is configured to demodulate a received electromagnetic signal into a low-frequency baseband signal. The demodulator then transmits the low-frequency baseband signal obtained through demodulation to the baseband processor for processing. The low-frequency baseband signal is processed by the baseband processor and then transmitted to the application processor. The application processor outputs an audio signal through an audio device (such as, but not limited to, speaker 370A or receiver 370B) or displays an image or video through display 394. In some embodiments, the modem processor may be an independent device. In some other embodiments, the modem processor may be independent of the processor 310 and may be located in the same component as the mobile communication module 350 or another functional module.

[0088] The wireless communication module 360 ​​is applied to the device-cloud collaborative content receiving device 300 and may provide wireless communication solutions, including a wireless local area network (WLAN) (e.g., a wireless fidelity (Wi-Fi) network), Bluetooth (BT), a global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC) technology, infrared (IR) technology, etc. The wireless communication module 360 ​​may be one or more devices integrating at least one communication processor module. The wireless communication module 360 ​​receives electromagnetic waves through the antenna 2, performs frequency modulation and filtering processing on the electromagnetic wave signal, and transmits the processed signal to the processor 310. The wireless communication module 360 ​​may further receive a signal to be transmitted from the processor 310, perform frequency modulation and amplification on the signal, and convert the signal into an electromagnetic wave for radiation via the antenna 2.

[0089] In some embodiments, antenna 1 and mobile communication module 350 in device-cloud collaborative content receiving device 300 are coupled, and antenna 2 and wireless communication module 360 ​​in device-cloud collaborative content receiving device 300 are coupled, allowing device-cloud collaborative content receiving device 300 to communicate with a network and another device by using a wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, IR technology, etc. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a BeiDou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS), and / or a geostationary satellite-based augmentation system (SBAS).

[0090] The device-cloud collaborative content receiving device 300 implements display functionality by using a GPU, a display 394, an application processor, etc. The GPU is a microprocessor for image processing and is connected to the display 394 and the application processor. The GPU is configured to perform mathematical and geometric calculations and to render graphics. The processor 310 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0091] The display 394 is configured to display images, videos, etc. The display 394 includes a display panel. The display panel may be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a mini-LED, a micro-LED, a micro-OLED, a quantum dot light-emitting diodes (QLED), etc. In some embodiments, the device-cloud collaborative content receiving device 300 may include one or N displays 394, where N is a positive integer greater than 1.

[0092] The device-cloud collaborative content receiving device 300 may implement image capture functionality through a camera 393, an ISP, a video codec, a GPU, a display 394, an application processor, and the like.

[0093] The ISP is configured to process data fed back by the camera 393. For example, during photography, the shutter is pressed and light is transmitted through the lens to the camera's photosensitive elements. The light signal is converted into an electrical signal, which the camera's photosensitive elements transmit to the ISP for processing to convert the electrical signal into a visible image. The ISP may further perform algorithmic optimization for image noise, brightness, and appearance. The ISP may further optimize parameters such as exposure and color temperature of the photographed scene. In some embodiments, the ISP may be located in the camera 393.

[0094] The camera 393 is configured to capture still images or video. An optical image of an object is generated through a lens and projected onto a photosensitive element. The photosensitive element may be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, which is then transmitted to an ISP for conversion into a digital image signal. The ISP outputs the digital image signal to a DSP for processing. The DSP converts the digital image signal into an image signal in a standard format, such as RGB or YUV. In some embodiments, the device-cloud collaborative content receiving device 300 may include one or N cameras 393, where N is a positive integer greater than 1.

[0095] The digital signal processor is configured to process digital signals and may further process other digital signals in addition to digital image signals. For example, when the device-cloud collaborative content receiving device 300 selects a frequency, the digital signal processor is configured to perform a Fourier transform on the frequency energy.

[0096] A video codec is configured to compress or decompress digital video. A device-cloud collaborative content receiving device 300 may support one or more video codecs. Thus, a device-cloud collaborative content receiving device 300 may play or record video in multiple coding formats, such as moving picture experts group (MPEG)-1, MPEG-2, MPEG-3, and MPEG-4.

[0097] The NPU is a neural network (NN) processor that can rapidly process input information and continuously self-learn by referencing the structure of biological neural networks, for example, the communication method between human brain neurons. Applications such as intelligent recognition of the device 300 receiving device-cloud collaborative content may be implemented through the NPU, for example, image recognition, face recognition, speech recognition, and text understanding.

[0098] The external memory interface 320 may be configured to connect to an external memory card, such as a Micro SD card, to expand the storage capabilities of the device-cloud collaborative content recipient device 300. The external memory card communicates with the processor 310 through the external memory interface 320 to implement data storage functions. For example, files such as music and videos are stored on the external memory card.

[0099] The internal memory 321 may be configured to store computer-executable program code, where the executable program code includes instructions. The processor 310 executes the instructions stored in the internal memory 321 to implement various functional applications and data processing of the device-cloud collaborative content receiving device 300. The internal memory 321 may include a program storage area and a data storage area. The program storage area may store an operating system, applications required by at least one function (e.g., audio playback function or image playback function), etc. The data storage area may store data created during use of the device-cloud collaborative content receiving device 300 (e.g., audio data, phone book, etc.). The internal memory 321 may also include high-speed random access memory or non-volatile memory, such as at least one magnetic disk storage device, flash storage device, and universal flash storage (UFS).

[0100] The device-cloud collaborative content receiving device 300 may implement audio functions, such as music playback and recording, by using an audio module 370, a speaker 370A, a receiver 370B, a microphone 370C, a headset jack 370D, an application processor, etc.

[0101] The audio module 370 is configured to convert digital audio information into an analog audio signal output, and is further configured to convert an analog audio input into a digital audio signal. The audio module 370 may be further configured to encode and decode the audio signal. In some embodiments, the audio module 370 may be located in the processor 310, or some functional modules of the audio module 370 may be located in the processor 310.

[0102] The speaker 370A, also referred to as a "loudspeaker," is configured to convert audio electrical signals into sound signals. The device-cloud collaborative content receiving device 300 may use the speaker 370A to listen to music or take hands-free calls.

[0103] Receiver 370B, also referred to as an "earpiece," is configured to convert an audio electrical signal into a voice signal. When a call is answered or voice information is received by using device-cloud collaborative content receiving device 300, receiver 370B may be held close to a person's ear to listen to the sound.

[0104] The microphone 370C, also referred to as a "mike" or "mic," is configured to convert audio signals into electrical signals. When making a phone call or transmitting audio information, a user may speak near the microphone 370C through the user's mouth to input the audio signal into the microphone 370C. At least one microphone 370C may be disposed in the device-cloud collaborative content receiving device 300. In some other embodiments, two microphones 370C may be disposed in the device-cloud collaborative content receiving device 300 to collect audio signals and implement noise reduction functions. In some other embodiments, three, four, or more microphones 370C may alternatively be disposed in the device-cloud collaborative content receiving device 300 to collect audio signals, implement noise reduction, identify sound sources, implement directional recording functions, etc.

[0105] The headset jack 370D is configured to connect to a wired headset and may be a USB interface 330, a 3.5 mm open mobile terminal platform (OMTP) standard interface, or a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0106] The pressure sensor 380A may be configured to sense a pressure signal and convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 380A may be disposed on the display 394. There are multiple types of pressure sensors 380A, such as resistive pressure sensors, inductive pressure sensors, and capacitive pressure sensors. A capacitive pressure sensor may include at least two parallel plates made of a conductive material. When a force is applied to the pressure sensor 380A, the capacitance between the electrodes changes. The device-cloud collaboration content receiving device 300 determines the pressure intensity based on the capacitance change. When a touch operation is performed on the display 394, the device-cloud collaboration content receiving device 300 detects the intensity of the touch operation through the pressure sensor 380A. The device-cloud collaboration content receiving device 300 may also calculate the touch position based on the detection signal of the pressure sensor 380A. In some embodiments, touch operations performed at the same touch position but with different touch operation intensities may correspond to different operation commands. For example, when a touch operation with a touch operation strength less than a first pressure threshold is performed on the SMS message application icon, instructions for viewing an SMS message are executed, and when a touch operation with a touch operation strength equal to or greater than the first pressure threshold is performed on the SMS message application icon, instructions for creating an SMS message are executed.

[0107] The gyro sensor 380B may be configured to determine the motion orientation of the device-cloud collaborative content receiving device 300. In some embodiments, the gyro sensor 380B may be used to determine the angular velocity of the device-cloud collaborative content receiving device 300 around three axes (i.e., axes x, y, and z). The gyro sensor 380B may be configured to achieve image stabilization when taking a photo. For example, when the shutter is pressed, the gyro sensor 380B detects the angle of jitter of the device-cloud collaborative content receiving device 300 and, based on the angle, calculates the distance the lens module needs to correct, allowing the lens to cancel the jitter of the device-cloud collaborative content receiving device 300 through reverse motion to achieve image stabilization. The gyro sensor 380B may also be used in navigation scenarios and motion-controlled game scenarios.

[0108] The barometric pressure sensor 380C is configured to measure barometric pressure. In some embodiments, the device-cloud collaborative content receiving device 300 calculates altitude by using the barometric pressure value measured by the barometric pressure sensor 380C to assist in positioning and navigation.

[0109] The magnetic sensor 380D includes a Hall effect sensor. The device-cloud collaborative content receiving device 300 may detect the opening and closing of the flip cover by using the magnetic sensor 380D. In some embodiments, when the device-cloud collaborative content receiving device 300 is a flip phone, the device-cloud collaborative content receiving device 300 may detect the opening and closing of the flip cover based on the magnetic sensor 380D. Furthermore, a function such as automatic unlocking of the flip cover may be set based on the detected open and closed states of the leather cover or the detected open and closed states of the flip cover.

[0110] The acceleration sensor 380E may detect the magnitude of acceleration of the device-cloud collaborative content receiving device 300 in various directions (typically along three axes). The magnitude and direction of gravity may be detected when the device-cloud collaborative content receiving device 300 is stationary. The acceleration sensor 380E may be further configured to identify the orientation of the user equipment, for use in applications such as switching between landscape and portrait modes or as a pedometer.

[0111] The distance sensor 380F is configured to measure distance. The device-cloud collaborative content receiving device 300 may measure distance using an infrared method or a laser method. In some embodiments, in a photography scene, the device-cloud collaborative content receiving device 300 may measure distance by using the distance sensor 380F to achieve quick focusing.

[0112] The optical proximity sensor 380G may include, for example, a light-emitting diode (LED) and a photodetector such as a photodiode. The light-emitting diode may be an infrared light-emitting diode. The device-cloud collaborative content receiving device 300 emits infrared light by using the light-emitting diode. The device-cloud collaborative content receiving device 300 detects infrared reflected light from nearby objects by using the photodiode. When sufficient reflected light is detected, it may be determined that an object is present in the vicinity of the device-cloud collaborative content receiving device 300. When insufficient reflected light is detected, the device-cloud collaborative content receiving device 300 may determine that no object is present in the vicinity of the device-cloud collaborative content receiving device 300. The device-cloud collaborative content receiving device 300 may use the optical proximity sensor 380G to detect that a user is holding the device-cloud collaborative content receiving device 300 close to their ear for a phone call to automatically turn off the screen to save power. The optical proximity sensor 380G may also be used in smart cover mode or pocket mode to automatically unlock or lock the screen.

[0113] The ambient light sensor 380L is configured to sense the brightness of ambient light. The device-cloud collaborative content receiving device 300 may adaptively adjust the brightness of the display 394 based on the sensed ambient light brightness. The ambient light sensor 380L may also be configured to automatically adjust the white balance when taking a photo. The ambient light sensor 380L may further cooperate with the optical proximity sensor 380G to detect whether the device-cloud collaborative content receiving device 300 is in a pocket to avoid accidental contact.

[0114] The fingerprint sensor 380H is configured to collect fingerprints, and the device-cloud collaborative content receiving device 300 may use the collected fingerprint characteristics to implement fingerprint-based unlocking, application lock access, fingerprint-based photo taking, fingerprint-based phone answering, etc.

[0115] The temperature sensor 380J is configured to detect temperature. In some embodiments, the device-cloud collaborative content receiving device 300 performs a temperature processing method based on the temperature detected by the temperature sensor 380J. For example, when the temperature reported by the temperature sensor 380J exceeds a threshold, the device-cloud collaborative content receiving device 300 reduces the performance of a processor near the temperature sensor 380J to reduce power consumption for thermal protection. In some other embodiments, when the temperature is lower than another threshold, the device-cloud collaborative content receiving device 300 heats the battery 342 to avoid abnormal shutdown of the device-cloud collaborative content receiving device 300 due to low temperature. In some other embodiments, when the temperature is below yet another threshold, the device-cloud collaborative content receiving device 300 boosts the output voltage of the battery 342 to avoid abnormal shutdown due to low temperature.

[0116] The touch sensor 380K may also be referred to as a "touch panel." The touch sensor 380K may be disposed on the display 394, with the touch sensor 380K and the display 394 forming a touch screen, also referred to as a "touch screen." The touch sensor 380K is configured to detect a touch action on or near the touch sensor 380K. The touch sensor may communicate the detected touch action to an application processor to determine the type of touch event. A visual output associated with the touch action may be provided on the display 394. In some other embodiments, the touch sensor 380K may alternatively be disposed on the surface of the device-cloud collaborative content recipient device 300 at a location different from the location of the display 394.

[0117] The bone conduction sensor 380M may acquire vibration signals. In some embodiments, the bone conduction sensor 380M may acquire vibration signals of the vibrating bones of the human vocal cords. The bone conduction sensor 380M may also contact the human's pulse to receive blood pressure pulsation signals. In some embodiments, the bone conduction sensor 380M may alternatively be disposed within a headset to acquire a bone conduction headset. The audio module 370 may acquire audio signals through analysis based on the vibration signals of the vibrating bones of the vocal cords and acquired by the bone conduction sensor 380M to implement audio functions. The application processor may analyze heartbeat information based on the blood pressure pulsation signals acquired by the bone conduction sensor 380M to implement heartbeat detection functions.

[0118] The buttons 390 include a power button, a volume button, etc. The buttons 390 may be mechanical buttons or touch buttons. The device-cloud collaborative content receiving device 300 may receive the button input and generate a button signal input related to user settings and function control of the device-cloud collaborative content receiving device 300.

[0119] The motor 391 may generate a vibration prompt. The motor 391 may be configured to provide an incoming vibration prompt and touch vibration feedback. For example, touch actions performed in different applications (e.g., taking pictures and playing audio) may correspond to different vibration feedback effects. For touch actions performed in different areas of the display 394, the motor 391 may also correspond to different vibration feedback effects. Also, different application scenarios (e.g., time reminder, information reception, alarm clock, game) may correspond to different vibration feedback effects. The touch vibration feedback effects may be further customized.

[0120] The indicator 392 may be an indicator light and may be configured to indicate charging status and power changes, or may be configured to indicate messages, missed calls, notifications, etc.

[0121] The SIM card interface 395 is configured to connect to a SIM card. The SIM card may be inserted into or removed from the SIM card interface 395 to connect or disconnect the device-cloud collaborative content receiving device 300. The device-cloud collaborative content receiving device 300 may support one or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 395 may support a nano SIM card, a micro SIM card, a SIM card, etc. Multiple cards may be inserted into the same SIM card interface 395 at the same time. The multiple cards may be the same type or different types. The SIM card interface 395 is compatible with different types of SIM cards. The SIM card interface 395 also accommodates an external memory card. The device-cloud collaborative content receiving device 300 interacts with a network by using a SIM card to implement functions such as conversation and data communication. In some embodiments, the device-cloud collaborative content receiving device 300 uses an eSIM, i.e., an embedded SIM card, which may be built into the device-cloud collaborative content receiving device 300 and cannot be separated from the device-cloud collaborative content receiving device 300.

[0122] It can be understood that the structures shown in the embodiments of the present application do not constitute specific limitations on the device on the device side. In some other embodiments of the present application, the device on the device side may include more or fewer components than those shown in the figures, or a combination of some of the components, or a division of some of the components, or a different arrangement of the components. The components shown in the figures may be implemented by hardware, software, or a combination of software and hardware.

[0123] The above embodiments describe a device-cloud collaborative content execution scenario. Editing the device-cloud collaborative content before it is executed is one of the most important stages in device-cloud collaborative content development. This application provides a method for previewing device-cloud collaborative content, which simplifies the editing process, helps users improve editing efficiency, and reduces labor intensity.

[0124] Based on this, Figure 4 is a flow diagram of a process 400 of a method for previewing device-cloud collaborative content according to the present application. As shown in Figure 4, the process 400 may be performed by a device-cloud collaborative content generation device (which may be referred to as a generation device for short) to implement device-cloud collaborative content editing and preview. The process 400 is described as a series of steps or operations. It should be understood that the process 400 is not limited to the order of execution shown in Figure 4, and may be performed in various orders and / or simultaneously. The process 400 includes the following steps:

[0125] Step 401: Display an editing interface, where the editing interface includes a preview control and a plurality of setting controls.

[0126] A number of configuration controls are used to set parameters associated with device-cloud collaborative content. In this embodiment of the present application, the parameters associated with device-cloud collaborative content include:

[0127] (1) Device-cloud collaborative content receiving device parameters The parameters are used to simulate the execution environment of the receiving device, and may include, for example, but not limited to, the type of receiving device, the model of the receiving device, the chip model of the receiving device, the battery level of the receiving device, the resolution of the receiving device, the volume of the receiving device, and the ambient temperature of the receiving device. The receiving device may be simulated on the generating device based on the aforementioned parameters, so that even if an actual receiving device is not used, a virtual receiving device may be built on the generating device based on the attributes of the receiving device. In this manner, the generating device may edit and generate the device-cloud collaborative content and further simulate the receiving device to present the edited and generated device-cloud collaborative content to the user, so that the user can quickly and intuitively preview the editing effect.

[0128] (2) Coding parameters The coding parameters are used to perform an encoding process on the device-cloud collaborative content to obtain a bitstream, or to perform a decoding process on the bitstream to reconstruct the device-cloud collaborative content. For example, the coding parameters may include, but are not limited to, video encoder / decoder parameters and audio encoder / decoder parameters. For device-cloud collaborative content, see the video / audio encoding / decoding method. Therefore, multiple encoding / decoding parameters are involved. The generating device may encode the device-cloud collaborative content based on the encoding parameters to obtain a bitstream, and the receiving device may decode the bitstream based on the coding parameters to obtain the reconstructed device-cloud collaborative content.

[0129] (3) Transmission parameters The transmission parameters are used to simulate the conditions and environments of bitstream transmission, and may include, but are not limited to, the transmission / reception frame rate, the maximum transmission / reception bit rate, the network delay, the network instability, the physical distance, the physical location, and the transmission channel. The bitstream obtained by encoding the device-cloud collaborative content needs to be transmitted from the generating device to the receiving device. To adapt to the conditions of the network or another transmission medium, the parameters transmitted by the generating device and received by the receiving device may be configured based on the aforementioned parameters. In addition, the conditions encountered when the bitstream is transmitted from the generating device to the receiving device, or within the network or another transmission medium, are simulated to account for the changes caused by the aforementioned conditions when the bitstream arrives at the receiving device.

[0130] (4) Instance attribute parameters of device-cloud collaborative content The instance attribute parameters of the device-cloud collaborative content are used to generate the device-cloud collaborative content and may include, for example, but are not limited to, the position, ratio, rotation, special effect intensity, and weight of the instance of the device-cloud collaborative content. These parameters are associated with the rendering, image quality, etc. of the device-cloud collaborative content and may have the most direct impact on the presentation effect of the receiving device. Changing one or more parameter values ​​may result in a change in the presentation effect of the receiving device.

[0131] In this embodiment of the present application, a plurality of setting controls included in the editing interface correspond to a plurality of parameters, and a user may change the parameter values ​​corresponding to the setting controls by operating the setting controls.

[0132] In a possible implementation, the editing interface may include multiple windows.

[0133] The first window may correspond to a parameter of a receiving device of the device-cloud collaborative content. The first window may include a plurality of controls, each of which corresponds to a parameter of the receiving device. For example, each of the controls may be used to set a type of the receiving device, a model of the receiving device, a chip model of the receiving device, a battery level of the receiving device, a resolution of the receiving device, a volume of the receiving device, and an ambient temperature of the receiving device.

[0134] The second window may correspond to a coding parameter. The second window may include a plurality of controls, the plurality of controls corresponding to the coding parameters. For example, the plurality of controls may be used to set a video encoder / decoder parameter and an audio encoder / decoder parameter, respectively.

[0135] The third window may correspond to a transmission parameter. The third window may include a plurality of controls, where the plurality of controls correspond to a transmission parameter. For example, the plurality of controls may be used to set a transmission / reception frame rate, a maximum transmission / reception bit rate, a network delay, a network instability, a physical distance, a physical location, and a transmission channel, respectively.

[0136] The fourth window may correspond to instance attribute parameters of the device-cloud collaborative content. The fourth window may include a plurality of controls, and the plurality of controls correspond to instance attribute parameters of the device-cloud collaborative content. For example, the plurality of controls may be used to set the position, ratio, rotation, special effect intensity, weight, etc. of the instance of the device-cloud collaborative content.

[0137] It should be noted that the four aforementioned windows do not constitute a limitation on the editing interface. The editing interface in this application may include fewer or more windows. This is related to the editing requirements of device-cloud collaborative content. The number of windows and content is not particularly limited in this application, nor are the controls included in each window particularly limited.

[0138] In a possible implementation, the editing interface may include multiple controls.

[0139] The first control may correspond to a parameter of a receiving device of the device-cloud collaborative content. The first control may include multiple sub-controls, where the multiple sub-controls correspond to parameters of the receiving device. For example, the multiple sub-controls are used to set a receiving device type, a receiving device model, a receiving device chip model, a receiving device battery level, a receiving device resolution, a receiving device volume, and an ambient temperature of the receiving device, respectively.

[0140] The second control may correspond to a coding parameter. The second control may include multiple sub-controls, each of which corresponds to a coding parameter. For example, the multiple sub-controls may be used to set a video encoder / decoder parameter and an audio encoder / decoder parameter, respectively.

[0141] The third control may correspond to a transmission parameter. The third control may include multiple sub-controls, where the multiple sub-controls correspond to the transmission parameters. For example, the multiple sub-controls are used to set a transmission / reception frame rate, a maximum transmission / reception bit rate, a network delay, a network instability, a physical distance, a physical location, and a transmission channel, respectively.

[0142] The fourth control may correspond to an instance attribute parameter of the device-cloud collaborative content. The fourth control may include multiple sub-controls, and the multiple sub-controls correspond to instance attribute parameters of the device-cloud collaborative content. For example, each of the multiple sub-controls is used to set the position, ratio, rotation, special effect intensity, weight, etc. of the instance of the device-cloud collaborative content.

[0143] It should be noted that the above four controls do not constitute limitations on the editing interface. The editing interface in this application may include fewer or more controls, depending on the editing requirements of device-cloud collaborative content. The number and content of the controls are not particularly limited in this application, nor are the sub-controls included in each control particularly limited.

[0144] The preview control may be used to trigger the presentation of the edited device-cloud collaborative content. The preview control may be presented in each window or control, although this is not specifically limited.

[0145] Step 402: In response to a trigger action on the preview control, display a preview interface based on parameters of the receiving device.

[0146] The preview interface is used to simulate an interface for displaying the device-cloud collaborative content on a receiving device.

[0147] As described above, the editing interface displays multiple setting controls. Based on this, a user may perform actions on the multiple setting controls in the editing interface, such as tapping a control, entering text / numbers, or dragging a slider, to modify the values ​​of parameters corresponding to the setting controls. After the user changes the values ​​of one or more parameters, the user may timely preview the impact and effects generated by modifying the device-cloud collaborative content, i.e., trigger a preview control in the editing interface.

[0148] In this embodiment of the present application, the generating device may first acquire the device-cloud collaborative content, simulate transmission conditions based on transmission parameters to simulate the device-cloud collaborative content received on the receiving device, simulate a receiving device based on parameters of the receiving device of the device-cloud collaborative content to simulate the device-cloud collaborative content acquired by the receiving device, process the received device-cloud collaborative content, and further display the device-cloud collaborative content on a preview interface.

[0149] Obtaining the device-cloud collaborative content may include pre-editing the content of the device-cloud collaborative content based on the requirements of the device-cloud collaborative content. This operation does not involve transmission or presentation. Therefore, the device-cloud collaborative content may be pre-edited and then stored. Subsequent testing may be performed only on the performance of the transmitting and receiving devices. Alternatively, the device-cloud collaborative content may be obtained from a pre-set file. That is, the user pre-edits the content of the device-cloud collaborative content based on the requirements of the device-cloud collaborative content and stores the content in the form of a file, or generates the device-cloud collaborative content based on the instance attribute parameters of the device-cloud collaborative content. In other words, in the editing process, the user may edit the content while testing the performance of the transmitting and receiving devices.

[0150] As described above, based on the transmission parameters, the conditions encountered when the bitstream is transmitted from the generating device to the receiving device, or within a network or over another medium, may be simulated to simulate the changes caused by the influence of the aforementioned conditions when the bitstream reaches the receiving device. Thus, even if the generating device does not actually transmit the bitstream, the generating device may further construct a virtual transmission path to simulate the data received by the receiving device after the bitstream is transmitted over the virtual transmission path.

[0151] The generating device may obtain a receiving device through simulation based on parameters of the receiving device of the device-cloud collaborative content, such that even if an actual receiving device is not used, a virtual receiving device may be built on the generating device based on attributes of the receiving device. Thus, the generating device may simulate a receiving device that processes the received data to obtain the device-cloud collaborative content on the receiving device.

[0152] In a possible implementation, the generating device obtains the device-cloud collaborative content, encodes the device-cloud collaborative content based on coding parameters to obtain a bitstream, transmits the bitstream based on transmission parameters, simulates a receiving device receiving the bitstream based on parameters of the receiving device of the device-cloud collaborative content, decodes the bitstream based on the coding parameters to simulate the device-cloud collaborative content received on the receiving device, and obtains a display of the receiving device obtained through the simulation, whereby the device-cloud collaborative content is displayed in a preview interface.

[0153] In the above process, the complete data processing process from the generating device to the receiving device can be simulated in detail. Specifically, the generating device obtains the device-cloud collaborative content based on the instance attribute parameters of the device-cloud collaborative content, encodes the device-cloud collaborative content based on the coding parameters to obtain a bitstream, and then transmits the bitstream based on the transmission parameters. In the bitstream transmission process, the generating device simulates the transmission environment as if the bitstream were transmitted in a real environment. The receiving device (obtained by the generating device through simulation based on the parameters of the receiving device) receives the bitstream, decodes the bitstream based on the coding parameters, the attributes of the receiving device, and the capabilities of the receiving device, and reconstructs the device-cloud collaborative content based on the instance attribute parameters of the device-cloud collaborative content.

[0154] The generating device simulates a virtual receiving device based on the parameters of the receiving device, and displays the content pictures acquired under the aforementioned parameter configurations to the user in the form of a preview interface, so that the user can quickly and intuitively see the effects that may be presented on the receiving device under the aforementioned configured parameters, in order to modify the corresponding parameters in time based on requirements.

[0155] 5a and 5b are diagrams of a device-cloud collaborative content editing and preview interface according to the present application. As shown in FIG. 5a, game engine editing is used as an example in this embodiment. The device-cloud collaborative content may be device-cloud collaborative content in a game. The game engine editor window (left window) includes a device-cloud collaborative content editing and preview user interface control. After a user triggers the control, the device-cloud collaborative content editing and preview user interface (right window, i.e., the aforementioned editing interface) is displayed. The interface includes a device-cloud collaborative content editing interface and a device-cloud collaborative content preview interface. The device-cloud collaborative content editing interface includes user interfaces for multiple device-cloud contents. The user interface for device-cloud content 1 is used as an example and includes controls for instance attribute parameters, coding parameters, and transmission parameters of the device-cloud collaborative content. The user interfaces for device-cloud contents 2 to N may or may not be exactly the same as the user interface for device-cloud content 1. This depends specifically on the requirements of the device-cloud content. This is not specifically limited. The device-cloud collaborative content preview interface includes controls within the simulated transmission user interface and the simulated device-side user interface (i.e., the preview controls described above). A user may trigger the controls to display the corresponding preview interfaces.

[0156] As shown in FIG. 5b, editing a game engine is used as an example of this embodiment. The device-cloud collaborative content may be device-cloud collaborative content in a game. The game engine editor window (left window) is displayed in a tree structure. The game engine editor window includes a device-cloud collaborative content editing and preview user interface option. After a user triggers the selection, a device-cloud collaborative content editing and preview user interface (right window, i.e., the aforementioned editing interface) is displayed. The interface includes a device-cloud collaborative content editing interface and a device-cloud collaborative content preview interface. The device-cloud collaborative content editing interface includes multiple device-cloud content options. The device-cloud content 1 option is used as an example and includes options for instance attribute parameters, coding parameters, and transmission parameters of the device-cloud collaborative content. The device-cloud content 2 to N options may or may not be exactly the same as the device-cloud content 1 option. This specifically depends on the requirements of the device-cloud content. This is not specifically limited. The device-cloud collaborative content preview interface includes options (i.e., the preview controls described above) in the simulated transmission user interface and the simulated device-side user interface. A user may trigger the controls to display the corresponding preview interface.

[0157] It should be noted that Figures 5a and 5b are merely two examples of device-cloud collaborative content editing and preview interfaces and do not constitute any limitation on the Graphical User Interface (GUI) layout of said interfaces, which is also not specifically limited in the embodiments of the present application.

[0158] In this application, the generating device obtains the device-cloud collaborative content through editing based on multiple parameters configured by the user in the editing interface, simulates the transmission process of the device-cloud collaborative content, simulates the receiving device to process the received bitstream to obtain the device-cloud collaborative content in the receiving device, and displays the content picture under the above configuration in the preview interface, which helps the user quickly understand the editing effect. The above process is repeated multiple times to obtain a final satisfactory effect and corresponding parameter configuration.

[0159] It is found that in this application, the generating device completes the entire editing process of device-cloud collaborative content by editing and simulating the receiving device, which allows the user to autonomously modify the parameter configuration and quickly obtain feedback, thereby simplifying the editing process, helping the user improve editing efficiency, and reducing labor intensity.

[0160] In the following, a specific embodiment is used to describe the technical solutions of the present application in detail.

[0161] Embodiment 1 This embodiment is applied to a scenario involving device-cloud collaborative content, which may include a production service scenario of a series of device-cloud collaborative technologies, such as device-cloud collaborative Computer Graphics Motion Vector (CGMV), device-cloud collaborative Dynamic Diffusion Global Illumination (DDGI), device-cloud collaborative Image-Based Lighting (IBL), device-cloud collaborative shadow mapping, and device-cloud collaborative object understanding and computation. Optionally, the method of this embodiment may be applied to an editor for a device-cloud collaborative game and deployed on the cloud side. The editor may display an editing interface and a preview interface. The editing interface includes a content editing interface and a simulation interface. The content editing interface may be used to edit instance attribute parameters, coding parameters, and transmission parameters of the device-cloud collaborative content, as well as parameters of a receiving device of the device-cloud collaborative content. The preview interface is used to present pictures of the device-cloud collaborative game.

[0162] The device-cloud collaboration techniques may be applied to one or more of game systems such as rendering, physics simulation, and skeletal animation. These device-cloud collaboration techniques may have one instance, several instances, or no instances in a game scenario. In this embodiment, the device side is an electronic device, such as a computer, a mobile phone, or a tablet. In this embodiment, network transmission is used.

[0163] The procedure of this embodiment is as follows: 1. Displaying entry to device-cloud collaborative content editing and preview user interface within the game editor interface.

[0164] For example, as shown in FIG. 6a (FIG. 6a is an illustration of a game editor interface), an entry to the device-cloud collaborative content editing and preview user interface is displayed in the game editor interface.

[0165] 2. Displaying a device-cloud collaborative content editing and preview user interface in response to a user triggering action for entry into the device-cloud collaborative content editing and preview user interface.

[0166] For example, as shown in FIG. 6b (FIG. 6b is a diagram of a device-cloud collaborative content editing and preview user interface), the device-cloud collaborative content editing and preview user interface includes a device-cloud collaborative content editing interface and a device-cloud collaborative content preview interface.

[0167] (a) The device-cloud collaborative content editing interface is i. a device-cloud CGMV editing user interface; ii. a device-cloud shadow mask edit user interface, which may further include a device-cloud shadow mask instance 1 user interface and a device-cloud shadow mask instance 2 user interface; iii. A device-cloud IBL editing user interface.

[0168] (b) The device-cloud collaborative content preview interface is i. a device-cloud co-simulation transmission user interface; ii. A device-cloud co-simulation device-side user interface.

[0169] For example, as shown in FIG. 6c (which is a diagram of a device-cloud collaborative content editing and preview user interface), the device-cloud collaborative content editing interface includes a device-cloud streaming editing user interface and a device-cloud shadow mask editing user interface, each of which includes associated settings of instance attribute parameters, coding parameters, and transmission parameters of the device-cloud collaborative content. The device-cloud collaborative content preview interface includes a device-cloud co-simulation transmission user interface and a device-cloud co-simulation device-side user interface, each of which includes associated settings of parameters of the receiving device of the device-cloud collaborative content.

[0170] In this embodiment, a mapping process is performed on the aforementioned parameters to simplify the parameters exposed to artists and facilitate their understanding. For example, as shown in FIG. 6d (FIG. 6d is a diagram of an editing interface in which pre-set parameters are mapped to user controls). The encoding / decoding parameter control of the device-cloud collaborative DDGI is used as an example. Multiple sets of encoder parameters are pre-set and mapped to a drop-down list control "encoding quality." In this way, artists only need to select an encoding quality level from the drop-down list based on a test case, and do not need to learn encoding knowledge.

[0171] For example, as shown in FIG. 6e (FIG. 6e is a diagram of the game editor interface), a user selects device-cloud DDGI_instance1 for editing and enters specific parameter values ​​into controls corresponding to one or more of the instance attribute parameters (transformation and rendering), coding parameters (encoding and decoding), and transmission parameters (transmission) of the device-cloud collaborative content.

[0172] For example, as shown in FIG. 6f (FIG. 6f is a diagram of a game editor interface), one or more parameters of the instance attribute parameters, coding parameters, and transmission parameters of the device-cloud collaborative content may be set in the device-cloud collaborative content editing user interface.

[0173] For example, as shown in FIG. 6g (FIG. 6g is a diagram of the terminal-cloud content preview interface), one or more parameters may be set separately in the simulation transmission parameter user interface and the simulation terminal-side execution user interface to obtain the execution environment of the receiving device through simulation.

[0174] 3. Modifying the corresponding parameters in response to trigger actions performed by the user on each of the aforementioned interfaces / controls.

[0175] 4. Displaying a cloud preview interface in response to a trigger action performed by a user on a preview control.

[0176] On the device side, cloud-side data (i.e., device-cloud collaborative data) for various device-cloud collaborative content is generated. Possible cases are as follows:

[0177] (a) Generating real-time device-cloud collaborative data; (b) generating device-cloud collaborative data offline; and (c) Read device-cloud collaboration data from the file.

[0178] If the user modifies the input to the aforementioned controls, steps 2-4 are repeated to generate new device-cloud collaboration data.

[0179] For example, as shown in FIG. 6h (FIG. 6h is a diagram of the preview interface), the user triggers the preview control and a preview rendering interface is displayed, allowing the user to intuitively see the effect achieved by modifying the aforementioned parameters.

[0180] 5. Save the user input and terminate execution.

[0181] In this embodiment, in the editing and preview states of the game engine editor, a preview of the presentation effect of device-cloud collaborative content for a specific terminal-side device under a specific network condition is completed, and the operational procedures of packaging and deployment, network configuration, and cloud-side and device-side configuration are omitted. The game engine editor provides a user interface and user controls for controlling the device-cloud collaborative content, and the operational procedures of modifying encoding / decoding and transmission parameters in the code are omitted. The user can directly modify the aforementioned parameters based on effect feedback to optimize the effect.

[0182] Embodiment 2 This embodiment is in a device-cloud collaborative game production service scenario in which device-cloud collaborative DDGI is applied. Device-cloud collaborative DDGI is applied to a game rendering system. Device-cloud collaborative DDGI has several instances in game scenarios. In this embodiment, the target device on the device side is a mobile device, such as a mobile phone or a tablet. In this embodiment, network transmission is used.

[0183] As shown in FIG. 7, the procedure of this embodiment is as follows:

[0184] 1. Display the editing interface of the device-cloud collaborative DDGI in the game editor, as shown in Fig. 6e.

[0185] In this embodiment, a mapping process is performed on user control parameters and program parameters to simplify the parameters exposed to the user and make it easier for the user to understand. As shown in Figure 7, multiple sets of encoder parameters are pre-configured and mapped to "encoding quality".

[0186] 2. The program responds to user input for the aforementioned controls and modifies the corresponding device-cloud collaborative DDGI attribute parameters, encoding / decoding parameters, and transmission parameters.

[0187] 3. Unlike the first embodiment, device-cloud collaborative data is generated in real time.

[0188] 4. Simulate device-cloud transmission to obtain device-side data, as shown in Figure 6f and Figure 6g.

[0189] (a) encoding cloud-side data based on the configuration of encoding / decoding parameters for the device-cloud collaborative content; (b) transmitting and receiving bitstreams based on the configuration of transmission parameters of the device-cloud collaborative content; (c) decoding the bitstream based on the configuration of encoding / decoding parameters of the device-cloud collaborative content; and (f) Reconstructing device-cloud data based on the configuration of simulation parameters presented to the device side as described above to obtain device-side data.

[0190] 5. Displaying the device-side presentation of the device-cloud collaborative DDGI in the edit window or preview window of the game engine editor by using the device-side data.

[0191] 6. Same as embodiment 1.

[0192] 7. Same as embodiment 1.

[0193] In this embodiment, a preview of the device-side device-cloud collaborative DDGI presentation effect is presented in the editing and preview states of the game engine editor, and the operational procedures of packaging and deployment, network configuration, and cloud-side and device-side configuration are omitted. The game engine editor provides a user interface and user controls for controlling the device-cloud collaborative DDGI, and the operational procedures of modifying encoding / decoding and transmission parameters in the code are omitted. The user can directly modify the aforementioned parameters based on effect feedback to optimize the effect.

[0194] Embodiment 3 This embodiment is directed to a device-cloud collaborative game production service scenario in which device-cloud collaborative reflection is applied, including device-cloud collaborative planar reflection, device-cloud collaborative reflection probe, device-cloud collaborative planar spatial reflection, and device-cloud collaborative ray-traced reflection.

[0195] The aforementioned device-cloud collaboration techniques are applied to a rendering system. These device-cloud collaboration techniques may have one instance, several instances, or no instances in a game scenario. In this embodiment, the target device on the device side is a mobile device, such as a mobile phone or a tablet. In this embodiment, network transmission is used.

[0196] As shown in Figure 8 (Figure 8 is a diagram of two game editors), two game engine editors are enabled on the cloud-side device, one functions as the device-cloud collaborative cloud side, and the other functions as the device-cloud collaborative device side. The procedure of this embodiment is as follows:

[0197] 1. Open two Game Editors for the same game project, i.e. Game Editor 1 and Game Editor 2.

[0198] (a) Game Editor 1 i. Displaying a device-cloud collaborative content editing interface, as shown in FIG. 6b; and ii. Select one or more instance objects of the device-cloud collaborative reflection to display the device-cloud collaborative content editing interface on the attribute panel of the instance object, as shown in FIG. 6e.

[0199] (b) In Game Editor 2 i. Displaying a device-cloud collaborative content editing and preview interface, as shown in FIG. 6b; and ii. Select one or more instance objects of the device-cloud collaborative reflection to display the device-cloud collaborative content editing interface on the attribute panel of the instance object, as shown in FIG. 6e.

[0200] In this embodiment, to simplify the parameters exposed to the editor and make it easier for the editor to understand, a mapping process is performed for the user-controlled parameters and program parameters on the device-cloud collaborative content editing interface. As shown in Fig. 6h, multiple sets of encoder parameters are pre-configured and mapped to "encoding quality".

[0201] 2. The program responds to the input of the editor for the aforementioned control and modifies the corresponding device-cloud collaborative reflection attribute parameters, encoding / decoding parameters, and transmission parameters.

[0202] 3. Different from embodiment 1 and embodiment 2, two game editor programs are started to simulate the device-cloud collaborative cloud side and the device-cloud collaborative device side respectively.

[0203] 4. Simulate device-cloud transmission to obtain device-side data.

[0204] (a) The game editor 1 program encodes various cloud-side data based on the configuration of encoding / decoding parameters for the device-cloud collaborative content described above.

[0205] (c) based on the configuration of the transmission parameters of the device-cloud collaborative content; i. The Game Editor 1 program transmits various cloud-side data, and ii. The Game Editor 2 program receives various cloud-side data.

[0206] (e) The Game Editor 2 program decodes the bitstream based on the configuration of the encoding / decoding parameters of the device-cloud collaborative content described above.

[0207] (f) The game editor 2 program reconstructs the device-cloud data based on the configuration of simulation parameters presented to the device side as described above to obtain the device-side data.

[0208] 5. Displaying a device-side presentation of device-cloud coordinated reflection in the editing window or preview window of Game Editor 2 by using device-side data.

[0209] 6. Same as embodiment 1.

[0210] 7. Same as embodiment 1.

[0211] In this embodiment, the editing and preview states of the game engine editor 2 present a preview of the presentation effect of device-cloud collaborative reflection on the device side, and the operational procedures of packaging and deployment, network configuration, and cloud-side and device-side configuration are omitted. The game engine editor provides a user interface and user controls for controlling device-cloud collaborative reflection, and the operational procedures of modifying encoding / decoding and transmission parameters in code are omitted. The editor can directly modify the aforementioned parameters based on effect feedback to optimize the effect.

[0212] Embodiment 4 This embodiment is a device-cloud collaborative game test service scenario in which a device-cloud collaborative shadow mask is applied. The device-cloud collaborative technology described above is applied to a rendering system, and one or more instances exist in the game scenario. In this embodiment, the target device on the device side is an electronic device such as a PC, a mobile phone, or a tablet. In this embodiment, a storage / loading method is used to simulate transmission.

[0213] As shown in FIG. 9 (FIG. 9 is a diagram simulating device-cloud transmission in a storage / reading manner), the service scenario is a test scenario in game development. Multiple groups of cloud-side data are generated on the cloud side, and the cloud-side data are stored in a storage medium. The device side is simulated to read the cloud-side data. The data is not transmitted in real time. Instead, the data is stored and read. The procedure of this embodiment is as follows:

[0214] 1. Display the editing interface in the game editor.

[0215] (a) As shown in Fig. 6b, the device-cloud collaborative content editing interface is displayed.

[0216] (b) As shown in Fig. 6e, one or more instance objects of the device-cloud collaborative shadow mask are selected, and the device-cloud collaborative content editing interface is displayed in the attribute panel of the instance object.

[0217] In this embodiment, in order to simplify the parameters exposed to the tester and facilitate the tester's understanding, a mapping process is performed for the user-controlled parameters and program parameters on the device-cloud collaborative content editing interface. As shown in Fig. 6h, multiple sets of encoder parameters are pre-configured and mapped to "encoding quality".

[0218] 2. The program responds to the tester's input for the aforementioned control and modifies the corresponding device-cloud collaborative shadow mask attribute parameters, encoding / decoding parameters, and transmission parameters.

[0219] 3. Generate cloud-side data (i.e., device-cloud data) of the device-cloud coordinated shadow data.

[0220] 4. Encode the cloud-side data based on the configuration of encoding / decoding parameters of the device-cloud collaborative content described above to obtain a bitstream.

[0221] 5. Based on the transmission parameter configuration of the device-cloud collaborative content, simulate the situation during device-cloud transmission, such as packet loss at the transmitter, process the bitstream, and store the bitstream in a storage medium or database.

[0222] 6. In the same game editor program or another game editor program, simulate device-to-cloud transmission to obtain device-side data.

[0223] (a) reading the bitstream from a storage medium or a database; (b) performing processing such as frame error processing on the bitstream based on the transmission parameter configuration of the device-cloud collaborative content to simulate device-cloud transmission; (c) decoding the bitstream based on the configuration of encoding / decoding parameters for the device-cloud collaborative content; (d) Reconstructing device-cloud data based on the configuration of simulation parameters presented to the device side as described above to obtain device-side data.

[0224] 7. Display a representation of the device-cloud cooperative reflection on the device side by using the device-side data in the editing window or preview window of the Game Editor 2.

[0225] 8. If the tester corrects the input to the control and performs the test again, repeat steps 2 through 5.

[0226] 9. Save the user input and terminate execution.

[0227] In this embodiment, the bitstream is stored and retrieved, so that the same device-cloud data can be used repeatedly multiple times, and the operational procedures of packaging and deployment, network configuration, and cloud-side and device-side configuration can be omitted. The game engine editor provides a user interface and user controls for controlling device-cloud collaborative reflection, and the operational procedures of modifying codec and transmission parameters in code are omitted. The tester can directly modify the aforementioned parameters based on the test requirements to meet the test case requirements.

[0228] 10 is a structural diagram of an apparatus 1000 for previewing device-cloud collaborative content according to the present application. As shown in FIG. 10, the apparatus 1000 for previewing device-cloud collaborative content in this embodiment may be used in a generating device of the device-cloud collaborative content. The apparatus 1000 for previewing device-cloud collaborative content may include a display module 1001.

[0229] The display module 1001 is configured to display an editing interface, the editing interface including a preview control and a plurality of setting controls, the plurality of setting controls being used to set parameters associated with the device-cloud collaborative content, the parameters associated with the device-cloud collaborative content including parameters of a receiving device of the device-cloud collaborative content. The display module 1001 is further configured to display a preview interface based on the parameters of the receiving device in response to a trigger operation on the preview control, the preview interface being used to simulate an interface for displaying the device-cloud collaborative content on the receiving device.

[0230] In a possible implementation, the parameters of the receiving device include at least one of the following: a type of the receiving device, a model of the receiving device, a chip model of the receiving device, a battery level of the receiving device, a resolution of the receiving device, a volume of the receiving device, and an ambient temperature of the receiving device.

[0231] In a possible implementation, the parameters associated with the device-cloud collaborative content further include coding parameters and / or transmission parameters, where the coding parameters are used to perform an encoding process on the device-cloud collaborative content to obtain a bitstream or a decoding process on the bitstream to reconstruct the device-cloud collaborative content, and the transmission parameters are used to simulate a situation and environment for transmitting the bitstream.

[0232] In a possible implementation, the display module 1001 is further configured to display a coding parameter setting window in response to a trigger operation on the first setting control, the coding parameter setting window including a plurality of first setting sub-controls, each of the plurality of first setting sub-controls being used to set video encoder / decoder parameters and audio encoder / decoder parameters, and the first setting control being one of the plurality of setting controls.

[0233] In a possible implementation, the display module 1001 is further configured to display a transmission parameter setting window in response to a trigger operation on the second setting control, the transmission parameter setting window including a plurality of second setting sub-controls, each of which is used to set a transmit / receive frame rate, a maximum transmit / receive bit rate, a network delay, a network instability, a physical distance, a physical location, and a transmission channel, and the second setting control is one of the plurality of setting controls.

[0234] In a possible implementation, the parameters associated with the device-cloud collaborative content further include instance attribute parameters of the device-cloud collaborative content, which are used to generate the device-cloud collaborative content.

[0235] In a possible implementation, the display module 1001 is further configured to display an instance attribute parameter setting window in response to a trigger operation on the third setting control, the instance attribute parameter setting window including a plurality of third setting sub-controls, each of the plurality of third setting sub-controls being used to set the position, ratio, rotation, special effect intensity, and weight of an instance of the device-cloud collaborative content, and the third setting control being one of the plurality of setting controls.

[0236] In possible implementations, the device-cloud collaborative content includes content of a device-cloud collaborative game, content of a device-cloud collaborative document, or content of a device-cloud collaborative live broadcast.

[0237] The device in this embodiment may be configured to realize the technical solution of the method embodiment shown in Figure 4. The implementation principle and its technical effect are similar. The details will not be described again in this specification.

[0238] In the implementation process, the steps in the aforementioned method embodiments may be implemented by using hardware integrated logic circuitry in a processor or by using instructions in the form of software. The processor may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, or a discrete hardware component. The general-purpose processor may be a microprocessor, or the processor may be any conventional processor, etc. The steps of the methods disclosed in the embodiments of the present application may be directly implemented and completed by a hardware encoding processor, or implemented and completed by a combination of hardware and software modules of the encoding processor. The software modules may be located in storage media that are mature technologies in this technology, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, or registers. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above-described method together with the processor's hardware.

[0239] The memory in the above embodiments may be volatile or nonvolatile memory, or may include both volatile and nonvolatile memory. Nonvolatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory may be random access memory (RAM) used as an external cache. By way of example and not limitation, many forms of RAM may be used, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchlink dynamic random access memory (SLDRAM), and direct rambus random access memory (DR RAM). It should be noted that memory in the systems and methods described herein includes, but is not limited to, these and any other suitable types of memory.

[0240] Those skilled in the art may recognize that, in combination with the examples described in the embodiments disclosed herein, the units and algorithm steps may be implemented by electronic hardware or a combination of computer software and electronic hardware. Whether a function is implemented by hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art may use various methods to implement the described functions for each specific application, but the implementation form should not be considered to exceed the scope of this application.

[0241] For the sake of convenience and simplicity, the detailed operation processes of the aforementioned systems, devices, and units shall refer to the corresponding processes of the aforementioned method embodiments, and will not be described in detail herein, which can be clearly understood by those skilled in the art.

[0242] In some embodiments provided in the present application, it should be understood that the disclosed system, device, and method may be realized in other manners. For example, the described device embodiment is merely an example. For example, the division into units is merely a logical division of function, and may be divided differently in actual implementation. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not implemented. In addition, the shown or described mutual couplings or direct couplings or communication connections may be implemented through some interfaces. Indirect couplings or communication connections between devices or units may be implemented in electronic, mechanical, or other forms.

[0243] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, and may be located in one location or distributed across multiple network units. To achieve the objectives of the solutions of the embodiments, some or all of the units may be selected based on actual requirements.

[0244] In addition, the functional units in the embodiments of the present application may be integrated into one processing unit, and each of the units may exist physically alone, or two or more units may be integrated into one unit.

[0245] When a function is implemented in the form of a software functional unit and sold or used as an independent product, the function may be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or a portion contributing to the prior art, or a portion of the technical solution may be implemented in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for instructing a computer device (such as a personal computer, a server, or a network device) to perform all or part of the steps of the method in the embodiments of the present application. The aforementioned storage medium includes any medium that can store program code, such as a USB flash drive, a removable hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, an optical disk, etc.

[0246] The above description is merely a specific implementation of the present application and is not intended to limit the scope of protection of the present application. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present application shall fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be subject to the scope of protection of the claims. [Explanation of symbols]

[0247] 1 antenna 2 antennas 200 Generation Device 211 processor 212 memory 213 Transceiver 214 Network Interface 215 Antenna 300 receiving devices 310 processor 320 External Memory Interface 321 internal memory 330 Universal Serial Bus Interface 340 Charging Management Module 341 Power Management Module 342 battery 350 Mobile Communication Module 360 Wireless Communication Module 370 Audio Module 370A Speaker 370B receiver 370C Microphone 370D Headset Jack 380 Sensor Module 380A Pressure Sensor 380B Gyro Sensor 380C Barometric Pressure Sensor 380D Magnetic Sensor 380E Accelerometer 380F Distance Sensor 380G Optical Proximity Sensor 380H Fingerprint Sensor 380J Temperature Sensor 380K touch sensor 380L Ambient Light Sensor 380M Bone Conduction Sensor 390 Button 391 Motor 392 indicator 393 Camera 394 Display 395 Subscriber Identity Module Card Interface 1000 devices 1001 Display Module

Claims

1. 1. A method for previewing device-cloud collaborative content, performed by a generating device of the device-cloud collaborative content, comprising: displaying an editing interface, the editing interface comprising a preview control and a plurality of setting controls, the plurality of setting controls being used to set parameters associated with the device-cloud collaborative content, the parameters associated with the device-cloud collaborative content including parameters of a receiving device of the device-cloud collaborative content; displaying a preview interface based on the parameters of the receiving device in response to a trigger action on the preview control, the preview interface being used to simulate an interface for displaying the device-cloud collaborative content on the receiving device; A method comprising:

2. 2. The method of claim 1, wherein the parameters of the receiving device include at least one of: a type of the receiving device, a model of the receiving device, a chip model of the receiving device, a battery level of the receiving device, a resolution of the receiving device, a volume of the receiving device, and an ambient temperature of the receiving device.

3. the parameters associated with the device-cloud collaborative content further include coding parameters and / or transmission parameters; the coding parameters are used to perform an encoding process on the device-cloud collaborative content to obtain a bitstream, or to perform a decoding process on the bitstream to reconstruct the device-cloud collaborative content; The method according to claim 1 or 2, wherein the transmission parameters are used to simulate the conditions and environment in which the bitstream is transmitted.

4. 4. The method of claim 3, further comprising the step of displaying the coding parameter setting window in response to a trigger operation on a first setting control, the coding parameter setting window comprising a plurality of first setting sub-controls, each of the plurality of first setting sub-controls being used to set a video encoder / decoder parameter or an audio encoder / decoder parameter, and the first setting control being one of the plurality of setting controls.

5. 5. The method of claim 3, further comprising the step of displaying a setting window for the transmission parameters in response to a trigger operation on a second setting control, the setting window for the transmission parameters comprising a plurality of second setting sub-controls, each of the plurality of second setting sub-controls being used to set a transmission / reception frame rate, a maximum transmission / reception bit rate, a network delay, a network instability, a physical distance, a physical location, or a transmission channel, and the second setting control being one of the plurality of setting controls.

6. the parameters associated with the device-cloud collaborative content further include instance attribute parameters of the device-cloud collaborative content; The method of claim 1 , wherein the instance attribute parameters are used to generate the device-cloud collaborative content.

7. 7. The method of claim 6, further comprising the step of displaying a setting window for the instance attribute parameter in response to a trigger operation on a third setting control, the setting window for the instance attribute parameter comprising a plurality of third setting sub-controls, each of the plurality of third setting sub-controls being used to set the position, ratio, rotation, special effect intensity, or weight of the instance of the device-cloud collaborative content, and the third setting control being one of the plurality of setting controls.

8. The method according to any one of claims 1 to 7, wherein the device-cloud collaborative content includes content of a device-cloud collaborative game, content of a device-cloud collaborative document, or content of a device-cloud collaborative live broadcast.

9. 1. An apparatus for previewing device-cloud collaborative content, used in a device that generates the device-cloud collaborative content, comprising: a display module configured to display an editing interface, the editing interface comprising: a preview control; and a plurality of setting controls, the plurality of setting controls being used to set parameters associated with the device-cloud collaborative content, the parameters associated with the device-cloud collaborative content including parameters of a receiving device of the device-cloud collaborative content; the display module is further configured to, in response to a trigger action on the preview control, display a preview interface based on the parameters of the receiving device, the preview interface being used to simulate an interface for displaying the device-cloud collaborative content on the receiving device.

10. 10. The apparatus of claim 9, wherein the parameters of the receiving device include at least one of: a type of the receiving device, a model of the receiving device, a chip model of the receiving device, a battery level of the receiving device, a resolution of the receiving device, a volume of the receiving device, or an ambient temperature of the receiving device.

11. the parameters associated with the device-cloud collaborative content further include coding parameters and / or transmission parameters; the coding parameters are used to perform an encoding process on the device-cloud collaborative content to obtain a bitstream, or to perform a decoding process on the bitstream to reconstruct the device-cloud collaborative content; 11. The apparatus according to claim 9 or 10, wherein the transmission parameters are used to simulate the conditions and environment in which the bitstream is transmitted.

12. 12. The apparatus of claim 11, wherein the display module is further configured to display a setting window of the coding parameters in response to a trigger operation on a first setting control, the setting window of the coding parameters comprising a plurality of first setting sub-controls, each of the plurality of first setting sub-controls being used to set a video encoder / decoder parameter or an audio encoder / decoder parameter, and the first setting control being one of the plurality of setting controls.

13. 13. The apparatus of claim 11 or 12, wherein the display module is further configured to display a setting window of the transmission parameters in response to a trigger operation on a second setting control, the setting window of the transmission parameters comprising a plurality of second setting sub-controls, each of the plurality of second setting sub-controls being used to set a transmission / reception frame rate, a maximum transmission / reception bit rate, a network delay, a network instability, a physical distance, a physical location, or a transmission channel, and the second setting control being one of the plurality of setting controls.

14. the parameters associated with the device-cloud collaborative content further include instance attribute parameters of the device-cloud collaborative content; The apparatus of claim 9 , wherein the instance attribute parameters are used to generate the device-cloud collaborative content.

15. 15. The device of claim 14, wherein the display module is further configured to display a setting window for the instance attribute parameter in response to a trigger operation on a third setting control, the setting window for the instance attribute parameter comprising a plurality of third setting sub-controls, each of the plurality of third setting sub-controls being used to set a position, a ratio, a rotation, a special effect intensity, or a weight of the instance of the device-cloud collaborative content, and the third setting control being one of the plurality of setting controls.

16. The apparatus of claim 9 , wherein the device-cloud collaborative content includes content of a device-cloud collaborative game, content of a device-cloud collaborative document, or content of a device-cloud collaborative live broadcast.

17. 1. An electronic device comprising: one or more processors; a memory configured to store one or more programs; Equipped with 9. An electronic device, wherein the one or more programs, when executed by the one or more processors, enable the one or more processors to perform the method of any one of claims 1 to 8.

18. 9. A computer-readable storage medium comprising a computer program, which, when executed by a computer, enables the computer to carry out the method of any one of claims 1 to 8.

19. 9. A computer program product comprising computer program code that, when executed on a computer, enables the computer to carry out the method of any one of claims 1 to 8.

Images