Resource processing method and apparatus, electronic device, and storage medium

Abstract

This application provides a resource processing method, apparatus, electronic device, and storage medium, applied to a display device, wherein the display device is communicatively connected to an external smart device. The method includes: acquiring a resource to be processed; determining the resource access permission, resource type, and hardware requirement level corresponding to the resource to be processed; determining, based on the resource access permission, whether only one of the systems of the display device and the external smart device has permission to access the resource to be processed; when only one has permission to access the resource to be processed, determining the system with the access permission as the target processing system; when both have access permission or neither has access permission, determining the target processing system based on the resource type; processing the resource to be processed using the target processing system to obtain a processing result; and displaying the processing result using the display device.

Description

Technical Field

[0001] This application relates to the field of smart home technology, and in particular to a resource processing method, apparatus, electronic device and storage medium. Background Technology

[0002] With the rapid development of ultra-high-definition display technology, Internet of Things (IoT) communication technology, and artificial intelligence interaction technology, televisions have gradually evolved from traditional content playback terminals into home interaction centers integrating audio-visual entertainment, information display, and smart home control. Meanwhile, mobile smart terminals, represented by smartphones, have become the most frequently used personal computing and communication devices by users, thanks to their powerful independent processing capabilities, rich application ecosystems, and mature camera capture capabilities.

[0003] Currently, some high-end television products are beginning to incorporate a secondary screen design, which involves adding a small auxiliary screen to the main body of the television. In this solution, both the main screen and the secondary screen are driven and controlled by a single system on the television.

[0004] However, when a single system drives both the main screen and the secondary screen simultaneously, the system load is concentrated on a single processor, resulting in resource shortages and potentially causing system lag. Summary of the Invention

[0005] This application provides a resource processing method, apparatus, electronic device, and storage medium to solve the problem in the prior art where, when a single system drives both the main screen and the secondary screen simultaneously, the system load is concentrated on a single processor, resulting in resource shortages and system lag.

[0006] In a first aspect, this application provides a resource processing method applied to a display device, wherein the display device is communicatively connected to an external smart device, comprising: Acquire resources to be processed; Determine the resource access permissions, resource type, and hardware requirement level corresponding to the resource to be processed; Based on the resource access permissions, it is determined whether only one of the display device's system and the external smart device's system has permission to access the resource to be processed; if only one has permission to access the resource to be processed, the system with the access permission is determined as the target processing system; if both have access permissions or neither has access permissions, the target processing system is determined based on the resource type, wherein: when the resource type is a first type, the display device's system is determined as the target processing system; when the resource type is a second type, the external smart device's system is determined as the target processing system; when the resource type is a third type, the target processing system is determined based on the hardware requirement level. The target processing system is used to process the resource to be processed to obtain the processing result; The processing result is displayed using the display device.

[0007] In one possible implementation, the method further includes: Determine auxiliary information from the current display information of the display device; The auxiliary information is sent to the external smart device via the communication connection so that the auxiliary information can be displayed on the external smart device.

[0008] In one possible implementation, determining the auxiliary information from the current display information of the display device includes: Get custom settings information; The auxiliary information is determined from the currently displayed information based on the custom settings.

[0009] In one possible implementation, the method further includes: Control commands are sent to the external intelligent device through the communication connection; The system receives the execution status returned by the external intelligent device through the communication connection, and the execution status is used to characterize the state of the resource to be processed after the external intelligent device executes the control command; Based on the execution status, update the display content of the processing result on the display device.

[0010] In one possible implementation, the method further includes: In response to an image acquisition request, the camera of the external smart device is invoked to acquire image data via the communication connection; Receive the image data returned by the external smart device through the communication connection; The image data is displayed using the display device.

[0011] In one possible implementation, the method further includes: Receive user commands via remote control device; Parse the user instruction to determine the instruction type of the user instruction; When the instruction type is the same as the instruction type corresponding to the external smart device, the user instruction is sent to the external smart device through the communication connection.

[0012] Secondly, this application provides a resource processing apparatus applied to a display device, the display device being communicatively connected to an external smart device, comprising: The acquisition module is used to acquire resources to be processed. The first determining module is used to determine the resource access permissions, resource type, and hardware requirement level corresponding to the resource to be processed; The second determining module is used to determine, based on the resource access permissions, whether only one of the systems of the display device and the external smart device has permission to access the resource to be processed; when only one has permission to access the resource to be processed, the system with the access permission is determined as the target processing system; when both have access permissions or neither has access permissions, the target processing system is determined based on the resource type, wherein: when the resource type is a first type, the system of the display device is determined as the target processing system; when the resource type is a second type, the system of the external smart device is determined as the target processing system; when the resource type is a third type, the target processing system is determined based on the hardware requirement level. The processing module is used to process the resource to be processed using the target processing system to obtain the processing result; The display module is used to display the processing result using the display device.

[0013] In one possible implementation, the apparatus further includes a transmitting module for: Determine auxiliary information from the current display information of the display device; The auxiliary information is sent to the external smart device via the communication connection so that the auxiliary information can be displayed on the external smart device.

[0014] In one possible implementation, the sending module is further configured to: Get custom settings information; The auxiliary information is determined from the currently displayed information based on the custom settings.

[0015] In one possible implementation, the device further includes an update module for: Control commands are sent to the external intelligent device through the communication connection; The system receives the execution status returned by the external intelligent device through the communication connection, and the execution status is used to characterize the state of the resource to be processed after the external intelligent device executes the control command; Based on the execution status, update the display content of the processing result on the display device.

[0016] In one possible implementation, the device further includes a calling module for: In response to an image acquisition request, the camera of the external smart device is invoked to acquire image data via the communication connection; Receive the image data returned by the external smart device through the communication connection; The image data is displayed using the display device.

[0017] In one possible implementation, the device further includes a distribution module for: Receive user commands via remote control device; Parse the user instruction to determine the instruction type of the user instruction; When the instruction type is the same as the instruction type corresponding to the external smart device, the user instruction is sent to the external smart device through the communication connection.

[0018] Thirdly, this application provides an electronic device, including: a processor and a memory, wherein the processor is configured to execute a resource processing program stored in the memory to implement the resource processing method described in any one of the first aspects.

[0019] Fourthly, this application provides a storage medium storing one or more programs that can be executed by one or more processors to implement the resource processing method described in any one aspect.

[0020] Fifthly, this application provides a computer program product containing instructions that, when run on a computer, causes the computer to perform any of the resource processing methods described above.

[0021] Compared with the prior art, the technical solution provided in this application has the following advantages: The method provided in this application involves a display device acquiring resources to be processed and determining their access permissions, resource type, and hardware requirement level. Based on these three parameters, a target processing system is determined between the display device's system and the system of an external intelligent device connected to the display device. The target processing system processes the resources to be processed, and the processing results are displayed on the display device. Compared to the prior art where a single system drives all functions, leading to resource constraints, this application distributes processing tasks originally concentrated in a single system to the display device system and the external intelligent device system for collaborative execution, thus avoiding system response lag caused by excessive load on a single system. Furthermore, since the target processing system is dynamically determined between the two systems, if a single system malfunctions, processing can continue by switching to the other system, improving system fault tolerance. Moreover, by reusing the existing processing capabilities of the external intelligent device, no additional hardware needs to be added to the display device, which helps reduce hardware costs. Attached Figure Description

[0022] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0023] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] One or more embodiments are illustrated by way of example with reference numerals in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0025] Figure 1 A flowchart illustrating an embodiment of a resource processing method provided in this application; Figure 2 A flowchart illustrating an embodiment of another resource processing method provided in this application; Figure 3 A flowchart illustrating another embodiment of a resource processing method provided in this application; Figure 4 A flowchart illustrating another embodiment of the resource processing method provided in this application; Figure 5 A block diagram illustrating an embodiment of a resource processing apparatus provided in this application; Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0027] The following disclosure provides numerous different embodiments or examples for implementing various structures of this application. To simplify the disclosure, specific examples of components and arrangements are described below. These are merely examples and are not intended to limit the scope of this application. Furthermore, reference numerals and / or letters may be repeated in different examples. Such repetition is for simplification and clarity and does not in itself indicate a relationship between the various embodiments and / or arrangements discussed.

[0028] To address the technical problem in existing technologies where a single system drives both the main screen and the secondary screen simultaneously, resulting in concentrated system load on a single processor, resource constraints, and potential system lag, this application provides a resource processing method that distributes processing tasks originally concentrated on a single system to the display device system and external smart device system for collaborative execution, thereby avoiding system lag caused by excessive load on a single system.

[0029] Figure 1 This is a flowchart illustrating an embodiment of a resource processing method provided in this application. In one embodiment, the method is applied to a display device (such as a television), which is communicatively connected to an external smart device (such as a smartphone, tablet, etc.). Figure 1 As shown, the method includes the following steps: S101. Obtain the resources to be processed.

[0030] Resources to be processed: refers to media content or application data that users request to play, display, or run. Resources to be processed include, but are not limited to, any one or more of the following: video resources (such as movies, TV series, and variety shows), audio resources (such as music and audiobooks), information resources (such as news, short videos, and live streams), game resources (such as lightweight casual games and large-scale 3D games), and social resources (such as video call data and instant messaging content).

[0031] In this embodiment, a user can initiate a resource acquisition request via button input, voice input, or pointing input using a TV remote control. The display device responds to the user's request by retrieving the corresponding resource to be processed from the TV's local storage medium, a local area network device connected to the display device, or an internet content server. For example, if a user inputs "Play XX movie" via voice input on the remote control, the display device receives and parses the voice command, then retrieves the video stream data corresponding to the movie from an installed video application or internet content source, using this video stream data as the resource to be processed. Similarly, if a user opens a short video application on the TV, the display device responds to this operation by retrieving video data from the recommended video list on the short video platform server as the resource to be processed.

[0032] S102. Determine the resource access permissions, resource type, and hardware requirement level corresponding to the resource to be processed.

[0033] Resource access permissions refer to whether the display device system and / or external smart device system have the legal right to access, decode, play, or run the resources to be processed. Specifically, resource access permissions are reflected in the membership level or subscription status of the accounts logged into on each system, such as whether video applications have activated VIP memberships or whether game applications have been purchased.

[0034] Resource type: This refers to the content category of the resource to be processed, indicating which system is suitable to process the resource. Resource types include audio-visual (such as movies, TV series, and music), information (such as news, short videos, and live streams), and games.

[0035] Hardware requirement level: This refers to the hardware performance requirements of the processing equipment for the resource to be processed. It measures the minimum or recommended hardware configuration required to run or process the resource. Hardware requirement levels include high-end and low-end. For example, game resources have different requirements for CPU (Central Processing Unit) computing power, GPU (Graphics Processing Unit) graphics rendering power, and memory capacity.

[0036] In this embodiment, after acquiring the resource to be processed, the display device further determines the information corresponding to the aforementioned three dimensions of the resource. Specifically, the display device determines the resource type and hardware requirement level by reading the resource's metadata, parsing the file header information corresponding to the resource, or calling the API (Application Programming Interface) of the relevant application; and determines the resource access permissions by querying information such as the user login status and membership validity period of the corresponding application on each system. For example, when the resource to be processed is a movie, the display device determines its resource type as audio-visual by reading the movie's media information, determines the TV system's resource access permissions by calling the video application's API to query the membership status of the application on the TV system, determines the external smart device system's resource access permissions by querying the login status of the same video application on the external smart device system, and determines its hardware requirement level based on parameters such as the movie's encoding format, resolution, and bitrate. For example, when the resource to be processed is a game, the display device determines its resource type as a game by reading the configuration information in the game's installation package or startup file, and determines its hardware requirement level as high-end or low-end based on the game's recommended configuration.

[0037] S103. Based on the resource access permissions, determine whether only one of the display device's system and the external smart device's system has permission to access the resource to be processed; when only one has permission to access the resource to be processed, determine the system with the access permission as the target processing system; when both have access permissions or neither has access permissions, determine the target processing system based on the resource type, wherein: when the resource type is a first type, determine the display device's system as the target processing system; when the resource type is a second type, determine the external smart device's system as the target processing system; when the resource type is a third type, determine the target processing system based on the hardware requirement level.

[0038] The system of a display device refers to the computing and processing system built into the display device itself, which has an operating system, processor, memory and storage units, and can independently run applications and process various media resources.

[0039] The system of an external intelligent device refers to the independent operating system, processor, memory, and storage unit of the external intelligent device that communicates with the display device. The external intelligent device system is independent of the display device system and each has complete computing capabilities.

[0040] Target processing system: refers to the system used to process the resources to be processed, determined from the display device system and external intelligent device system after comprehensive judgment based on resource access permissions, resource type and hardware requirement level.

[0041] The first type refers to audio-visual resources, such as movies, TV series, variety shows, and music. These resources can usually be handled well by a television system for a good playback experience. The second type refers to informational resources, such as news articles, short videos, and live streams. These resources are more mature in the mobile ecosystem and are better handled by external smart device systems. The third type refers to gaming resources, such as lightweight casual games and large-scale 3D games. These resources have higher hardware performance requirements, and the appropriate system to handle them needs to be determined based on the hardware requirements.

[0042] In this embodiment, the display device first determines the target processing system based on resource access permissions: when only one of the display device system and the external smart device system has permission to access the resource to be processed, the display device directly identifies the system with the access permission as the target processing system to ensure that the resource can be played or run normally and to avoid processing failure due to insufficient permissions; when both have access permissions or neither has access permissions, the display device further determines the target processing system based on the resource type: if the resource type is the first type (audio-visual), the display device system is identified as the target processing system to utilize the TV system's direct decoding and playback capabilities for audio-visual resources; if the resource type is the second type (information), the external smart device system is identified as the target processing system to utilize the advantages of a more mature mobile information application ecosystem and more timely content updates; if the resource type is the third type (game), the system is further identified based on hardware requirements, and a system that can meet the requirements for smooth resource operation is selected as the target processing system. For example, when the resource to be processed is a large-scale 3D game with high-end hardware requirements, the display device determines the external smart device system as the target processing system, and the processor and graphics processor of the external smart device perform real-time rendering calculations of the game screen to ensure smooth operation at high image quality; when the resource to be processed is a lightweight 2D casual game with low-end hardware requirements, the display device determines its own system as the target processing system, and the TV system runs the game directly without calling the external device.

[0043] This solution enables the display device to dynamically select the most suitable processing system based on a comprehensive assessment of three dimensions: resource access permissions, resource type, and hardware requirement level. This achieves reasonable scheduling and efficient utilization of resources across both systems, thereby distributing processing tasks that were originally concentrated in a single system to the display device system and external intelligent device systems for collaborative execution. This avoids system response lag caused by excessive load on a single system, and when one system malfunctions, processing can be continued by switching to the other system, thus improving system fault tolerance.

[0044] S104. The target processing system is used to process the resource to be processed to obtain the processing result.

[0045] Processing result: refers to the data such as video streams, audio streams, and interactive screens that can be directly output and displayed after the target processing system performs processing operations such as decoding, rendering, loading, or computing on the resources to be processed.

[0046] In this embodiment, after determining the target processing system in step S103, the display device uses the target processing system to perform substantive processing operations on the resource to be processed, and obtains the processing result. When the target processing system is a display device system, the display device directly uses its own hardware resources (processor, graphics processor, decoder, memory, etc.) to decode, render, load, or compute the resource to be processed, obtaining processing results such as video streams, audio streams, or interactive screens that can be directly output for display. For example, when the resource to be processed is a movie and the target processing system is a television system, the display device directly calls its own hardware decoder to decode and render the video file, generating continuous display frames as the processing result. When the target processing system is an external smart device system, the display device sends the resource to be processed to the external smart device through a communication connection, and the external smart device uses its own hardware resources to decode, render, or compute the data, and then sends the processed data back to the display device through the communication connection. For example, when the resource to be processed is a large-scale 3D game and the target processing system is an external smart device system, the display device sends the game data to the external smart device, and the processor and graphics processor of the external smart device perform real-time rendering calculations of the game screen. The external smart device then sends the rendered screen frame data back to the display device through a communication connection.

[0047] S105. Display the processing result using the display device.

[0048] In this embodiment, the display device receives or acquires the processing result obtained by the target processing system after processing the resource to be processed, and displays the processing result on the display screen of the display device. When the target processing system is a display device system, the display device directly outputs the processing result obtained by its own system to its display screen for display. For example, when a television system completes decoding a movie video stream, it plays the decoded display frame on the television screen. When the target processing system is an external smart device system, the display device receives the processing result (such as a rendered game frame) returned by the external smart device through a communication connection and displays it on the display device screen. For example, when an external smart device system completes rendering of a large game, it returns the rendered frame to the display device through a wired communication connection, and the display device displays it on its screen after receiving it.

[0049] The technical solution provided in this application involves a display device acquiring resources to be processed and determining their access permissions, resource type, and hardware requirement level. Based on these three parameters, a target processing system is determined between the display device's system and the system of an external intelligent device connected to the display device. The target processing system processes the resources to be processed, and the processing results are displayed on the display device. Compared to existing solutions where a single system drives all functions, leading to resource constraints, this application distributes processing tasks originally concentrated in a single system to the display device system and the external intelligent device system for collaborative execution. This avoids system response lag caused by excessive load on a single system. Furthermore, since the target processing system is dynamically determined between the two systems, if one system malfunctions, processing can be continued by switching to the other system, improving system fault tolerance. Moreover, by reusing the existing processing capabilities of the external intelligent device, no additional hardware needs to be added to the display device, which helps reduce hardware costs.

[0050] Figure 2 A flowchart illustrating an embodiment of another resource processing method provided in this application. Figure 2 The process shown is in Figure 1 Based on the illustrated process, the following steps are included: S201. Determine auxiliary information from the current display information of the display device; S202. The auxiliary information is sent to the external smart device through the communication connection so that the auxiliary information can be displayed on the external smart device.

[0051] For ease of understanding, the following provides a unified explanation of S201-S202: Auxiliary information refers to additional information displayed on the secondary screen, besides the processing results of the resource to be processed (i.e., the core content displayed on the main TV screen). Examples include one or more of the following: time information, weather information, smart home device status information (such as temperature, humidity, and security status), schedule information, family messages, and user photo album content. Auxiliary information is typically permanently displayed and can be continuously presented without frequent user interaction.

[0052] In this embodiment, the external smart device is configured as a secondary display terminal of the display device. The display device and the external smart device have a fixed primary / secondary screen division of labor. The display device is locked as the primary display terminal, used to carry core audio-visual, game, and social media content displayed on a large screen; the external smart device is locked as a fixed physical secondary screen, used to display auxiliary information. Based on this fixed division of labor, the display device determines auxiliary information from the currently displayed information and sends this information to the external smart device via a communication connection for display. The primary / secondary screen division of labor between the display device and the external smart device is fixed and not temporary, unlike traditional screen mirroring technologies where the mobile phone screen acts as a temporary mirroring window and reverts to independence upon disconnection. For example, under normal viewing conditions, the display device acts as the primary display screen playing movies or TV series. Simultaneously, the display device extracts auxiliary information such as weather, time, and smart home device status from the currently displayed information and sends it to the external smart device via a communication connection. The external smart device permanently displays this auxiliary information on its screen, forming a dual-screen collaborative experience of "primary screen entertainment, secondary screen information." For example, users can also set the secondary screen to scroll through the contents of the phone's photo album. The display device determines the album contents from the currently displayed information as auxiliary information based on the user's settings and sends them to an external smart device for scrolling display.

[0053] This solution enables the secondary screen of external smart devices to achieve normalized functional display, rather than a temporary projection window, thereby solidifying the division of labor between the primary and secondary screens and improving the user experience.

[0054] In one embodiment, to enable customization of the content displayed on the secondary screen, step S201 may specifically include the following steps: obtaining custom setting information; and determining the auxiliary information from the currently displayed information based on the custom setting information.

[0055] Custom settings information: refers to information set by the user according to their own needs to indicate the content or display mode of the secondary screen, including: user-specified auxiliary information types (such as weather, time, smart home device status, album content, or one or more) and / or display modes (such as scrolling display, carousel display, persistent display, etc.). Users can input custom settings information through the TV's settings interface, remote control buttons, or voice commands.

[0056] In this embodiment, the display device first obtains the user's input custom settings information, and then filters information matching the user's needs from the currently displayed information as auxiliary information based on the custom settings information. For example, if the user selects "Show only weather and device status" through the TV's settings interface, the display device, after obtaining the custom settings information, filters weather information and smart home device status information from the currently displayed information as auxiliary information and sends them to an external smart device for display. For example, if the user sets "Scroll to display photo album on secondary screen" via voice command, the display device, after obtaining the custom settings information, filters photo album content from the currently displayed information as auxiliary information and sends it to an external smart device for display in a scrolling display manner. For example, the user can also set the secondary screen to display schedules, family messages, etc., and the display device filters and determines the corresponding auxiliary information from the currently displayed information based on the custom settings information.

[0057] This solution allows users to freely customize the content displayed on the secondary screen according to their own needs, thereby achieving personalized customization of the secondary screen display function, meeting the diverse needs of different users for secondary screen display content, and improving the user experience.

[0058] Figure 3 A flowchart illustrating another embodiment of a resource processing method provided in this application. Figure 3 The process shown is in Figure 1 Based on the illustrated process, the following steps are included: S301. Send control commands to the external smart device through the communication connection; S302. Receive the execution status returned by the external intelligent device through the communication connection, wherein the execution status is used to characterize the state of the resource to be processed after the external intelligent device executes the control command; S303. Based on the execution status, update the display content of the processing result on the display device.

[0059] For ease of understanding, S301-S303 are explained uniformly below: Control commands: These are instructions sent by the display device to external smart devices to control the external smart devices to perform specific operations or control the content being processed. Examples include any one or more of the following: playback control commands (play, pause, stop, next track, previous track, etc.), game control commands (directional control, motion control, etc.), and application control commands (switch applications, return to home page, etc.).

[0060] Execution status: refers to the running or playback status information of the resource to be processed after the external smart device executes the control command. It is used to characterize the status change of the resource to be processed after the control command is executed. For example, when the control command is "next song", the execution status is the state after the playback content of the resource to be processed switches from the current song to the next song; when the control command is "pause", the execution status is the state after the resource to be processed switches from the playback state to the paused state.

[0061] In this embodiment, the display device sends a control command to an external smart device via a communication connection. After the external smart device receives and executes the control command, the state of the resource to be processed changes. The external smart device uses the state of the resource to be processed after executing the control command as the execution state and sends it back to the display device via the communication connection. After receiving the execution state, the display device updates the display content of the processing result based on the execution state, so that the current processing result displayed on the display device is synchronized with the state of the resource to be processed after the external smart device executes the control command. For example, when a user issues a "switch to the next episode" command via a TV remote control, the display device sends the control command to the external smart device that is playing a TV series. After the external smart device performs the switching operation, it sends the current episode number, playback progress, and other execution states back to the display device. The display device then switches the display content from the current episode to the next episode and updates the playback progress bar accordingly. For example, when a user issues a "pause" command via a TV remote control, the display device sends the control command to an external smart device that is playing a video. After the external smart device performs the pause operation, it sends the pause status back to the display device, which then pauses the video being played.

[0062] With this solution, users can remotely control the processing of resources by external smart devices using a TV remote control. The content displayed on the TV device is synchronized in real time with the status of the resources to be processed on the external smart device, achieving unified control and status synchronization across devices. This avoids the inconvenience of users needing to operate external smart devices at close range, and improves the convenience of interaction and user experience.

[0063] Figure 4 A flowchart illustrating another embodiment of a resource processing method provided in this application. Figure 4 The process shown is in Figure 1 Based on the illustrated process, the following steps are included: S401. In response to an image acquisition request, the camera of the external smart device is invoked to acquire image data through the communication connection; S402, Receive the image data returned by the external smart device through the communication connection; S403. Display the image data using the display device.

[0064] For ease of understanding, S401-S403 are explained uniformly below: Image acquisition request: This refers to the request information used to trigger the camera of an external smart device to perform image acquisition operations. Image acquisition requests can be initiated by the user via a TV remote control, for example, when the user starts a video call, motion-sensing game, AI fitness course, or takes a photo or video, the display device will generate an image acquisition request accordingly.

[0065] Image data: refers to the raw image information or pre-processed image information collected by the camera of an external smart device in response to an image acquisition request.

[0066] In this embodiment, in response to an image acquisition request, the display device invokes the camera of an external smart device via a communication connection, controlling the camera of the external smart device to start and acquire image data. After acquiring the image data, the camera of the external smart device returns the image data to the display device via the communication connection. The display device receives the image data and displays it on its screen. For example, when a user initiates a video call via a TV remote control, the display device generates an image acquisition request and sends it to the external smart device via the communication connection. The external smart device, in response to the request, invokes its front-facing camera to acquire the user's video feed and transmits the acquired video stream data back to the display device in real time. The display device displays the video stream on its screen, allowing the user to conduct video calls with the other party on the large TV screen, achieving a hands-free video call experience. For example, when a user initiates a motion-sensing game via a TV remote control, the display device generates an image acquisition request and sends it to the external smart device. The external smart device invokes its camera to acquire the user's limb movement image and transmits the acquired limb skeletal data back to the display device in real time. The display device displays the game screen and performs game interaction based on the limb skeletal data.

[0067] This solution eliminates the need for additional cameras on display devices. Video capture and display can be achieved directly using the existing cameras on external smart devices, avoiding the inconvenience caused by display devices lacking cameras or requiring external cameras. Furthermore, when external smart devices are placed in the designated slot on the display device, the camera faces the user directly, ensuring that the user is centered in the shot and that their full body movements are accurately captured, thus enhancing the interactive experience of video calls and motion-sensing games.

[0068] In one embodiment, to enable remote, contactless control of an external smart device from a display device, the method may further include the following steps: receiving a user instruction via a remote control device; parsing the user instruction to determine its instruction type; and when the instruction type corresponds to the instruction type of the external smart device, sending the user instruction to the external smart device via the communication connection.

[0069] Remote control device: refers to an input device used to send user commands to the display device. Remote control devices include, but are not limited to, button remote controls, voice remote controls, pointing remote controls, and mobile terminals with remote control applications installed. Users can initiate user commands through button input, voice input, or pointing input via the remote control device. User command: refers to control information input by the user through the remote control device to control the display device or external smart device to perform specific operations. User commands include, but are not limited to, any one or more of the following: playback control commands (play, pause, switch, etc.), function activation commands (start video call, open application, etc.), and navigation commands (direction selection, return, etc.). Command type: refers to the category of the execution object corresponding to the user command, used to characterize whether the user command is a local command for the display device system or an external command for the external smart device system. Command types include display device local command types and external smart device command types.

[0070] In this embodiment, the display device receives user input commands via a remote control device. These user commands can be button commands, voice commands, or pointing commands. Upon receiving the user command, the display device parses it to determine the command type. When the parsing determines that the command type corresponds to the command type of an external smart device, the display device sends the command to the external smart device via a communication connection, and the external smart device executes the command. For example, if a user issues a voice command, "Play the next song on the phone," via a voice remote control, the display device receives and parses the voice command, determines that its command type corresponds to the command type of an external smart device, and sends the "next song" command to the external smart device via a communication connection, allowing the external smart device to control the currently playing media content to switch to the next song. Similarly, if a user presses the "back" button via a button remote control, the display device parses the command type of the button input. When the command type corresponds to the command type of an external smart device, the display device sends the "back" command to the external smart device, allowing the external smart device to return to the previous screen. For example, a user can select and click an application icon of an external smart device on the TV screen by pointing the remote control at it. The display device parses the pointing instruction, determines that the instruction type is the same as the instruction type of the external smart device, and sends the instruction to launch the application to the external smart device through the communication connection, so that the external smart device can launch the corresponding application.

[0071] With this solution, users can remotely control external smart devices using a TV remote control without needing to touch them at close range. The TV remote control itself has multiple input methods such as buttons, voice, and pointing, providing flexible and diverse control options. This enables cross-device, full-domain control, avoiding the inconvenience of switching between the TV remote control and external smart devices, and improving the convenience of interaction and user experience.

[0072] Figure 5 This is a block diagram illustrating an embodiment of a resource processing apparatus provided in this application. (See diagram below.) Figure 5 As shown, the device includes: Module 51 is used to acquire resources to be processed; The first determining module 52 is used to determine the resource access permission, resource type and hardware requirement level corresponding to the resource to be processed; The second determining module 53 is used to determine, based on the resource access permissions, whether only one of the systems of the display device and the external smart device has permission to access the resource to be processed; when only one has permission to access the resource to be processed, the system with the access permission is determined as the target processing system; when both have access permissions or neither has access permissions, the target processing system is determined based on the resource type, wherein: when the resource type is a first type, the system of the display device is determined as the target processing system; when the resource type is a second type, the system of the external smart device is determined as the target processing system; when the resource type is a third type, the target processing system is determined based on the hardware requirement level. Processing module 54 is used to process the resource to be processed using the target processing system to obtain a processing result; Display module 55 is used to display the processing result using the display device.

[0073] In one possible implementation, the apparatus further includes a transmitting module for: Determine auxiliary information from the current display information of the display device; The auxiliary information is sent to the external smart device via the communication connection so that the auxiliary information can be displayed on the external smart device.

[0074] In one possible implementation, the sending module is further configured to: Get custom settings information; The auxiliary information is determined from the currently displayed information based on the custom settings.

[0075] In one possible implementation, the device further includes an update module for: Control commands are sent to the external intelligent device through the communication connection; The system receives the execution status returned by the external intelligent device through the communication connection, and the execution status is used to characterize the state of the resource to be processed after the external intelligent device executes the control command; Based on the execution status, update the display content of the processing result on the display device.

[0076] In one possible implementation, the device further includes a calling module for: In response to an image acquisition request, the camera of the external smart device is invoked to acquire image data via the communication connection; Receive the image data returned by the external smart device through the communication connection; The image data is displayed using the display device.

[0077] In one possible implementation, the device further includes a distribution module for: Receive user commands via remote control device; Parse the user instruction to determine the instruction type of the user instruction; When the instruction type is the same as the instruction type corresponding to the external smart device, the user instruction is sent to the external smart device through the communication connection.

[0078] like Figure 6 As shown in the figure, this application provides a device including a processor 111, a communication interface 112, a memory 113, and a communication bus 114, wherein the processor 111, the communication interface 112, and the memory 113 communicate with each other through the communication bus 114. Memory 113 is used to store computer programs; In one embodiment of this application, the processor 111, when executing a program stored in the memory 113, implements the resource processing method provided in any of the foregoing method embodiments, applied to a display device, the display device being communicatively connected to an external smart device, including: Acquire resources to be processed; Determine the resource access permissions, resource type, and hardware requirement level corresponding to the resource to be processed; Based on the resource access permissions, it is determined whether only one of the display device's system and the external smart device's system has permission to access the resource to be processed; if only one has permission to access the resource to be processed, the system with the access permission is determined as the target processing system; if both have access permissions or neither has access permissions, the target processing system is determined based on the resource type, wherein: when the resource type is a first type, the display device's system is determined as the target processing system; when the resource type is a second type, the external smart device's system is determined as the target processing system; when the resource type is a third type, the target processing system is determined based on the hardware requirement level. The target processing system is used to process the resource to be processed to obtain the processing result; The processing result is displayed using the display device.

[0079] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the resource processing method provided in any of the foregoing method embodiments.

[0080] This application also provides a computer program product containing instructions that, when run on a computer, cause the computer to perform any of the resource processing methods described in the above embodiments.

[0081] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0082] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented using software plus a general-purpose hardware platform, or of course, using hardware. Based on this understanding, the above technical solutions, in essence or the parts that contribute to the related technology, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0083] It should be understood that the terminology used herein is for the purpose of describing particular exemplary embodiments only and is not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms “a,” “an,” and “described” as used herein may also include the plural forms. The terms “comprising,” “including,” “containing,” and “having” are inclusive and therefore indicate the presence of the stated features, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, elements, components, and / or combinations thereof. The method steps, processes, and operations described herein are not construed as requiring them to be performed in a particular order described or illustrated unless the order of performance is explicitly indicated. It should also be understood that additional or alternative steps may be used.

[0084] The above description is merely a specific embodiment of this application, enabling those skilled in the art to understand or implement this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A resource processing method, characterized in that, Applied to a display device that is communicatively connected to an external smart device, the method includes: Acquire resources to be processed; Determine the resource access permissions, resource type, and hardware requirement level corresponding to the resource to be processed; Based on the resource access permissions, it is determined whether only one of the display device's system and the external smart device's system has permission to access the resource to be processed; if only one has permission to access the resource to be processed, the system with the access permission is determined as the target processing system; if both have access permissions or neither has access permissions, the target processing system is determined based on the resource type, wherein: when the resource type is a first type, the display device's system is determined as the target processing system; when the resource type is a second type, the external smart device's system is determined as the target processing system; when the resource type is a third type, the target processing system is determined based on the hardware requirement level. The target processing system is used to process the resource to be processed to obtain the processing result; The processing result is displayed using the display device.

2. The method according to claim 1, characterized in that, The method further includes: Determine auxiliary information from the current display information of the display device; The auxiliary information is sent to the external smart device via the communication connection so that the auxiliary information can be displayed on the external smart device.

3. The method according to claim 2, characterized in that, Determining auxiliary information from the current display information of the display device includes: Get custom settings information; The auxiliary information is determined from the currently displayed information based on the custom settings.

4. The method according to claim 1, characterized in that, The method further includes: Control commands are sent to the external intelligent device through the communication connection; The system receives the execution status returned by the external intelligent device through the communication connection, and the execution status is used to characterize the state of the resource to be processed after the external intelligent device executes the control command; Based on the execution status, update the display content of the processing result on the display device.

5. The method according to claim 1, characterized in that, The method further includes: In response to an image acquisition request, the camera of the external smart device is invoked to acquire image data via the communication connection; Receive the image data returned by the external smart device through the communication connection; The image data is displayed using the display device.

6. The method according to claim 1, characterized in that, The method further includes: Receive user commands via remote control device; Parse the user instruction to determine the instruction type of the user instruction; When the instruction type is the same as the instruction type corresponding to the external smart device, the user instruction is sent to the external smart device through the communication connection.

7. A resource processing device, characterized in that, Applied to a display device that communicates with an external smart device, the device includes: The acquisition module is used to acquire resources to be processed. The first determining module is used to determine the resource access permissions, resource type, and hardware requirement level corresponding to the resource to be processed; The second determining module is configured to determine, based on the resource access permissions, whether only one of the systems of the display device and the external smart device has permission to access the resource to be processed; when only one has permission to access the resource to be processed, the system with the access permission is determined as the target processing system; when both have access permissions or neither has access permissions, the target processing system is determined based on the resource type, wherein: when the resource type is a first type, the system of the display device is determined as the target processing system; when the resource type is a second type, the system of the external smart device is determined as the target processing system; when the resource type is a third type, the target processing system is determined based on the hardware requirement level. The processing module is used to process the resource to be processed using the target processing system to obtain the processing result; The display module is used to display the processing result using the display device.

8. An electronic device, characterized in that, include: A processor and a memory, the processor being configured to execute a resource processing program stored in the memory to implement the resource processing method of any one of claims 1-6.

9. A storage medium, characterized in that, The storage medium stores one or more programs, which can be executed by one or more processors to implement the resource processing method according to any one of claims 1-6.

Images