Cloud desktop data interaction method and electronic device

Abstract

This application provides a cloud desktop data interaction method and an electronic device, relating to the field of data processing. The method includes: responding to an input operation by sending an instruction data packet, the instruction data packet including encrypted data corresponding to privacy data and non-privacy data; receiving a first invocation instruction; based on the first invocation instruction, decrypting the encrypted data using decryption parameters, and obtaining a privacy data processing result based on the decrypted privacy data; receiving the non-privacy data processing result, and synchronously presenting a screen in conjunction with the privacy data processing result. This application distinguishes between privacy data and non-privacy data, improving data security by processing privacy data locally and improving data processing efficiency by processing non-privacy data in the cloud.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of data processing, and particularly relates to a cloud desktop data interaction method and an electronic device. BACKGROUND

[0002] Desktop virtualization refers to virtualizing a terminal-side desktop, which can be accessed through a network at any time and in any place.

[0003] However, in a traditional cloud desktop scenario, when a user accesses a cloud server through a terminal, the cloud desktop system mainly provides basic image transmission functions, and lacks intelligent analysis and automatic processing capabilities for user operation data.

[0004] Therefore, a new cloud desktop data interaction method is urgently needed to solve the above problems. SUMMARY

[0005] The present application provides a cloud desktop data interaction method and an electronic device, which distinguishes between private data and non-private data, processes private data locally to improve data security, and processes non-private data in the cloud to improve data processing efficiency.

[0006] According to a first aspect of the present application, a cloud desktop data interaction method is provided, applied to a terminal, the terminal being network-connected with a cloud server, and the method comprising: in response to an input operation, sending an instruction data packet, the instruction data packet comprising encrypted data corresponding to private data and non-private data; receiving a first calling instruction, the first calling instruction containing a decryption parameter; Based on the first calling instruction, the encrypted data is decrypted using the decryption parameter, and a private data processing result is obtained based on the decrypted private data; receiving a non-private data processing result, and synchronously presenting a picture in combination with the private data processing result.

[0007] In one embodiment, after responding to the input operation and before sending the instruction data packet, the method further comprises: encrypting the private data to obtain the encrypted data; obtaining the instruction data packet based on the encrypted data and the non-private data.

[0008] In one embodiment, before encrypting the private data, the method further comprises: performing text recognition on input content corresponding to the input operation, and extracting text content; analyzing the text content and determining whether the text content belongs to the private data or the non-private data.

[0009] In an embodiment, before the privacy data is encrypted, the method further comprises: performing text recognition on input content corresponding to the input operation, and extracting text content; performing analysis on the text content, and determining that the text content belongs to the privacy data or the non-privacy data.

[0010] In an embodiment, the terminal is further deployed with a BERT-base light model and an NLP model. The analysis on the text content comprises: using the BERT-base light model and the NLP model to analyze the text content.

[0011] In an embodiment, the non-privacy data processing result is obtained by calling a public cloud model service to process the non-privacy data by the cloud server.

[0012] In an embodiment, the method further comprises: generating and executing a standardized instruction set, the standardized operation instruction set comprising at least one instruction of an input class, a click class, a navigation class, and a data processing class.

[0013] In an embodiment, the terminal and the cloud server transmit data through a VGTP protocol.

[0014] In a second aspect of the embodiments of the present application, a cloud desktop data interaction method is further provided, applied to a cloud server, the method comprising: receiving a post-analysis instruction data packet, and determining a calling instruction based on an analysis result; the instruction data packet comprising encrypted data corresponding to privacy data and non-privacy data, the calling instruction comprising a first calling instruction corresponding to the privacy data and a second calling instruction corresponding to the non-privacy data, the second calling instruction being used to instruct calling a public cloud model service to process the non-privacy data to obtain a non-privacy processing result; issuing the first calling instruction and the non-privacy data processing result to a terminal; the first calling instruction containing a decryption parameter, the first calling instruction being used to instruct the terminal to use the decryption parameter to decrypt the encrypted data, and to obtain a privacy data processing result based on the decrypted privacy data.

[0015] In a third aspect of the embodiments of the present application, an electronic device is further provided, comprising a processor and a memory, the memory storing at least one computer instruction, the instruction being loaded and executed by the processor to execute the method of the first aspect and any one of the embodiments, and / or the method of the second aspect and any one of the embodiments.

[0016] In a fourth aspect of the embodiments of this application, a cloud desktop system is also provided, including multiple terminals connected via a network and a server as described in the third aspect, wherein multiple cloud desktops are deployed in the server; wherein the terminals can execute the methods described in the first aspect and any one thereof, and the server can execute the methods described in the second aspect and any one thereof.

[0017] The cloud desktop data interaction method provided in this application embodiment can achieve the following improvements: First, privacy and data security: Sensitive information (such as passwords and accounts) is processed through a localized model, avoiding the risks of public cloud transmission and achieving data encryption. Second, dynamic model scheduling: Local or public cloud models are dynamically selected based on data sensitivity, improving the efficiency of non-sensitive data processing and reducing resource waste. Third, real-time response capability: A high-efficiency transmission mechanism based on the VGTP protocol ensures millisecond-level response speed of the AI ​​decision-making module in the cloud desktop environment. Fourth, functional completeness: Modules such as OCR and UI element detection are integrated, supporting complex operations (such as automatic login and product search), covering enterprise-level application scenarios. Fifth, security audit mechanism: Operation logs are stored using blockchain to ensure immutability and meet enterprise compliance requirements. Sixth, cross-platform compatibility: Support for switching APIs from multiple cloud vendors, compatible with different cloud service environments, and improving system flexibility. Seventh, user transparency: The model switching process is transparent to the user, and the continuity of operation is not affected, improving the user experience.

[0018] In a fifth aspect of the embodiments of this application, a computer-readable storage medium is also provided, the storage medium storing at least one computer instruction, the instruction being loaded by a processor and executing the methods described in the first aspect and any one thereof, and / or the methods described in the second aspect and any one thereof.

[0019] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0020] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0021] Figure 1 This is a schematic diagram of the structure of a cloud desktop system provided by related technologies; Figure 2 This is a schematic diagram of the structure of a cloud desktop system provided in an embodiment of this application; Figure 3 This is a flowchart illustrating a cloud desktop data interaction method provided in an embodiment of this application; Figure 4This is an interactive schematic diagram of a cloud desktop data interaction method provided in an embodiment of this application. Detailed Implementation

[0022] To further illustrate the technical means and effects adopted by this application in order to achieve the intended purpose of the invention, the following detailed description of the specific implementation methods, structures, features and effects of this application is provided in conjunction with the accompanying drawings and preferred embodiments.

[0023] Cloud desktop is a desktop delivery model based on cloud computing and virtualization technologies. This technology runs the user's desktop operating system and application software on physical servers in a remote data center, and uses virtualization technology to divide the server's computing resources (such as CPU, memory, storage, and network) into multiple isolated virtual desktop instances. Users can remotely access their own virtual desktop (also known as a cloud desktop) via a network using terminals such as thin clients, personal computers, laptops, tablets, or mobile devices, achieving an operating experience consistent with their local desktop.

[0024] For example, Figure 1 A schematic diagram of the structure of a cloud desktop system provided for related technologies.

[0025] like Figure 1 As shown, the cloud desktop system may include: multiple terminals (also referred to as zero terminals or local hosts) and a cloud server, wherein one or more virtual machines may run on the cloud server. Each terminal connects to one of the virtual machines on the cloud server to exchange data with that virtual machine.

[0026] Cloud desktop systems offer advantages such as centralized management, flexible deployment, ease of maintenance, and secure control, and are widely used in scenarios such as enterprise offices, education and training, and government offices.

[0027] Combination Figure 1 As shown, in existing technologies connecting local hosts to multiple cloud desktops, cloud desktop systems primarily provide basic graphics transmission functions, lacking the ability to intelligently analyze and automatically process user operation data. Currently, a solution of installing an AI assistant on a cloud server has been proposed, but this solution faces the following two challenges.

[0028] First, there is the risk of privacy data leakage. User-input data may contain sensitive information such as passwords and account details. Directly accessing an AI assistant installed on a cloud server could pose security risks. Second, there are functional limitations. The AI ​​assistant cannot dynamically select processing methods based on the sensitivity of the data, resulting in low efficiency in processing non-privacy data.

[0029] In view of this, this application provides a novel cloud desktop data interaction method. The terminal categorizes data into private and non-private data before sending it to a cloud server. The cloud server parses the data and sends a command to the terminal for processing the private data, which is then handled by a local model on the terminal. For non-private data, an AI model on the cloud server is invoked for processing. This approach ensures the security of private data while improving the processing efficiency of non-private data.

[0030] The following is combined with Figures 2 to 4 The embodiments provided in this application will be described in detail.

[0031] Figure 2 This is a schematic diagram of the structure of a cloud desktop system provided in an embodiment of this application.

[0032] The cloud desktop system provided in this application embodiment may include multiple terminals (local hosts) and a cloud server. Multiple virtual machines may run in the cloud server, and each virtual machine corresponds to a cloud desktop.

[0033] like Figure 2 As shown, taking a local host connecting to a cloud desktop through a cloud server as an example, the local host can transfer data with the cloud desktop.

[0034] Optionally, the local host may include a transport layer, a model layer, and a processing layer.

[0035] The transport layer is used to communicate with the cloud server and establish a connection with the cloud desktop. For example, the transport layer is used to retrieve the cloud server desktop screen in real time based on the VGTP protocol, which supports encrypted transmission and dynamic resolution adaptation.

[0036] Optionally, the model layer may include a recognition module, a parsing module, a data type discrimination module, and an operation instruction generation and execution module. The model layer is a lightweight intelligent core on the terminal.

[0037] (1) The recognition module is used to recognize the text content in the received cloud image. The recognition module may include an object detection model and a deep learning model.

[0038] Optionally, the object detection model can be a lightweight YOLOv8 model, and the deep learning model can be a model that implements Optical Character Recognition (OCR) technology.

[0039] For example, the YOLOv8 lightweight model is used to recognize UI elements in the screen, such as buttons, input boxes, drop-down menus, pop-ups, and other common controls. OCR technology (such as Tesseract 5.0 + self-developed optimized algorithm) is used to recognize the text content in UI elements, supporting mixed recognition of Chinese and English, numbers, special symbols, and handwriting.

[0040] In this embodiment, the YOLOv8 lightweight model can directly locate elements such as buttons, input boxes, and drop-down menus on the screen through visual recognition, with a recognition accuracy of ≥99.2%. It does not rely on underlying code, making it more robust. Furthermore, the model is small in size and has a fast inference speed. Combining the YOLOv8 lightweight model with OCR technology can automatically extract data from the screen.

[0041] The aforementioned recognition module can also be used to recognize user-inputted speech content. For example, it can be based on WebRTC real-time transcription to convert user-inputted speech content into text content, achieving a "speak and output" effect.

[0042] (2) The parsing module is used to perform semantic understanding and intent parsing on the text content. The parsing module may include BERT-based lightweight models, NLP models, etc.

[0043] For example, the BERT-base lightweight model is used to perform semantic understanding and intent parsing on the recognition results of the recognition module, such as understanding and parsing the text content recognized in UI elements, and / or understanding and parsing the text content recognized from speech content.

[0044] Optionally, in the parsing module, text fragments of "suspected targets" can be quickly filtered out based on regular expression matching, and then contextual semantic verification can be performed. In this way, logical judgment can be made on the matching results to filter out mismatches that do not conform to semantic rules.

[0045] (3) The data type discrimination module is used to discriminate the results parsed by the parsing module (i.e., the parsed text content) to determine whether the data type is private or non-private. After determination, the data type is marked, which indicates whether the data type is private or non-private.

[0046] Alternatively, the data type discrimination module can further subdivide data types and classify them according to a multi-level sensitivity classification system. For example, the built-in three-level sensitivity classification system can be used to classify data types into three types: high-sensitivity data, medium-sensitivity data, and non-sensitive data.

[0047] It should be understood that highly sensitive data may include account passwords, ID card numbers, bank card numbers, trade secrets, etc.; moderately sensitive data may include departmental accounts receivable data, project progress information, etc.; and non-sensitive data may include public document queries, general command operations, etc.

[0048] Here, enterprises or users can customize sensitive word libraries, such as importing, exporting, or updating sensitive word libraries in real time. The sensitivity classification system can be uniformly configured through the cloud server.

[0049] (4) Operation instruction generation and execution module, used to generate standardized operation instruction sets, such as input, click, navigation, and data processing instructions, which can also be called AI instructions. Supports virtual keyboard simulation (avoiding system keyboard hook detection), precise mouse coordinate positioning, and shortcut key combination execution.

[0050] It should be noted that the operation command generation and execution module has a built-in operation conflict detection mechanism. When the user's manual operation conflicts with the AI ​​command, the user's manual operation will be given priority and the task corresponding to the AI ​​command will be suspended.

[0051] The processing layer is used for data interaction with the model layer. The processing layer may include a privacy data processing module and a non-privacy processing module. The privacy data processing module is used to process privacy-sensitive data, or high-sensitivity data and medium-sensitivity data. In practice, different processing steps can be performed for high-sensitivity data and medium-sensitivity data.

[0052] For example, a strategy of "segmented encryption and hardware binding" can be used for privacy-sensitive or highly sensitive data, while a strategy of "overall encryption and hardware binding" can be used for medium-sensitive data. In this way, core information such as account passwords can be stored in segments in an encrypted partition on the local host, or stored entirely in an encrypted area on the local host and bound to a hardware device via fingerprint, preventing migration to other terminals. The decryption key can then be dynamically distributed by a cloud server and set to expire immediately after a single use.

[0053] The privacy data processing module can also interact with the operation instruction generation and execution module of the model layer to support automated operations, such as: automatic form filling, login verification, privacy data anonymization display (e.g., replacing the middle 8 digits of the ID card number with *), and decompressing encrypted files.

[0054] The non-privacy processing module is used to process non-privacy or non-confidential data. For example, it can call public cloud model services on cloud servers through an API gateway. The API gateway is deployed on cloud servers, and public cloud model services can include Alibaba Cloud PAI and Tencent Cloud TI-ONE. The non-privacy processing module supports Natural Language Generation (NLG), data visualization, intelligent search, and automatic SQL generation. API communication uses HTTPS encryption, and request parameters include timestamps and signature verification to prevent unauthorized calls.

[0055] In addition, the non-privacy processing module also supports local caching of hot data, such as frequently used search keywords and fixed-format report templates. The cache validity period can be configured as needed, such as the default of 1 hour, which can improve response speed.

[0056] Based on the local host architecture, when the data type discrimination module detects private data, it can initiate a local model sandbox environment. This sandbox environment, combined with operation commands, generates and executes models to perform automated operations, such as automatically filling out forms and performing login verification. It should be understood that the local model sandbox environment is an isolated, controlled local computing space. Operation logs are stored in encrypted form.

[0057] When non-privacy data is detected using the data type discrimination module, the public cloud API is called through the HTTPS channel to receive the returned automated instructions, and the automated instructions are injected through the VGTP protocol transport layer.

[0058] It should be understood that a cloud server can be a single server or a server cluster, and this application embodiment does not limit this.

[0059] Optionally, the cloud server may include a VGTP server, an AI decision-making module, and a public cloud model interface. The VGTP server establishes a protocol-peer connection with the local host, supporting pixel-level image transmission and command data interaction. The AI ​​decision-making module has the capabilities of command parsing, task type judgment, and model invocation scheduling, and also connects to the public cloud model service through the public cloud model interface.

[0060] Figure 3 and Figure 4 This illustration shows a flowchart of a cloud desktop data interaction method provided in an embodiment of this application, which can be applied to the above. Figure 2 The cloud desktop system shown. For example... Figure 3 As shown, the cloud desktop data interaction method 10 includes the following steps S10 to S40, which will be described in detail below.

[0061] S10. In response to the input operation, the local host sends the instruction data packet to the cloud server.

[0062] Input operations are used to indicate input operations performed by the user on the local host, such as text input operations and voice input operations.

[0063] It should be understood that cloud desktops are deployed on cloud servers, and local hosts establish connections with cloud desktops through these servers. Then, via the VGTP protocol, they can receive and display the screen transmitted by the server.

[0064] Optionally, in response to an input operation, target processing is performed on the input content corresponding to the input operation. Target processing may include recognition processing, parsing processing, data type discrimination processing, and encryption processing.

[0065] It should be understood that the identification process can be invoked. Figure 2 The recognition module shown can be executed, and the parsing process can be called. Figure 2 The parsing module shown can be executed, and the data type discrimination process can be called. Figure 2 The data type discrimination module shown is executed.

[0066] It should be understood that after performing targeted processing on the input content, the local host can determine whether the input content contains private or non-private data, and encrypt the input content marked as private data according to a preset encryption method. For more information, please refer to [link / reference needed]. Figure 2 As shown, different encryption methods are used to encrypt content with different sensitivity levels.

[0067] Then, the encrypted data obtained by encrypting the privacy data on the local host, along with the non-privacy data, which is the integrated instruction data packet, is transmitted to the cloud server via the VGTP protocol.

[0068] S20: The local host receives the first call instruction and non-privacy data processing results issued by the cloud server.

[0069] It should be understood that after receiving the instruction data packet, the cloud server parses it and determines the calling instruction based on the parsing result. The calling instruction includes a first calling instruction corresponding to the privacy data and a second calling instruction corresponding to the non-privacy data.

[0070] The first invocation instruction contains decryption parameters. The first invocation instruction is used to instruct the terminal to decrypt the encrypted data using the decryption parameters and perform automated operations on the decrypted private data.

[0071] Automated operations can be called Figure 2 The operation instruction generation and execution module shown is executed.

[0072] The second invocation command is used to instruct the public cloud model service to process non-privacy data.

[0073] S30. Based on the first call instruction, the local host decrypts the encrypted data using the decryption parameters and performs automated operations on the decrypted privacy data to obtain the privacy data processing result.

[0074] S40: The local host receives the results of processing non-privacy data and combines them with the results of processing privacy data to present the screen synchronously.

[0075] For example, consider a corporate office scenario. In this scenario, the local host serves as the employee's terminal, such as a zero-terminal device, and is embedded with a VGTP protocol chip. The cloud server is an enterprise cloud server cluster. The local host is pre-installed with a lightweight AI model during initialization, while the cloud server deploys the VGTP server, AI decision-making module, and associated public cloud model services.

[0076] The first step is for employees to input information through their local host, such as through voice or text input, instructing them to "Log in to the enterprise OA system xxx, username xxx, password xxx, and search for the latest industry policies and add them to the document".

[0077] The second step involves the local host using a locally deployed lightweight AI model to identify and analyze the data, determining that the OA system address, username, and password are private data, while "searching for the latest industry policies and adding them to documents" is non-private data. The private data is then encrypted with a dynamic password and integrated with the non-private data into a data instruction package, which is then transmitted to the cloud server via the VGTP protocol.

[0078] Thirdly, after the AI ​​decision-making module on the cloud server parses the data instruction packet, it sends a first invocation command to the local host. The local host uses the decryption parameters carried in the first invocation command to decrypt the privacy parameters and automatically complete the OA login operation. Simultaneously, the cloud server also sends a second invocation command to invoke the public cloud model service to obtain industry policies, generate execution instructions to add to the document, and transmit them to the local host via the VGTP protocol, synchronously updating the cloud desktop screen. This enables secure and efficient completion of office operations.

[0079] The cloud desktop data interaction method provided in this application embodiment can achieve the following improvements: First, privacy and data security: Sensitive information (such as passwords and accounts) is processed through a localized model, avoiding the risks of public cloud transmission and achieving data encryption. Second, dynamic model scheduling: Local or public cloud models are dynamically selected based on data sensitivity, improving the efficiency of non-sensitive data processing and reducing resource waste. Third, real-time response capability: A high-efficiency transmission mechanism based on the VGTP protocol ensures millisecond-level response speed of the AI ​​decision-making module in the cloud desktop environment. Fourth, functional completeness: Modules such as OCR and UI element detection are integrated, supporting complex operations (such as automatic login and product search), covering enterprise-level application scenarios. Fifth, security audit mechanism: Operation logs are stored using blockchain to ensure immutability and meet enterprise compliance requirements. Sixth, cross-platform compatibility: Support for switching APIs from multiple cloud vendors, compatible with different cloud service environments, and improving system flexibility. Seventh, user transparency: The model switching process is transparent to the user, and the continuity of operation is not affected, improving the user experience.

[0080] like Figure 4 As shown, step S10 may include steps S101 to S112.

[0081] S101, The local host sends a screen acquisition request to the cloud server.

[0082] Optionally, in response to a request, after the VGTP server establishes a protocol peering connection with the local host, the local host can send a frame request to the cloud server via the VGTP protocol. The local host is used to receive request operations; the request operation can indicate a startup operation for the cloud desktop client software deployed on the local host, or, if the cloud desktop client software is set to start automatically at boot, the request operation can indicate a boot operation for the local host.

[0083] S102: The cloud server encodes and captures the real-time screen of the cloud desktop and transmits it to the local host via the VGTP protocol.

[0084] Optionally, the cloud server can use a hardware interface to capture the real-time screen of the cloud desktop, and after adaptive compression encoding, transmit it to the local host via the VGTP protocol.

[0085] S103, the local host receives and displays the cloud desktop screen.

[0086] Optionally, the local host can decode the received screen data and display the real-time cloud desktop screen on the interface, maintaining a preset screen refresh rate during transmission to ensure real-time interaction. For example, maintaining a screen refresh rate of 60 frames per second.

[0087] S104. In response to the input operation, the local host performs text recognition on the input content and extracts the text content.

[0088] Input operations are used to indicate input operations performed by the user on the local host, such as text input operations performed through the text input module or voice input operations performed through the voice input module.

[0089] The recognition module on the local host can perform text recognition on the input content and extract the text content; it can also perform text recognition on the cloud desktop screen and extract the text content.

[0090] S105. The local host parses the text content and determines the data type.

[0091] The parsing module on the local host parses the extracted text content and transmits the parsing results to the data type discrimination module. The data type discrimination module then judges the parsing results to determine whether the data is private or non-private. Private data may include highly sensitive and moderately sensitive data such as usernames, passwords, and identity information; non-private data may include non-sensitive data such as public information queries and general operation instructions.

[0092] S106. The local host generates an instruction data packet by combining the data type and uploads it to the cloud server.

[0093] The privacy data processing module on the local host can encrypt privacy data, such as using a dynamic key encryption mechanism, to generate encrypted data. The non-privacy data processing module can maintain the original format of the non-privacy data. Then, the local host can integrate the encrypted data packets and non-privacy data into a command data packet, which is then transmitted to the cloud server via the VGTP protocol.

[0094] S107. The cloud server parses the instruction data packet and determines the calling instruction based on the parsing result.

[0095] The local host can upload the instruction data packet to the VGTP server in the cloud server, and then the VGTP server forwards it to the AI ​​decision module; the AI ​​decision module parses the instruction data packet and distinguishes between privacy data association operations and non-privacy data association operations.

[0096] Operations involving association with non-privacy data include item search, adding items to the shopping cart, and querying publicly available information. Instructions for operations involving association with privacy data can be referred to as first invocation instructions, and instructions for operations involving association with non-privacy data can be referred to as second invocation instructions.

[0097] S108. For the privacy data in the parsing results, the cloud server sends the first call instruction to the local host.

[0098] Optionally, for privacy-sensitive data, a first invocation command can be issued to the local host via the VGTP protocol to instruct the local host to perform automated operations on the privacy-sensitive data. This first invocation command may include decryption parameters for the encrypted privacy-sensitive data.

[0099] S109. In response to the first call instruction, the local host performs automated operations on the privacy data and obtains the privacy data processing results.

[0100] Optionally, upon receiving the first invocation command, the local host can respond by decrypting the encrypted result corresponding to the privacy data using the decryption parameters, and then execute the corresponding automated operation. This operation is completed using local computing power and does not upload the original privacy data to the cloud server, thus ensuring high security.

[0101] S110. For non-privacy data in the parsing results, the cloud server sends a second call instruction.

[0102] The second invocation instruction is used to instruct the public cloud model service to process non-privacy data through the public cloud model service interface.

[0103] S111, the public cloud model service processes non-privacy data and returns the non-privacy data processing results to the local host.

[0104] The public cloud model service can return the non-privacy data processing results to the AI ​​decision-making module of the cloud server. After integrating the processing results, the AI ​​decision-making module returns the integrated non-privacy data processing results to the local host through the VGTP protocol.

[0105] It should be understood that public cloud model services utilize cloud computing power to complete complex calculations and logical analyses, generating operation results and execution instructions that contain non-privacy data.

[0106] The steps S108 and S110 described above can be performed sequentially or simultaneously, and this application does not limit this.

[0107] S112. The local host receives the non-privacy data processing results and, in conjunction with the privacy data processing results, synchronously presents the screen.

[0108] After the local host receives the non-privacy data processing result, the operation instruction generation and execution module in the model layer can execute the corresponding interface feedback operation. At the same time, the updated cloud desktop real-time screen is pulled through the VGTP protocol to realize the synchronous presentation of operation results and screen.

[0109] Based on the cloud desktop data interaction method described in the above embodiments, this application also provides an electronic device, including a processor and a memory, wherein the memory stores at least one computer instruction, which is loaded and executed by the processor to perform the steps in the cloud desktop data interaction method described in any of the above embodiments.

[0110] Based on the cloud desktop data interaction method described in the above embodiments, this application also provides a computer-readable storage medium. For example, a non-transitory computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a CD-ROM, magnetic tape, a floppy disk, or an optical data storage device. This storage medium stores computer instructions for executing the steps in the cloud desktop data interaction method described in any of the above embodiments, which will not be elaborated further here.

[0111] The above description is merely a preferred embodiment of this application and is not intended to limit this application in any way. Although this application has been disclosed above with reference to preferred embodiments, it is not intended to limit this application. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the technical solution of this application. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of this application without departing from the scope of the technical solution of this application shall still fall within the scope of the technical solution of this application.

Claims

1. A cloud desktop data interaction method, characterized in that, Applied to a terminal that is connected to a cloud server network, the method includes: In response to an input operation, a command data packet is sent, the command data packet including encrypted data and non-privacy data corresponding to privacy data; Receive a first invocation instruction, which includes decryption parameters; Based on the first invocation instruction, the encrypted data is decrypted using the decryption parameters, and the privacy data processing result is obtained based on the decrypted privacy data. The system receives the results of processing non-privacy data and combines them with the results of processing privacy data to simultaneously display the screen.

2. The method according to claim 1, characterized in that, The method further includes, after responding to an input operation and before sending an instruction data packet: The privacy data is encrypted to obtain the encrypted data; The instruction data packet is obtained based on the encrypted data and the non-privacy data.

3. The method according to claim 2, characterized in that, Before encrypting the privacy data, the method further includes: The input content corresponding to the input operation is subjected to text recognition, and the text content is extracted; The text content is parsed to determine whether it belongs to the private data or the non-private data.

4. The method according to claim 3, characterized in that, The terminal is equipped with a lightweight YOLOv8 model and a model based on OCR technology. The input content corresponding to the input operation is subjected to text recognition, and the text content is extracted, including: Using the YOLOv8 lightweight model and the OCR-based model, text recognition is performed on the data content corresponding to the input operation, and the text content is extracted.

5. The method according to claim 3, characterized in that, The terminal is also equipped with a BERT-based lightweight model and an NLP model. The text content is parsed, including: The text content is parsed using the BERT-base lightweight model and the NLP model.

6. The method according to any one of claims 1 to 5, characterized in that, The non-privacy data processing result is obtained by the cloud server calling the public cloud model service to process the non-privacy data.

7. The method according to claim 6, characterized in that, The method further includes: Generate and execute a standardized instruction set, wherein the standardized operation instruction set includes at least one type of instruction from the categories of input, click, navigation, and data processing.

8. The method according to claim 7, characterized in that, The terminal and the cloud server transmit data via the VGTP protocol.

9. A cloud desktop data interaction method, characterized in that, Applied to a cloud server, the method includes: receiving and parsing an instruction data packet, and determining a call instruction based on the parsing result; the instruction data packet includes encrypted data and non-private data corresponding to privacy data, and the call instruction includes a first call instruction corresponding to the privacy data and a second call instruction corresponding to the non-private data, wherein the second call instruction is used to instruct a public cloud model service to process the non-private data to obtain a non-private processing result; The system sends the first invocation instruction and the non-privacy data processing result to the terminal. The first invocation instruction includes decryption parameters and is used to instruct the terminal to decrypt the encrypted data using the decryption parameters and obtain the privacy data processing result based on the decrypted privacy data.

10. An electronic device, characterized in that, It includes a processor and a memory, the memory storing at least one computer instruction, the instruction being loaded and executed by the processor to perform the method as claimed in any one of claims 1 to 7, and / or the method as claimed in claim 8 or 9.

Images