Generation method, device and equipment of security keyboard and storage medium

By deploying the secure keyboard on the server and rendering it on the client, the problem that traditional secure keyboards are easily cracked is solved, data security and resource utilization are improved, personalized needs are met, and the normal push and pull of video streams are maintained.

CN119066723APending Publication Date: 2024-12-03TENCENT TECHNOLOGY (SHENZHEN) CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202310627313.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-30
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

The software toolkit of traditional security keyboards is easily cracked, resulting in reduced security of input or transmission of data, and it is difficult to prevent screen recording and screen capture and other operational behaviors.

Method used

Deploy the secure keyboard on the server, and pass the rendering parameters to the client in the form of a keyboard screen to avoid the impact of abnormal operation behavior, and do not need a software toolkit with built-in secure keyboard on the client, reducing resource usage and event reception at the same time The layer responds to interactive events and realizes personalized development without the need for key mapping position conventions.

Benefits of technology

It improves the security of input data and transmitted data of the security keyboard, reduces application resource usage, meets personalized development needs, and does not affect the normal push and pull of the video stream during interaction.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119066723A_ABST
    Figure CN119066723A_ABST
Patent Text Reader

Abstract

The embodiment of the invention discloses a safety keyboard generation method and device, equipment and a storage medium. The safety keyboard generation method and device are used for improving safety of input data and transmission data. The method comprises the steps that when a security keyboard calling request is received, a server issues a security keyboard rendering parameter corresponding to a session identifier to a client; the client performs rendering based on the security keyboard rendering parameter to obtain an initial security keyboard, and displays the initial security keyboard on a display page of the client; when the client receives an interaction event of the target object for the initial security keyboard, transmitting the interaction event to an event receiving layer in the client, and performing keyboard rewriting on the interaction event through the event receiving layer to obtain keyboard rewriting interaction data; the server generates rewriting security keyboard rendering parameters based on the keyboard rewriting interaction data; and the client performs secure keyboard rendering on the rewritten secure keyboard rendering parameters to obtain a redrawn secure keyboard, and displays the redrawn secure keyboard on a display page of the client.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] Embodiments of the present application relate to the field of multimedia processing technology, and in particular to a method, device, equipment, and storage medium for generating a secure keyboard. Background Art

[0002] With the development of Internet and mobile communication technology, more and more virtual product or data transaction processes, or secure login of web pages or software programs may require input of passwords, verification codes, etc., that is, transaction security and login security can be protected by using a secure keyboard.

[0003] Among them, the acquisition of traditional secure keyboards is usually based on the terminal device pulling keyboard data from the server through a network protocol such as the https protocol, and then obtaining the keyboard content entered in the terminal device, and encrypting it back to the server, so that the server can decode the data to determine whether the input is correct.

[0004] Since the terminal device stores the retrieved keyboard data in the terminal device through the network protocol, the keyboard data is then parsed and rendered on the terminal device through the software toolkit of the built-in secure keyboard, namely SDK. However, the software toolkit of the secure keyboard built into the terminal device is easily cracked by some abnormal operating behaviors (such as shell smashing, etc.), making it difficult to protect the secure keyboard during operations such as screen recording and screenshots, thereby reducing the security of input data or transmitted data through the secure keyboard. Summary of the invention

[0005] The embodiments of the present application provide a method, apparatus, device and storage medium for generating a secure keyboard. On the one hand, by deploying the keyboard on a server and transmitting the secure keyboard rendering parameters to the client in the form of a keyboard screen, the influence of abnormal operation behavior can be avoided, and there is no need for a built-in secure keyboard software toolkit on the client to reduce the resource usage of the application. On the other hand, there is no need to establish an advance agreement on the key mapping position to meet the business needs of personalized development of the secure keyboard. In addition, in the interaction of the secure keyboard, by passing the response to the event to the lower event receiving layer, the normal push and pull of the secure keyboard video stream can be received in a timely manner without affecting the normal push and pull of the secure keyboard video stream.

[0006] An embodiment of the present application provides a method for generating a secure keyboard, including:

[0007] When receiving a secure keyboard call request carrying a session identifier from a client, the server sends secure keyboard rendering parameters corresponding to the session identifier to the client;

[0008] When the client receives the secure keyboard rendering parameters, the client renders the secure keyboard based on the secure keyboard rendering parameters to obtain an initial secure keyboard, and displays the initial secure keyboard on a display page of the client;

[0009] When the client receives an interaction event of the target object for the initial secure keyboard, the interaction event is transmitted to the event receiving layer in the client, and the interaction event is keyboard rewritten by the event receiving layer to obtain keyboard rewriting interaction data;

[0010] When receiving the keyboard rewrite interaction data uploaded by the client, the server generates rewrite safe keyboard rendering parameters based on the keyboard rewrite interaction data and sends them to the client;

[0011] The client performs secure keyboard rendering on the rewritten secure keyboard rendering parameters to obtain a redrawn secure keyboard, and displays the redrawn secure keyboard on a display page of the client.

[0012] On the other hand, the present application provides a device for generating a secure keyboard, comprising:

[0013] The processing unit is configured to, when receiving a secure keyboard call request carrying a session identifier sent by a client, cause the server to send secure keyboard rendering parameters corresponding to the session identifier to the client;

[0014] A display unit, configured to, when the client receives the secure keyboard rendering parameters, render the client based on the secure keyboard rendering parameters to obtain an initial secure keyboard, and display the initial secure keyboard on a display page of the client;

[0015] The processing unit is further configured to, when the client receives an interaction event of the target object with respect to the initial secure keyboard, transmit the interaction event to the event receiving layer in the client, and perform keyboard rewriting on the interaction event through the event receiving layer to obtain keyboard rewriting interaction data;

[0016] The processing unit is further configured to, when receiving keyboard rewriting interaction data uploaded by the client, generate rewriting security keyboard rendering parameters based on the keyboard rewriting interaction data and send them to the client;

[0017] The display unit is also used for the client to perform secure keyboard rendering on the rewritten secure keyboard rendering parameters to obtain a redrawn secure keyboard, and display the redrawn secure keyboard on a display page of the client.

[0018] In a possible design, in an implementation of another aspect of the embodiment of the present application, the processing unit may be specifically used for:

[0019] When receiving a secure keyboard call request carrying a session identifier from a client, the server sends secure keyboard rendering parameters corresponding to the session identifier to the client through a streaming protocol, wherein the secure keyboard rendering parameters at least include a keyboard diagram, a keyboard color, and a key spacing;

[0020] The display unit can be specifically used for: when the client receives the secure keyboard rendering parameters through the streaming protocol, the client renders based on the secure keyboard rendering parameters to obtain an initial secure keyboard, and displays the initial secure keyboard on a display page of the client.

[0021] In a possible design, in an implementation of another aspect of the embodiment of the present application, the display unit may be specifically used for:

[0022] The client reads at least one frame of video data through a streaming media protocol and converts the data into an object to obtain keyboard video stream data;

[0023] The client decodes the secure keyboard rendering parameter according to the decoding algorithm corresponding to the session identifier to obtain the decoded secure keyboard rendering parameter;

[0024] The client performs secure keyboard rendering based on the keyboard video stream data and the decoded secure keyboard rendering parameters to obtain an initial secure keyboard, and displays the initial secure keyboard on a display page of the client.

[0025] In a possible design, in an implementation of another aspect of the embodiment of the present application,

[0026] An acquisition unit, used to acquire the position of a player of the client, and initialize a secure keyboard drawing window based on the position of the player, wherein the player is used to support the operating environment of the secure keyboard, and the player includes a view layer and an event receiving layer, and the view layer is located at an upper level of the event receiving layer;

[0027] The display unit can be specifically used to: initialize the secure keyboard drawing window, draw and render the video stream data and the decoded secure keyboard rendering parameters, obtain the initial secure keyboard and the keyboard style corresponding to the initial secure keyboard, and display the initial secure keyboard on the display page of the client, wherein the keyboard style includes at least the key spacing, keyboard color and key text.

[0028] In a possible design, in an implementation of another aspect of the embodiment of the present application, the processing unit may be specifically used for:

[0029] Adjust the style of the initial secure keyboard based on the keyboard style and generate interaction events;

[0030] The client passes the interaction event from the view layer corresponding to the initial security keyboard to the event receiving layer through the event hub;

[0031] Respond to the interaction event through the event receiving layer, and obtain the interaction behavior coordinates and interaction behavior path corresponding to the interaction behavior on the display page based on the interaction event;

[0032] Encapsulate the interaction behavior coordinates and interaction behavior path into a keyboard rewrite message body and upload it to the server through the streaming media protocol;

[0033] The processing unit can be specifically used for: when receiving the keyboard rewrite message body uploaded by the client, the server generates rewrite security keyboard rendering parameters based on the keyboard rewrite message body, and sends them to the client.

[0034] In a possible design, in an implementation of another aspect of the embodiment of the present application,

[0035] The storage unit is used to convert the decoded secure keyboard rendering parameters into data in a dictionary format and store the converted data in the client.

[0036] In a possible design, in an implementation of another aspect of the embodiment of the present application,

[0037] The processing unit is further used to construct a keyboard mirroring platform for the server, and to modify the network protocol corresponding to each server node into a long link in the keyboard mirroring platform;

[0038] The processing unit is also used to establish a communication protocol between the client and the server corresponding to each server node based on the long link, wherein the communication protocol is a streaming media protocol.

[0039] In a possible design, in an implementation of another aspect of the embodiment of the present application,

[0040] The acquisition unit is further used to acquire the network behavior currently running in the background of the client, wherein the network behavior includes one or more of refreshing, downloading, uploading or storing;

[0041] The acquisition unit is further used to acquire the operating environment quality value corresponding to the client based on the network behavior currently running in the background;

[0042] The processing unit is also used to determine the network behavior to be suspended from the network behavior currently running in the background based on the environment quality threshold range that the running environment quality value meets, and perform a suspension operation on the network behavior to be suspended.

[0043] In a possible design, in an implementation of another aspect of the embodiment of the present application,

[0044] The acquiring unit is further configured to acquire the encryption identifier corresponding to the client based on the session request when the server receives the session establishment request sent by the client;

[0045] The processing unit is also used for the server to establish a session authentication between the server and the client based on the encryption identifier.

[0046] In a possible design, in an implementation of another aspect of the embodiment of the present application, the processing unit may be specifically used for:

[0047] The server decrypts the encrypted identifier. If the decryption result is successful, a session identifier is generated and sent to the client for binding.

[0048] If the decryption result is a decryption failure, a session establishment failure prompt is sent to the client.

[0049] In a possible design, in an implementation of another aspect of the embodiment of the present application, the acquisition unit may be specifically used to:

[0050] When the server receives the session establishment request sent by the client, the server obtains the object identity based on the session request, and sends the object identity authentication data corresponding to the object identity, the public key corresponding to the object identity authentication data, and the keyboard type supported by the current object identity to the client;

[0051] The client randomly generates a key and encrypts the key based on the public key in the object authentication data to obtain an encrypted identifier;

[0052] The processing unit can be used for:

[0053] The server decrypts the encrypted identifier using the private key corresponding to the object identity authentication data. If the decryption result is successful, a session identifier is generated.

[0054] The server establishes a mapping relationship between the session identifier and the key, and sends the session identifier after the mapping relationship is established to the client, so that the client binds the session identifier with the object identifier.

[0055] In a possible design, in an implementation of another aspect of the embodiment of the present application, the processing unit may be specifically used for:

[0056] The server parses the keyboard rewrite message body through a decryption algorithm to obtain the keyboard rewrite interaction data;

[0057] The server converts the keyboard rewrite interaction data into keyboard operation events;

[0058] The server redraws the keyboard video stream data and the secure keyboard rendering data based on the keyboard operation event to obtain the rewritten secure keyboard rendering parameters.

[0059] On the other hand, the present application provides a computer device, including: a memory, a processor, and a bus system;

[0060] Wherein, the memory is used to store programs;

[0061] The processor is used to implement the above-mentioned methods when executing the program in the memory;

[0062] The bus system is used to connect the memory and the processor so that the memory and the processor can communicate with each other.

[0063] Another aspect of the present application provides a computer-readable storage medium, in which instructions are stored. When the computer-readable storage medium is run on a computer, the computer is enabled to execute the above-mentioned methods.

[0064] It can be seen from the above technical solutions that the embodiments of the present application have the following beneficial effects:

[0065] When receiving a secure keyboard call request carrying a session identifier sent by the client, the server sends the secure keyboard rendering parameters corresponding to the session identifier to the client, so that the client can perform secure keyboard rendering on the secure keyboard rendering parameters to obtain the initial secure keyboard, and display the initial secure keyboard on the client's display page. Then, when the client receives an interaction event corresponding to the initial secure keyboard, it passes the interaction event to the event receiving layer, and performs keyboard rewriting on the interaction event through the event receiving layer to obtain keyboard rewriting interaction data, so that the server can generate rewritten secure keyboard rendering parameters based on the keyboard rewriting interaction data, and send them to the client. Then, the client renders the rewritten secure keyboard rendering parameters to obtain a redrawn secure keyboard displayed on the client's display page. Through the above method, on the one hand, by deploying the keyboard on the server and transmitting the secure keyboard rendering parameters to the client in the form of a keyboard screen, so that the client decodes and displays the keyboard screen, it is possible to avoid the impact of abnormal operation behaviors such as shell smashing and decompilation that are easy to handle when using network protocols for data transmission, and there is no need for a built-in secure keyboard software toolkit on the client to reduce the resource usage of the application; on the other hand, the keyboard can be rewritten on the client in response to interactive events through the event receiving layer, and then uploaded to the server to obtain the rewritten secure keyboard rendering parameters, so that the client can re-render based on the rewritten secure keyboard rendering parameters to obtain a personalized redrawn secure keyboard, without the need to establish an advance agreement on the key mapping position, so as to meet the business needs of personalized development of the secure keyboard, and in the interaction of the secure keyboard, by transmitting the event response to the lower-level event receiving layer, it is possible to receive interactive behaviors in a timely manner without affecting the normal push and pull of the secure keyboard video stream. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 This is a schematic diagram of the architecture of a data object control system in an embodiment of the present application;

[0067] Figure 2 is a flow chart of an embodiment of a method for generating a secure keyboard in an embodiment of the present application;

[0068] Figure 3 is another embodiment flow chart of the method for generating a secure keyboard in an embodiment of the present application;

[0069] Figure 4 is another embodiment flow chart of the method for generating a secure keyboard in an embodiment of the present application;

[0070] Figure 5 is another embodiment flow chart of the method for generating a secure keyboard in an embodiment of the present application;

[0071] Figure 6 is another embodiment flow chart of the method for generating a secure keyboard in an embodiment of the present application;

[0072] Figure 7 is another embodiment flow chart of the method for generating a secure keyboard in an embodiment of the present application;

[0073] Figure 8 is another embodiment flow chart of the method for generating a secure keyboard in an embodiment of the present application;

[0074] Fig. 9 is another embodiment flow chart of the method for generating a secure keyboard in an embodiment of the present application;

[0075] Fig.10 is another embodiment flow chart of the method for generating a secure keyboard in an embodiment of the present application;

[0076] Fig.11 is another embodiment flow chart of the method for generating a secure keyboard in an embodiment of the present application;

[0077] Fig.12 is another embodiment flow chart of the method for generating a secure keyboard in an embodiment of the present application;

[0078] Fig.13 is another embodiment flow chart of the method for generating a secure keyboard in an embodiment of the present application;

[0079] Fig.14 It is a schematic diagram of a principle flow of a method for generating a secure keyboard in an embodiment of the present application;

[0080] Fig.15 It is a schematic diagram of the principle of rendering a secure keyboard by a client in the method for generating a secure keyboard in an embodiment of the present application;

[0081] Fig.16 It is a schematic diagram of the effect of an event receiving layer of a client of the method for generating a secure keyboard in an embodiment of the present application;

[0082] Fig.17 It is a schematic diagram of a page effect of a secure keyboard in the method for generating a secure keyboard in an embodiment of the present application;

[0083] Fig.18 is a schematic diagram of another page effect of a secure keyboard in the method for generating a secure keyboard in an embodiment of the present application;

[0084] Fig.19 It is a schematic diagram of the principle of establishing a session between a client and a server in the method for generating a secure keyboard in an embodiment of the present application;

[0085] Fig. 20 is a schematic diagram of an embodiment of a device for generating a secure keyboard in an embodiment of the present application;

[0086] Fig.21 It is a schematic diagram of an embodiment of a computer device in an embodiment of the present application. DETAILED DESCRIPTION

[0087] The embodiments of the present application provide a method, apparatus, device and storage medium for generating a secure keyboard, which are used to avoid the impact of abnormal operation behavior by deploying the keyboard on a server and transmitting the secure keyboard rendering parameters to the client in the form of a keyboard screen, and there is no need for a built-in secure keyboard software toolkit on the client to reduce the resource usage of the application; on the other hand, there is no need to establish an advance agreement on the key mapping position to meet the business needs of personalized development of the secure keyboard, and in the interaction of the secure keyboard, by transmitting the response to the event to the lower event receiving layer, it is possible to receive the interactive behavior in a timely manner without affecting the normal push and pull of the secure keyboard video stream.

[0088] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein, for example. In addition, the terms "including" and "corresponding to" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0089] To facilitate understanding, some terms or concepts involved in the embodiments of the present application are first explained.

[0090] 1. HTTP: Hypertext Transfer Protocol (HTTP) is an application layer protocol for transmitting hypermedia documents (such as HTML). It is designed for communication between web browsers and web servers, but can also be used for other purposes. HTTP follows the classic client-server model, where the client opens a connection to make a request and then waits until a response is received from the server.

[0091] 2. Secure keyboard: A virtual secure keyboard that allows input by clicking keys on the virtual keyboard without pressing keys on the real keyboard, to prevent background software in many systems from recording keystrokes and stealing passwords. The secure keyboard component is built into the client and provides a randomly distributed virtual secure keyboard that protects input data from multiple dimensions, including the bottom layer, before startup, during input, and after input, effectively avoiding the leakage of important personal information.

[0092] 3. Server cluster: Server cluster refers to the collection of many servers to provide the same service together. To the client, it looks like there is only one server. The cluster can use multiple computers for parallel computing to achieve high computing speed, and can also use multiple computers for backup, so that if any machine fails, the entire system can still operate normally.

[0093] 4. Real Time Messaging Protocol (RTMP): It is a protocol family based on TCP long link, which includes RTMP basic protocol and multiple variants such as RTMPT, RTMPS and RTMPE. The embodiment of the present application can use this protocol to transmit the rendering data of the secure keyboard.

[0094] It can be understood that in the specific implementation of the present application, related data such as security keyboard rendering parameters are involved. When the above embodiments of the present application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0095] It is understandable that the method for generating a secure keyboard as disclosed in the present application specifically involves cloud technology, and cloud technology is further introduced below. Cloud technology refers to a hosting technology that unifies a series of resources such as hardware, software, and network in a wide area network or a local area network to realize the calculation, storage, processing, and sharing of data. Cloud technology is a general term for network technology, information technology, integration technology, management platform technology, application technology, etc. based on the cloud computing business model application, which can form a resource pool, which is used on demand and is flexible and convenient. Cloud computing technology will become an important support. The background services of the technical network system require a large amount of computing and storage resources, such as video websites, picture websites, and more portal websites. With the high development and application of the Internet industry, in the future, each item may have its own identification mark, and all need to be transmitted to the background system for logical processing. Data of different levels will be processed separately, and all kinds of industry data require strong system backing support, which can only be achieved through cloud computing.

[0096] Cloud computing is a computing model that distributes computing tasks across a resource pool consisting of a large number of computers, enabling various application systems to obtain computing power, storage space, and information services as needed. The network that provides resources is called a "cloud". From the user's perspective, the resources in the "cloud" are infinitely scalable and can be obtained at any time, used on demand, expanded at any time, and paid for by use.

[0097] As a provider of basic cloud computing capabilities, a cloud computing resource pool (referred to as a cloud platform, generally referred to as an IaaS (Infrastructure as a Service) platform) will be established, and various types of virtual resources will be deployed in the resource pool for external customers to choose to use. The cloud computing resource pool mainly includes: computing devices (virtualized machines, including operating systems), storage devices, and network devices.

[0098] According to the logical function division, the PaaS (Platform as a Service) layer can be deployed on the IaaS (Infrastructure as a Service) layer, and the SaaS (Software as a Service) layer can be deployed on the PaaS layer. SaaS can also be deployed directly on IaaS. PaaS is a platform for software operation, such as databases, web containers, etc. SaaS is a variety of transaction software, such as web portals, SMS mass senders, etc. Generally speaking, SaaS and PaaS are upper layers relative to IaaS.

[0099] Secondly, cloud security refers to the general term for security software, hardware, users, organizations, and security cloud platforms based on cloud computing business model applications. Cloud security integrates emerging technologies and concepts such as parallel processing, grid computing, and unknown virus behavior judgment. Through a large number of networked clients, it monitors abnormal software behavior in the network, obtains the latest information on Trojans and malicious programs on the Internet, and sends it to the server for automatic analysis and processing, and then distributes virus and Trojan solutions to each client.

[0100] Secondly, cloud storage is a new concept extended and developed from the concept of cloud computing. A distributed cloud storage system (hereinafter referred to as storage system) refers to a storage system that uses cluster applications, grid technology, and distributed storage file systems to bring together a large number of different types of storage devices (storage devices are also called storage nodes) in the network through application software or application interfaces to work together and provide external data storage and transaction access functions.

[0101] At present, the storage method of the storage system is: create a logical volume, and when creating a logical volume, allocate physical storage space for each logical volume. The physical storage space may be composed of disks of a storage device or several storage devices. The client stores data on a logical volume, that is, stores the data on the file system. The file system divides the data into many parts, each of which is an object. The object contains not only data but also additional information such as data identification (ID, ID entity). The file system writes each object into the physical storage space of the logical volume, and the file system records the storage location information of each object, so that when the client requests to access the data, the file system can allow the client to access the data according to the storage location information of each object.

[0102] The process of the storage system allocating physical storage space to a logical volume is as follows: based on the estimated capacity of the objects stored in the logical volume (this estimate often has a large margin relative to the actual capacity of the objects to be stored) and the grouping of independent redundant disk arrays (RAID, Redundant Array of Independent Disks), the physical storage space is pre-divided into stripes. A logical volume can be understood as a stripe, thereby allocating physical storage space to the logical volume.

[0103] It should be understood that the generation method based on the secure keyboard provided in this application can be applied to various scenarios, including but not limited to cloud technology, artificial intelligence, maps, smart transportation, etc., and is used to establish a long link between the server and the client, implement the streaming media protocol between the server and the client, deploy the keyboard on the server to run, and transmit the keyboard screen to the client through the streaming media protocol, so that the client can decode and display the keyboard screen, and upload it to the server after the client rewrites the keyboard through the event receiving layer to obtain the rewritten secure keyboard rendering parameters, and then send it to the client to render and obtain a personalized redrawn secure keyboard. The server cluster analyzes and decodes the data, redraws the screen, and finally sends it to the client to be applied to scenarios such as game product live broadcast scenarios, audio and video safe playback scenarios, virtual product trading scenarios, and software program safe login scenarios.

[0104] In order to solve the above problems, the present application proposes a method for generating a secure keyboard, which is applied to Figure 1 The data object control system shown is shown in Figure 1 , Figure 1 FIG. 1 is a schematic diagram of an architecture of a data object control system in an embodiment of the present application. Figure 1As shown, when the server receives a secure keyboard call request carrying a session identifier sent by the client, the server sends the secure keyboard rendering parameters corresponding to the session identifier to the client, so that the client can perform secure keyboard rendering on the secure keyboard rendering parameters to obtain the initial secure keyboard, and display the initial secure keyboard on the client's display page. Then, when the client receives an interaction event corresponding to the initial secure keyboard, the interaction event is passed to the event receiving layer, and the interaction event is keyboard rewritten through the event receiving layer to obtain keyboard rewrite interaction data, so that the server can generate a rewritten secure keyboard rendering parameter based on the keyboard rewrite interaction data, and send it to the client. Then, the client renders the rewritten secure keyboard rendering parameters to obtain a redrawn secure keyboard displayed on the client's display page. Through the above method, on the one hand, by deploying the keyboard on the server and transmitting the secure keyboard rendering parameters to the client in the form of a keyboard screen, so that the client decodes and displays the keyboard screen, it is possible to avoid the impact of abnormal operation behaviors such as shell smashing and decompilation that are easy to handle when using network protocols for data transmission, and there is no need for a built-in secure keyboard software toolkit on the client to reduce the resource usage of the application; on the other hand, the keyboard can be rewritten on the client in response to interactive events through the event receiving layer, and then uploaded to the server to obtain the rewritten secure keyboard rendering parameters, so that the client can re-render based on the rewritten secure keyboard rendering parameters to obtain a personalized redrawn secure keyboard, without the need to establish an advance agreement on the key mapping position, so as to meet the business needs of personalized development of the secure keyboard, and in the interaction of the secure keyboard, by transmitting the event response to the lower-level event receiving layer, it is possible to receive interactive behaviors in a timely manner without affecting the normal push and pull of the secure keyboard video stream.

[0105] Understandably, Figure 1 Only one type of terminal device is shown in the figure. In actual scenarios, more types of terminal devices may participate in the data processing process. Terminal devices include but are not limited to mobile phones, computers, intelligent voice interaction devices, smart home appliances, car terminals, etc. The specific number and type depend on the actual scenario and are not limited here. Figure 1 One server is shown in the figure, but in actual scenarios, multiple servers may also be involved, especially in the scenario of multi-model training interaction. The number of servers depends on the actual scenario and is not limited here.

[0106] It should be noted that in this embodiment, the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms. The terminal device and the server can be directly or indirectly connected through wired or wireless communication, and the terminal device and the server can be connected to form a blockchain network, which is not limited in this application.

[0107] Combined with the above introduction, the following will introduce the method for generating a secure keyboard in this application. Figure 2 In one embodiment of the present application, a method for generating a secure keyboard includes:

[0108] In step S101, when receiving a secure keyboard call request carrying a session identifier sent by a client, the server sends secure keyboard rendering parameters corresponding to the session identifier to the client;

[0109] In an embodiment of the present application, when the server receives a secure keyboard call request sent by the client, it can first determine whether the secure keyboard call request carries a session identifier. Then, after determining that the secure keyboard call request carries a session identifier, the server can call the secure keyboard rendering parameters according to the session identifier and send them to the client. The keyboard can be deployed on one side of the server, that is, the generation of the secure keyboard is run on the server side, and the keyboard screen after rendering is compressed and transmitted to the client through the network. That is, the server can call the secure keyboard rendering parameters according to the session identifier and send them to the client, so that the client can subsequently render the corresponding secure keyboard according to the secure keyboard rendering parameters, so that there is no need to install any high-end processor and graphics card in the client to generate a secure keyboard, and the client page does not need to install the software tool kit for the secure keyboard, which can reduce the operating pressure of the client to a certain extent.

[0110] The session identifier is an identifier uniquely assigned by the server to the client, which is used to indicate that the session between the client and the server is successfully established and data transmission can be performed stably.

[0111] The secure keyboard rendering parameters include the keyboard image drawn in the server and the rendering parameters corresponding to the keyboard image. The rendering parameters corresponding to the keyboard image are the custom parameters of the secure keyboard. For example, if there are multiple views in an application, the parameters corresponding to each view at different screen sizes and in different positions include keyboard icons, keyboard titles, fill colors, key spacing, etc.

[0112] Specifically, Fig.14 As shown, when the user of a certain client (such as game player S who uses a mobile phone to experience game A) needs to perform secure login or secure transaction of virtual products, a secure keyboard call request can be sent to the server through the client to obtain the corresponding secure keyboard. Therefore, when the server receives the secure keyboard call request sent by the client, it can first determine whether the secure keyboard call request carries a session identifier. It can be understood that if the secure keyboard call request does not carry a session identifier, that is, the session between the client and the server is not successfully established, such as the handshake between the client and the server fails, the server abandons the secure keyboard call request to maintain system security.

[0113] On the contrary, if the secure keyboard call request carries a session identifier, that is, the session between the client and the server is successfully established, such as a successful handshake between the client and the server, the server can obtain secure keyboard rendering parameters that can adapt to the client's screen size based on the session identifier and send them to the client.

[0114] In step S102, when the client receives the secure keyboard rendering parameters, the client renders based on the secure keyboard rendering parameters to obtain an initial secure keyboard, and displays the initial secure keyboard on a display page of the client;

[0115] In an embodiment of the present application, when the client receives the security keyboard rendering parameters sent by the server, the client can decode the security keyboard rendering parameters to obtain keyboard stream data. Then, the client can perform a secure keyboard rendering based on the keyboard stream data to obtain an initial secure keyboard, and display the initial secure keyboard on the display page of the client to facilitate the user object to quickly obtain the secure keyboard for display.

[0116] It is understandable that before the client decodes the secure keyboard rendering parameters, it can first determine the basic environment in which the client runs the secure keyboard. It can be understood that the client will create different types of players for different environments, for example, Fig.15 As shown, when the basic environment of the client running the secure keyboard is that the loaded page is h5, the player is created as a dom node of video; or, when the basic environment of the client running the secure keyboard is that the loaded page is an Android system, the player can be MediaPlayer; or, when the basic environment of the client running the secure keyboard is that the loaded page is an iOS system, the player can be AVPlayer, etc., and no specific restrictions are made here. Among them, although the kernels of different types of players are different and the adapted systems are different, in the embodiment of the present application, the customized decoder will be bound to the corresponding player in the form of a plug-in to help the player better adapt to the system.

[0117] Among them, the plug-in, i.e., the decoder, can not only read information such as bit rate, format, duration, and video extension from the file header, but also obtain custom parameters of the keyboard from the streaming media protocol. Taking the live broadcast protocol such as RTMP as an example, the embodiment of the present application can put the custom parameter information in the videoData field to facilitate data reading.

[0118] Specifically, Fig.14 As shown, when the client receives the secure keyboard rendering parameters sent by the server, the client can decode the secure keyboard rendering parameters through a plug-in (ie, a decoder) (eg, Fig.14 The client plug-in decoding shown in FIG. 1 is used to decode the keyboard stream data, that is, the keyboard video stream data (such as Fig.14 Normal video stream as shown) and decoded secure keyboard rendering parameters (such as Fig.14 )

[0119] Furthermore, if Fig.14 As shown, after obtaining the keyboard stream data, the client can perform secure keyboard rendering and image drawing based on the keyboard stream data (such as Fig.14 , such as Fig.15 As shown, specifically, the image database, i.e., the glut library, and the secure keyboard drawing window (e.g., a double buffer window, etc.) may be initialized first, and then, the keyboard stream data may be rendered and the image may be drawn using the image database, i.e., the glut library, and the secure keyboard drawing window (e.g., a double buffer window, etc.), so as to draw an initial secure keyboard and a custom parameter adjustment image corresponding to the initial secure keyboard, so that the initial secure keyboard and the custom parameter adjustment image corresponding to the initial secure keyboard may be displayed on the display page of the client (e.g., a loading page supported by a player).

[0120] Among them, initializing the image database, namely the glut library, can be by calling the glutInit function, passing in the unmodified argc variable address and argv pointer in the main function, and then, the position of the player can be obtained through the argc variable address and argv pointer, and the position of the player can be adjusted to display the actual position of the window to clarify the flag required by the current secure keyboard, so that a secure keyboard that can adapt to the screen size of the client can be rendered better and more accurately.

[0121] Among them, the image drawing modes include GLUT_DOUBLE (double buffer window), GLUT_RGBA (such as Fig.15 RGBA color mode as shown), GLUT_DEPTH (depth test), GLUT_STENCIL (as shown Fig.15The template buffer shown in the figure) is not specifically limited here, wherein the RGBA color mode is a required parameter, indicating the color components of each pixel point required when rendering the secure keyboard, such as Fig.15 As shown, the two modes GLUT_RGBA and GLUT_STENCIL are usually used when rendering a secure keyboard.

[0122] For example, Fig.17 As shown, when the basic environment for the client to run the secure keyboard is that the loaded page is an Android system, an initial secure keyboard (such as Fig.17 The 26-letter key security keyboard shown in the figure) and the custom parameter adjustment image corresponding to the initial security keyboard (such as Fig.17 The border parameter adjustment shown in the figure, such as border width, border brightness, etc., and the key parameter adjustment, such as character spacing, character size, etc., and text color adjustment, etc., are not specifically limited here). It is understandable that when the user object clicks the keyboard, the key needs to be highlighted so that the user object can better perceive the data input and the interactive experience with the secure keyboard.

[0123] For example, Fig.18 As shown, when the basic environment for the client to run the secure keyboard is that the loaded page is the iOS system, an initial secure keyboard (such as Fig.18 The security keyboard with the number 9 key as shown) and the custom parameter adjustment image corresponding to the initial security keyboard (such as Fig.18 The adjustments of the key border parameters shown include border thickness, border radius, border color, and border pressed color, etc., parameter adjustments of key spacing, key color filling, and key text, etc., which are not specifically limited here).

[0124] In step S103, when the client receives an interaction event of the target object for the initial secure keyboard, the interaction event is transmitted to the event receiving layer in the client, and the interaction event is keyboard rewritten by the event receiving layer to obtain keyboard rewriting interaction data;

[0125] In an embodiment of the present application, after the target object generates an interactive behavior on the client, the client can upload the interactive information to the server so that the server can redraw the keyboard based on the uploaded interactive information and obtain the keyboard rewrite interactive data, so that the client can rewrite the keyboard interactive data and re-render a customized secure keyboard that meets the personalized needs of the user object, that is, it meets the needs of different user objects for different keyboard styles and different key contents, and there is no need to establish an advance agreement on the key mapping position. It can also receive interactive behaviors in a timely manner without affecting the normal push and pull of the secure keyboard video stream.

[0126] It is understandable that since the container of the secure keyboard belongs to the player, and the functions or parameters passed by players with different kernels to process click events are inconsistent, for example, the player of the iOS system calls function A to generate a click behavior, and the player of the Android system calls function B. Since there are many third-party players on the market, if different players need to be adapted, all players must be enumerated, which is a huge workload and difficult to achieve backward compatibility and maintenance.

[0127] Therefore, this embodiment will convert interactive events such as clicks, long presses or slides generated in the client's player into unified binary stream data that are easy for the computer to recognize, such as converted to the event center (i.e., event distribution manager, which is used to manage and distribute received events) through a plug-in (i.e., a section of event detection code, or an event detection thread, which can package the key positions and touch point inputs of the target object on the initial security keyboard into unified binary stream data that is easy for the computer to recognize), so that the decoder plug-in in the client can be unified after the current frame of the keyboard video stream is decoded. Fig.16 As shown, the event hub will be in the view layer where the player is located (such as Fig.16 An event receiving area of ​​the same size, namely, an event receiving layer, is generated below the upper 26-digit letter key security keyboard (as shown in the figure), so that when the client's player receives an interactive event caused by the target object's key input or touch of the touch point on the initial security keyboard, this embodiment can first shield the response to the interactive event in the player, and then, through the plug-in (i.e., the event detection thread), the key input and touch point input data or coordinate values ​​of the target object on the initial security keyboard, i.e., the interactive event, are packaged or converted into unified binary stream data for easy computer recognition, and uploaded to the event center, and then the binary stream data corresponding to the interactive event is passed to the lower layer of the view layer, i.e., the event receiving layer, for event response, thereby eliminating the need to enumerate all players and enabling better backward compatibility and maintenance.

[0128] Specifically, Fig.14As shown in the figure, after the initial security keyboard is drawn, if the user object adjusts the image of the initial security keyboard and the custom parameters corresponding to the initial security keyboard in the display page of the player, and performs operations such as clicking a certain button, switching keyboards, confirming, deleting, and retracting the keyboard, an interaction behavior between the user object and the client is generated, so the client will receive the interaction event corresponding to the initial security keyboard. The client can first generate an event receiving layer with the same size as the view layer below the view layer corresponding to the initial security keyboard through the event center (such as Fig.14 Add the lower layer of the player to receive events as shown).

[0129] Further, the interaction event is passed to the event receiving layer, and the interaction event is keyboard rewritten by the event receiving layer (such as Fig.14 The client rewrites the touch event touches method shown in the figure), which can be specifically determined in the event receiving layer by rewriting the gesture trigger event touchesBegan (that is, when the user object's finger begins to touch the player, but is actually transmitted to the event receiving layer), the gesture move event touchesMoved (that is, when the user object's finger presses down and starts to move), the gesture stop event touchesCancelled (that is, interruption, such as the user object's long press gesture stops, generally used for handwriting keyboards), and the gesture end event touchesEnded (that is, the interaction between the user object and the player ends). The shape function glutReshapeFunc can be called with the coordinates of the player where the user object touches, and the function address sizeMove for detecting the change in the width and height of the player is passed in. The width and height are passed in, that is, the viewport is set in sizeMove to perform the coordinate conversion (x, y) in the player, and the operation behavior path is recorded to obtain the keyboard rewriting interaction data. Then, the keyboard rewriting interaction data can be encapsulated, that is, the coordinates and the operation behavior path are encapsulated in the RTMP message body and uploaded to the server.

[0130] In step S104, when receiving the keyboard rewriting interaction data uploaded by the client, the server generates rewriting security keyboard rendering parameters based on the keyboard rewriting interaction data, and sends them to the client;

[0131] In an embodiment of the present application, when the server receives the keyboard rewrite interaction data uploaded by the client, the server can parse the keyboard rewrite interaction data, and redraw the rendering data based on the parsed keyboard rewrite interaction data to generate rewrite security keyboard rendering parameters. Then, the rewrite security keyboard rendering parameters can be sent to the client so that the client can perform secure keyboard rendering based on the rewrite security keyboard rendering parameters to obtain the display page of the redrawn security keyboard displayed on the client, so as to better meet the user's personalized needs for the secure keyboard.

[0132] Specifically, Fig.14 As shown, when the server receives the keyboard rewrite interaction data uploaded by the client, the server parses the keyboard rewrite interaction data through a decryption algorithm (for example, the SM2 decryption algorithm) to obtain the parsed keyboard rewrite interaction data, so that the server can convert the parsed keyboard rewrite interaction data into keyboard operation events, for example, click a key, switch keyboards, confirm, delete, retract keyboard and other operation events. Then, based on the keyboard operation events, the server can redraw the keyboard image and organize it to obtain the corresponding video stream data, as well as adjust the custom rendering parameters and redraw the image to obtain the rewrite security keyboard rendering parameters, and send them to the client (such as Fig.14 The sent data is re-rendered as indicated).

[0133] In step S105, the client performs secure keyboard rendering on the rewritten secure keyboard rendering parameters to obtain a redrawn secure keyboard, and displays the redrawn secure keyboard on a display page of the client;

[0134] In an embodiment of the present application, when the client receives the rewrite secure keyboard rendering parameters sent by the server, the client can decode the rewrite secure keyboard rendering parameters to obtain the rewrite keyboard stream data. The client can perform secure keyboard rendering based on the rewrite keyboard stream data to obtain the redrawn secure keyboard, and display the redrawn secure keyboard on the client's display page to facilitate the user object to quickly obtain a personalized secure keyboard for display.

[0135] Specifically, Fig.14 As shown, when the client receives the rewritten secure keyboard rendering parameters re-sent by the server, the client can decode the rewritten secure keyboard rendering parameters through a plug-in (ie, a decoder) (eg, Fig.14 The client plug-in decoding shown in the figure) is used to decode the rewritten keyboard stream data, that is, the rewritten keyboard video stream data and the decoded secure keyboard rendering parameters.

[0136] Furthermore, if Fig.14As shown, after obtaining the rewritten keyboard stream data, the client can perform secure keyboard rendering and image drawing based on the rewritten keyboard stream data, such as Fig.15 As shown, specifically, it can be through calling the image database, namely the glut library and the secure keyboard drawing window (for example, a double buffer window, etc.), and then, the image database, namely the glut library and the secure keyboard drawing window (for example, a double buffer window, etc.) can be used to render and draw the image of the rewritten keyboard stream data, so as to draw the rewritten secure keyboard and the custom parameter adjustment image corresponding to the rewritten secure keyboard, so that the rewritten secure keyboard and the custom parameter adjustment image corresponding to the rewritten secure keyboard can be displayed on the display page of the client (such as the loading page supported by the player).

[0137] In an embodiment of the present application, a method for generating a secure keyboard is provided. Through the above method, on the one hand, by deploying the keyboard on a server to run, and by transmitting the secure keyboard rendering parameters to the client in the form of a keyboard screen, so that the client decodes and displays the keyboard screen, the influence of abnormal operation behaviors such as shell smashing and decompilation that are easy to handle when using a network protocol for data transmission can be avoided, and there is no need for a built-in secure keyboard software toolkit on the client to reduce the resource usage of the application; on the other hand, the keyboard can be rewritten on the client in response to an interactive event through an event receiving layer, and then uploaded to the server to obtain the rewritten secure keyboard rendering parameters, so that the client can re-render based on the rewritten secure keyboard rendering parameters to obtain a personalized redrawn secure keyboard, without the need to establish an advance agreement on the key mapping position, so as to meet the business needs of personalized development of the secure keyboard, and in the interaction of the secure keyboard, by transmitting the event response to the lower-level event receiving layer, it is possible to receive the interactive behavior in a timely manner while not affecting the normal push and pull of the secure keyboard video stream.

[0138] Optionally, in the above Figure 2 Based on the corresponding embodiment, in another optional embodiment of the method for generating a secure keyboard provided in the embodiment of the present application, as Figure 3 As shown, when receiving a secure keyboard call request carrying a session identifier sent by a client, step S101 sends secure keyboard rendering parameters corresponding to the session identifier to the client, including: step S301; step S102 includes step S302;

[0139] In step S301, when receiving a secure keyboard call request carrying a session identifier sent by a client, the server sends secure keyboard rendering parameters corresponding to the session identifier to the client through a streaming protocol, wherein the secure keyboard rendering parameters at least include a keyboard diagram, a keyboard color, and a key spacing;

[0140] In step S302, when the client receives the secure keyboard rendering parameters through the streaming protocol, the client renders the secure keyboard based on the secure keyboard rendering parameters to obtain an initial secure keyboard, and displays the initial secure keyboard on a display page of the client.

[0141] In an embodiment of the present application, when the server receives a secure keyboard call request sent by the client, it can first determine whether the secure keyboard call request carries a session identifier. Then, after determining that the secure keyboard call request carries a session identifier, the server can call the secure keyboard rendering parameters according to the session identifier, and send the secure keyboard rendering parameters to the client through the streaming media protocol. The keyboard can be deployed on one side of the server, that is, the generation of the secure keyboard is run on the server side, and the streaming media protocol used is a long connection through the TCP protocol. What is sent to the client is the video stream screen, thereby avoiding abnormal operating behaviors that are easy to occur when using conventional network transmission protocols for data transmission, such as the risk of being snooped on the bottom layer by shell smashing, decompilation, etc., thereby improving the security of data input and data transmission through the secure keyboard.

[0142] The streaming media protocol refers to the process of sending data in segments over the Internet by sequential streaming or real-time streaming after a series of media data (such as sound stream, video stream, text stream, image stream, animation stream, etc.) is compressed. The streaming media protocol includes the streaming media network transmission protocol (HTTP Live Streaming, HLS), RTMP, RTSP, RTCP and RTP, etc., which are not specifically limited here. The commonly used streaming media protocol in this embodiment is the RTMP protocol.

[0143] Among them, the safe keyboard rendering parameters include keyboard graphics, keyboard color (such as Fig.18 keyboard color as indicated) and key spacing (as indicated Fig.18 The key spacing shown in FIG. Fig.17 The border width, border brightness, character spacing, character size, etc. shown are not specifically limited here.

[0144] Specifically, Fig.14 As shown, when the user of a certain client (such as game player S who uses a mobile phone to experience game A) needs to perform secure login or secure transaction of virtual products, a secure keyboard call request can be sent to the server through the client to obtain the corresponding secure keyboard. Therefore, when the server receives the secure keyboard call request sent by the client, it can first determine whether the secure keyboard call request carries a session identifier. It can be understood that if the secure keyboard call request does not carry a session identifier, that is, the session between the client and the server is not successfully established, such as the handshake between the client and the server fails, the server abandons the secure keyboard call request to maintain system security.

[0145] On the contrary, if the secure keyboard call request carries a session identifier, that is, the session between the client and the server is successfully established, such as a successful handshake between the client and the server, the server can obtain the secure keyboard rendering parameters that can adapt to the screen size of the client based on the session identifier (that is, the keyboard image drawn on the server, and the custom rendering parameters of the keyboard such as border width, border brightness, character spacing, character size, etc.), and send it to the client.

[0146] Furthermore, if Fig.14 As shown, when the client receives the secure keyboard rendering parameters sent by the server through the streaming protocol, the client can decode the secure keyboard rendering parameters through the plug-in (ie, the decoder) (eg, Fig.14 The client plug-in decoding shown in the figure) is to first pull a frame of compressed video from the streaming media protocol through the plug-in for decoding to obtain the keyboard video stream data (such as Fig.14 Normal video stream as shown), and secure keyboard rendering parameters can be decoded by plugins (such as Fig.14 )

[0147] Furthermore, if Fig.14 As shown, the keyboard video stream data obtained (such as Fig.14 Normal video stream as shown) and decoded secure keyboard rendering parameters (such as Fig.14 After the custom rendering parameters are obtained, the client can perform secure keyboard rendering and image drawing based on the keyboard video stream data and the decoded secure keyboard rendering parameters (such as Fig.14 , such as Fig.15 As shown, specifically, the image database, i.e., the glut library, and the secure keyboard drawing window (e.g., a double buffer window, etc.) may be initialized first, and then, the keyboard stream data may be rendered and the image may be drawn using the image database, i.e., the glut library, and the secure keyboard drawing window (e.g., a double buffer window, etc.), so as to draw an initial secure keyboard and a custom parameter adjustment image corresponding to the initial secure keyboard, so that the initial secure keyboard and the custom parameter adjustment image corresponding to the initial secure keyboard may be displayed on the display page of the client (e.g., a loading page supported by a player).

[0148] Optionally, in the above Figure 3 Based on the corresponding embodiment, in another optional embodiment of the method for generating a secure keyboard provided in the embodiment of the present application, as Figure 4As shown, the keyboard stream data includes keyboard video stream data and decoded secure keyboard rendering parameters; in step S302, when the client receives the secure keyboard rendering parameters through the streaming media protocol, the client decodes the secure keyboard rendering parameters to obtain keyboard stream data, including:

[0149] In step S401, the client reads at least one frame of video data through a streaming media protocol and converts the data into an object to obtain keyboard video stream data;

[0150] In step S402, the client decodes the secure keyboard rendering parameters according to a decoding algorithm corresponding to the session identifier to obtain decoded secure keyboard rendering parameters.

[0151] In step S403, the client performs secure keyboard rendering based on the keyboard video stream data and the decoded secure keyboard rendering parameters to obtain an initial secure keyboard, and displays the initial secure keyboard on a display page of the client.

[0152] In an embodiment of the present application, when the server receives a secure keyboard call request sent by the client, it can first determine whether the secure keyboard call request carries a session identifier. Then, after determining that the secure keyboard call request carries a session identifier, the server can call the secure keyboard rendering parameters according to the session identifier and send them to the client. The client reads at least one frame of video data through the streaming media protocol and performs data objectization to obtain the keyboard video stream data. The client decodes the secure keyboard rendering parameters according to the decoding algorithm corresponding to the session identifier to obtain the decoded secure keyboard rendering parameters. Then, the client performs secure keyboard rendering based on the keyboard video stream data and the decoded secure keyboard rendering parameters to obtain an initial secure keyboard, and displays the initial secure keyboard on the display page of the client. The keyboard can be deployed on the server side, that is, the generation of the secure keyboard is run on the server side, and the keyboard screen after rendering is compressed and transmitted to the client through the network, that is, the server can call the secure keyboard rendering parameters according to the session identifier and send them to the client, so that the client can subsequently render the corresponding secure keyboard according to the secure keyboard rendering parameters, thereby reducing the operating pressure of the client to a certain extent.

[0153] Specifically, Fig.14As shown, when the user of a certain client (such as game player S who uses a mobile phone to experience game A) needs to perform secure login or secure transaction of virtual products, a secure keyboard call request can be sent to the server through the client to obtain the corresponding secure keyboard. Therefore, when the server receives the secure keyboard call request sent by the client, it can first determine whether the secure keyboard call request carries a session identifier. It can be understood that if the secure keyboard call request does not carry a session identifier, that is, the session between the client and the server is not successfully established, such as the handshake between the client and the server fails, the server abandons the secure keyboard call request to maintain system security.

[0154] On the contrary, if the secure keyboard call request carries a session identifier, that is, the session between the client and the server is successfully established, such as a successful handshake between the client and the server, the server can obtain the secure keyboard rendering parameters that can adapt to the screen size of the client based on the session identifier (that is, the keyboard image drawn on the server, and the custom rendering parameters of the keyboard such as border width, border brightness, character spacing, character size, etc.), and send it to the client.

[0155] Furthermore, if Fig.14 As shown, when the client receives the secure keyboard rendering parameters sent by the server through the streaming protocol, the client can decode the secure keyboard rendering parameters through the plug-in (ie, the decoder) (eg, Fig.14 The client plug-in decoding shown in the figure) is to first pull a frame of compressed video from the streaming media protocol through the plug-in for decoding, read a frame of video data through the streaming media protocol and objectify the data to decode and parse the frame rate, image width and height, pixel format, sampling rate, number of channels, decoding timestamp and other information of the stream, that is, the keyboard video stream data (such as Fig.14 At the same time, the plug-in can be used to perform secure keyboard rendering parameters (such as Fig.14 The custom rendering parameters shown in the figure are decoded to obtain the decoded secure keyboard rendering parameters.

[0156] Furthermore, if Fig.14 As shown, the keyboard video stream data obtained (such as Fig.14 Normal video stream as shown) and decoded secure keyboard rendering parameters (such as Fig.14 After the custom rendering parameters are obtained, the client can perform secure keyboard rendering and image drawing based on the keyboard video stream data and the decoded secure keyboard rendering parameters (such as Fig.14 , such as Fig.15As shown, specifically, the image database, i.e., the glut library, and the secure keyboard drawing window (e.g., a double buffer window, etc.) may be initialized first, and then, the keyboard stream data may be rendered and the image may be drawn using the image database, i.e., the glut library, and the secure keyboard drawing window (e.g., a double buffer window, etc.), so as to draw an initial secure keyboard and a custom parameter adjustment image corresponding to the initial secure keyboard, so that the initial secure keyboard and the custom parameter adjustment image corresponding to the initial secure keyboard may be displayed on the display page of the client (e.g., a loading page supported by a player).

[0157] Optionally, in the above Figure 4 Based on the corresponding embodiment, in another optional embodiment of the method for generating a secure keyboard provided in the embodiment of the present application, as Figure 5 As shown, in step S401, the client reads at least one frame of video data through a streaming media protocol to convert it into a data object, and before obtaining the video stream data, the method further includes: step S501; step S103 includes: step S502;

[0158] In step S501, the position of the player of the client is obtained, and a secure keyboard drawing window is initialized based on the position of the player, wherein the player is used to support the operating environment of the secure keyboard, and the player includes a view layer and an event receiving layer, and the view layer is located at an upper level of the event receiving layer;

[0159] In step S502, by initializing the secure keyboard drawing window, the video stream data and the decoded secure keyboard rendering parameters are image-drawn and rendered to obtain an initial secure keyboard and a keyboard style corresponding to the initial secure keyboard, and the initial secure keyboard is displayed on the display page of the client, wherein the keyboard style includes at least key spacing, keyboard color, and key text.

[0160] In an embodiment of the present application, before obtaining the video stream data, the position of the client's player can be obtained first, and the secure keyboard drawing window can be initialized based on the position of the player. Then, after obtaining the keyboard stream data, the secure keyboard drawing window can be initialized to perform image drawing and rendering on the video stream data and the decoded secure keyboard rendering parameters to obtain the initial secure keyboard and the keyboard style corresponding to the initial secure keyboard, and the initial secure keyboard can be displayed on the client's display page to facilitate personalized adjustment of the secure keyboard.

[0161] Specifically, before the keyboard stream data is obtained, Fig.15As shown, specifically, the image database, namely the glut library, may be initialized first by calling the glutInit function, passing in the unmodified argc variable address and the argv pointer in the main function, and then, the position of the player may be obtained through the argc variable address and the argv pointer, and the position of the player may be adjusted to be displayed at the actual position of the window, so as to clarify the flag required by the current secure keyboard, so that a secure keyboard that can adapt to the screen size of the client may be rendered better and more accurately, and at the same time, a corresponding drawing window may be generated by initializing the secure keyboard drawing window (for example, a double buffer window, etc.), namely, by initializing the image drawing mode.

[0162] Among them, the double buffer window is used to solve the screen flickering caused by multiple drawing operations by using a memory buffer. When double buffering is enabled, all drawing operations are first rendered to the memory buffer instead of the drawing surface on the screen, and after all drawing operations are completed, the memory buffer is directly copied to the drawing surface associated with it.

[0163] Furthermore, after the keyboard stream data is obtained, the image database, i.e., the glut library, and the secure keyboard drawing window (e.g., a double buffer window, etc.) can be used to render and draw the keyboard stream data to obtain the initial secure keyboard and the custom parameter adjustment image corresponding to the initial secure keyboard, i.e., the initial secure keyboard and the keyboard style corresponding to the initial secure keyboard, so that the initial secure keyboard and the custom parameter adjustment image corresponding to the initial secure keyboard can be displayed on the display page of the client (e.g., the loading page supported by the player), wherein the keyboard style at least includes the key spacing (e.g., Fig.18 Key spacing as indicated), keyboard color (as indicated Fig.18 The keyboard color indicated) and key text, etc., also include Fig.17 The border width, border brightness, character spacing, character size, etc. shown are not specifically limited here.

[0164] Optionally, in the above Figure 5 Based on the corresponding embodiment, in another optional embodiment of the method for generating a secure keyboard provided in the embodiment of the present application, as Figure 6 As shown, the keyboard rewriting interaction data includes interaction behavior coordinates and interaction behavior paths; Step S104, when the client receives an interaction event corresponding to the initial secure keyboard, transmits the interaction event to the event receiving layer, and performs keyboard rewriting on the interaction event through the event receiving layer to obtain keyboard rewriting interaction data, including: Steps S601 to S604; Step S105 includes: Step S605;

[0165] In step S601, the style of the initial secure keyboard is adjusted based on the keyboard style, and an interaction event is generated;

[0166] In step S602, the client transmits the interaction event from the view layer corresponding to the initial security keyboard to the event receiving layer through the event hub;

[0167] In step S603, the interactive event is responded to through the event receiving layer, and the interactive behavior coordinates and the interactive behavior path corresponding to the interactive behavior on the display page are obtained based on the interactive event;

[0168] In step S604, the interaction behavior coordinates and the interaction behavior path are encapsulated into a keyboard rewriting message body, and uploaded to the server via a streaming media protocol;

[0169] In step S605, upon receiving the keyboard rewrite message body uploaded by the client, the server generates rewrite security keyboard rendering parameters based on the keyboard rewrite message body, and sends the parameters to the client.

[0170] In an embodiment of the present application, after an interactive behavior is generated on the client, the client uploads the interactive information to the server, that is, when the client receives the operation behavior of the user object, the style of the initial secure keyboard can be adjusted based on the keyboard style, and a corresponding interactive event can be generated, so that the client can pass the interactive event from the view layer corresponding to the initial secure keyboard to the event receiving layer through the event center, respond to the interactive event through the event receiving layer, and obtain the interactive behavior coordinates and interactive behavior path corresponding to the interactive behavior on the display page based on the interactive event, encapsulate the interactive behavior coordinates and the interactive behavior path as a keyboard rewrite message body, and upload it to the server through the streaming media protocol, so that the server can generate a rewrite secure keyboard rendering parameter based on the uploaded keyboard rewrite message body, so that the client can rewrite the secure keyboard rendering parameters to re-render a customized secure keyboard that meets the personalized needs of the user object, that is, to meet the needs of different user objects for different keyboard styles and different key contents, and there is no need to establish an advance agreement on the key mapping position, and it can also receive interactive behaviors in a timely manner without affecting the normal push and pull of the secure keyboard video stream.

[0171] It is understandable that since the container of the secure keyboard belongs to the player, and the functions or parameters passed by players with different kernels to process click events are inconsistent, for example, the player of the iOS system calls function A to generate a click behavior, and the player of the Android system calls function B. Since there are many third-party players on the market, if different players need to be adapted, all players must be enumerated, which is a huge workload and difficult to achieve backward compatibility and maintenance.

[0172] Therefore, in this embodiment, when the user object wants to adjust the keyboard style, the corresponding operation behavior can be input through the client's player, so that when the client receives the user object's operation behavior, the style of the initial security keyboard can be adjusted based on the keyboard style, that is, the corresponding interaction events generated by the operation behaviors such as clicking, long pressing or sliding generated in the client's player.

[0173] Furthermore, the interactive events can be converted to the event center for unified management through the plug-in, and the interactive events can be passed from the view layer corresponding to the initial security keyboard to the event receiving layer. Specifically, after the current frame of the keyboard video stream is decoded, the decoder plug-in in the client can Fig.16 As shown, the event hub will be in the view layer where the player is located (such as Fig.16 An event receiving area of ​​the same size, namely, an event receiving layer, is generated below the upper 26-digit letter key security keyboard (as shown in the figure), so that when the client's player receives an interactive event, this embodiment can first shield the response of the interactive event in the player, and then convert the interactive event to the event center through the plug-in, and then pass it to the lower layer of the view layer, namely, the event receiving layer, for event response, thereby eliminating the need to enumerate all players and enabling better backward compatibility and maintenance.

[0174] The keyboard rewriting message body refers to encapsulating the coordinates and the operation behavior path, that is, the interaction behavior coordinates and the interaction behavior path, in the RTMP message body.

[0175] Specifically, Fig.14 As shown in the figure, after the initial security keyboard is drawn, if the user object adjusts the image of the initial security keyboard and the custom parameters corresponding to the initial security keyboard in the display page of the player, and performs operations such as clicking a certain button, switching keyboards, confirming, deleting, and retracting the keyboard, an interaction behavior between the user object and the client is generated, so the client will receive the interaction event corresponding to the initial security keyboard. The client can first generate an event receiving layer with the same size as the view layer below the view layer corresponding to the initial security keyboard through the event center (such as Fig.14 Add the lower layer of the player to receive events as shown).

[0176] Further, the interaction event is passed to the event receiving layer, and the interaction event is keyboard rewritten by the event receiving layer (such as Fig.14The client rewrites the touch event touches method shown in the figure), which can be specifically determined in the event receiving layer by rewriting the gesture trigger event touchesBegan (that is, when the finger of the user object begins to touch the player, but is actually transmitted to the event receiving layer), the gesture movement event touchesMoved (that is, when the finger of the user object presses down and starts to move), the gesture stop event touchesCancelled (that is, interruption, such as the long press gesture of the user object stops, generally used for handwriting keyboards), and the gesture end event touchesEnded (that is, the interaction between the user object and the player ends). The shape function glutReshapeFunc can be called with the coordinates of the player touched by the user object, and the function address sizeMove for detecting the change of the width and height of the player is passed in. The width and height are passed in, that is, the viewport is set in sizeMove, to perform the coordinate conversion (x, y) in the player, so as to obtain the interactive behavior coordinates, and record the operation behavior path, so as to obtain the interactive behavior path, thereby sorting out the keyboard rewriting interactive data, and then, the keyboard rewriting interactive data can be encapsulated, that is, the coordinates and the operation behavior path are encapsulated in the RTMP message body, and the keyboard rewriting message body is obtained and uploaded to the server.

[0177] Furthermore, after the server receives the keyboard rewrite message body uploaded by the client, the server can parse the keyboard rewrite message body to obtain keyboard rewrite interaction data. The server converts the keyboard rewrite interaction data into keyboard operation events. Based on the keyboard operation events, the server redraws the keyboard video stream data and secure keyboard rendering data, generates rewrite secure keyboard rendering parameters, and sends them to the client.

[0178] Optionally, in the above Figure 4 Based on the corresponding embodiment, in another optional embodiment of the method for generating a secure keyboard provided in the embodiment of the present application, as Figure 7 As shown, in step S402, after the client decodes the secure keyboard rendering parameter according to the decoding algorithm corresponding to the session identifier to obtain the decoded secure keyboard rendering parameter, the method further includes:

[0179] In step S701, the secure keyboard rendering parameters are decoded, data conversion is performed according to a dictionary format, and the converted data is stored in the client.

[0180] Specifically, Fig.15As shown, after the client decodes the decoded secure keyboard rendering parameters to obtain the decoded secure keyboard rendering parameters, the decoded secure keyboard rendering parameters include parameters corresponding to each view at different screen sizes and in different positions if there are multiple views in the application, including keyboard icons, keyboard titles, fill colors, key intervals and other parameters, which can be saved in the view_input.config file in the form of a dictionary to facilitate faster data reading or parsing of the file in the dictionary.

[0181] Optionally, in the above Figure 3 Based on the corresponding embodiment, in another optional embodiment of the method for generating a secure keyboard provided in the embodiment of the present application, as Figure 8 As shown, in step S301, when receiving a secure keyboard call request carrying a session identifier sent by the client, before the server sends the secure keyboard rendering parameters corresponding to the session identifier to the client through the streaming media protocol, the method further includes:

[0182] In step S801, a keyboard mirroring platform is constructed for the server, and the network protocol corresponding to each server node is modified to a long link in the keyboard mirroring platform;

[0183] In step S802, based on the long link, a communication protocol between the client and the server corresponding to each server node is established, wherein the communication protocol is a streaming media protocol.

[0184] It is understandable that since cloud computing is far away from users, network quality will affect the user experience, resulting in a failure to get a timely response after touching the keyboard. This can be achieved by moving computing closer to the user and performing point selection analysis on the user through big data, so that cloud computing can cover the location area of ​​the user as much as possible.

[0185] Therefore, this embodiment constructs a keyboard mirroring platform for the server, and modifies the network protocol corresponding to each server node in the keyboard mirroring platform to a long link. Then, based on the long link, a communication protocol between the client and the server corresponding to each server node can be established, so that cloud computing can cover the location area of ​​the user object as much as possible.

[0186] Specifically, Fig.14As shown, this embodiment is to separate the secure keyboard business platform on the server to divide it into secure keyboard and other types of RTMP channels in storage, and then use the K8S cluster to construct the image warehouse, that is, to construct the keyboard image platform for the server. Among them, corresponding image agents are set in different system architectures and containers to construct the regional boundary agent into the long link TCP or network transmission protocol Https of the external network. Specifically, it can be composed of components edge cloud and edge agent, and the service subject Master of the region is connected to the service node Node, so that the regional boundary agent can be changed from the original network transmission protocol Https protocol to TCP long link, so that it can be convenient to push the RTMP stream based on the TCP long link, and be responsible for the downstream communication of each server node of the agent, so that when the user object actively communicates with the server apiserver through the normal page request in the player, and operates the container on the edge node through the image warehouse, that is, the keyboard image platform in the cloud, the edge cloud can be used for proxy request, wherein the edge cloud and the edge agent use TCP long link, so that the message body in the RTMP uploaded by the client to the server can be processed preferentially.

[0187] Optionally, in the above Figure 8 Based on the corresponding embodiment, in another optional embodiment of the method for generating a secure keyboard provided in the embodiment of the present application, as Fig. 9 As shown, after step S802 establishes a communication protocol between the client and the server corresponding to each server node based on the long link, the method further includes:

[0188] In step S901, the network behavior currently running in the background of the client is obtained, wherein the network behavior includes one or more of refreshing, downloading, uploading or storing;

[0189] In step S902, based on the current background network behavior, the operating environment quality value corresponding to the client is obtained;

[0190] In step S903, based on the environment quality threshold range that the running environment quality value complies with, the network behavior to be suspended is determined from the network behaviors currently running in the background, and a suspension operation is performed on the network behavior to be suspended.

[0191] It is understandable that since cloud computing is far away from users, network quality will affect the user experience, resulting in a failure to get a timely response after touching the keyboard. Therefore, the network quality requirements of the secure keyboard network can also be met by improving the network quality of the user's current environment.

[0192] Therefore, after the client turns on RTMP, this embodiment obtains the network behavior currently running in the background of the client, such as the network behavior of the application in the background such as refreshing, downloading, uploading and storing, and based on the network behavior currently running in the background, obtains the operating environment quality value corresponding to the client (the rates corresponding to the network behaviors such as refreshing, downloading, uploading and storing, i.e., refreshing rate, downloading rate, uploading rate and storing rate), and based on the environmental quality threshold range that the operating environment quality value meets, the operating environment quality value can be compared with the environmental quality threshold range, and the network behavior that does not fall within the environmental quality threshold range is determined as the network behavior to be suspended, and the suspension operation is performed on the network behavior to be suspended. Among them, suspension is just a state of temporary elimination or cleaning.

[0193] Optionally, in the above Figure 2 Based on the corresponding embodiment, in another optional embodiment of the method for generating a secure keyboard provided in the embodiment of the present application, as Fig.10 As shown, in step S101, when receiving a secure keyboard call request carrying a session identifier sent by the client, the server sends the secure keyboard rendering parameters corresponding to the session identifier to the client, and the method further includes:

[0194] In step S1001, when the server receives a session establishment request sent by the client, the server obtains an encryption identifier corresponding to the client based on the session request;

[0195] In step S1002, the server establishes session authentication between the server and the client based on the encryption identifier.

[0196] In an embodiment of the present application, when the server receives a session establishment request sent by the client, the server obtains the encryption identifier corresponding to the client based on the session request, and then, based on the encryption identifier, the server establishes a session authentication between the server and the client to maintain stable data transmission between the client and the server.

[0197] Specifically, Fig.19 As shown, when the user object enters the input page of the player from the client, the client can send a handshake request to the server when the input box has not been focused or the keyboard has not popped up, that is, establish a session request, so that the server can send the object identity authentication data corresponding to the object identifier, the public key corresponding to the object identity authentication data, and the keyboard type supported by the current object identity.

[0198] Furthermore, if Fig.19 As shown, the client randomly generates a key secretKey according to the current page (such as Fig.19The secretKey is asymmetric encrypted with the public key corresponding to the object identity authentication data to obtain the secretID (such as Fig.19 As shown in the secretID, that is, the encryption identifier, it can be understood that the subsequent client can use the secretKey to perform SM2 symmetric encryption on the timestamp of the completion of initializing the security keyboard, and then send it to the background server to maintain the security of data transmission.

[0199] Furthermore, if Fig.19 As shown, the server can use the private key corresponding to the object identity authentication data to decrypt the encrypted identifier secretID. If the decryption obtains secretKey, that is, the identity authentication is successful, the server can assign a session identifier to the client and map the session identifier and the key. It is understandable that the server can subsequently use the key to perform SM2 symmetric decryption on the timestamp. If there is result data, it can also use secretKey to perform SM2 encryption and return the session identifier and result data to the client together.

[0200] Furthermore, after the client receives the response, each subsequent streaming data pull will carry the session identifier and be encrypted with the key secretKey for SM2 to maintain the security of data transmission.

[0201] Optionally, in the above Fig.10 Based on the corresponding embodiment, in another optional embodiment of the method for generating a secure keyboard provided in the embodiment of the present application, as Fig.11 As shown, in step S1002, the server establishes a session authentication between the server and the client based on the encryption identifier, including:

[0202] In step S1101, the server decrypts the encrypted identifier. If the decryption result is successful, a session identifier is generated and sent to the client for binding.

[0203] In step S1102, if the decryption result is a decryption failure, a session establishment failure prompt is sent to the client.

[0204] Specifically, Fig.19 As shown, when the user object enters the input page of the player from the client, the client can send a handshake request to the server when the input box has not been focused or the keyboard has not popped up, that is, establish a session request, so that the server can send the object identity authentication data corresponding to the object identifier, the public key corresponding to the object identity authentication data, and the keyboard type supported by the current object identity.

[0205] Furthermore, if Fig.19As shown, the client randomly generates a key secretKey according to the current page (such as Fig.19 The secretKey is asymmetric encrypted with the public key corresponding to the object identity authentication data to obtain the secretID (such as Fig.19 The secretID indicated is the encryption identifier.

[0206] Furthermore, if Fig.19 As shown, the server can use the private key corresponding to the object identity authentication data to decrypt the encrypted identifier secretID. If the decryption result is successful, that is, the decryption obtains the same key secretKey, then the identity authentication is successful. The server can assign a session identifier to the client, and map the session identifier and the key. Then, the session identifier is sent to the client to be bound to the object identifier.

[0207] On the contrary, if the decryption result is a decryption failure, that is, the decryption key secretKey is not obtained, or the obtained keys are different, then the identity authentication fails, and a session establishment failure prompt is sent to the client via instant message, email or pop-up window, etc., without specific restrictions here.

[0208] Optionally, in the above Fig.11 Based on the corresponding embodiment, in another optional embodiment of the method for generating a secure keyboard provided in the embodiment of the present application, as Fig.12 As shown, in step S1001, when the server receives a session establishment request sent by the client, the server obtains the encryption identifier corresponding to the client based on the session request, including: step S1201 and step S1202; step S1101 includes: step S1203 and step S1204;

[0209] In step S1201, when the server receives a session establishment request sent by the client, the server obtains the object identity based on the session request, and sends the object identity authentication data corresponding to the object identity, the public key corresponding to the object identity authentication data, and the keyboard type supported by the current object identity to the client;

[0210] In step S1202, the client randomly generates a key, and encrypts the key based on the public key in the object identity authentication data to obtain an encrypted identifier;

[0211] In step S1203, the server decrypts the encrypted identifier using the private key corresponding to the object identity authentication data. If the decryption result is successful, a session identifier is generated;

[0212] In step S1204, the server establishes a mapping relationship between the session identifier and the key, and sends the session identifier with the mapping relationship established to the client, so that the client binds the session identifier with the object identifier.

[0213] In an embodiment of the present application, when the server receives a session establishment request sent by the client, the server obtains the object identifier based on the session request, and sends the object identity authentication data corresponding to the object identifier, the public key corresponding to the object identity authentication data, and the keyboard type supported by the current object identity to the client. At the same time, the client randomly generates a key, and encrypts the key based on the public key in the object identity authentication data to obtain an encrypted identifier. Then, the server decrypts the encrypted identifier, that is, decrypts the encrypted identifier using the private key corresponding to the object identity authentication data. If the decryption result is a successful decryption, a session identifier is generated, so that the server establishes a mapping relationship between the session identifier and the key, and sends the session identifier after the mapping relationship is established to the client, so that the client binds the session identifier to the object identifier.

[0214] Specifically, Fig.19 As shown, when the user object enters the input page of the player from the client, the client can send a handshake request to the server when the input box has not been focused or the keyboard has not popped up, that is, establish a session request, so that the server can send the object identity authentication data corresponding to the object identifier, the public key corresponding to the object identity authentication data, and the keyboard type supported by the current object identity.

[0215] Furthermore, if Fig.19 As shown, the client randomly generates a key secretKey according to the current page (such as Fig.19 The secretKey is asymmetric encrypted with the public key corresponding to the object identity authentication data to obtain the secretID (such as Fig.19 The secretID indicated is the encryption identifier.

[0216] Furthermore, if Fig.19 As shown, the server can use the private key corresponding to the object identity authentication data to decrypt the encrypted identifier secretID. If the decryption result is successful, that is, the decryption obtains the same key secretKey, then the identity authentication is successful. The server can assign a session identifier to the client, and map the session identifier and the key. Then, the session identifier is sent to the client to be bound to the object identifier.

[0217] Optionally, in the above Figure 6 Based on the corresponding embodiment, in another optional embodiment of the method for generating a secure keyboard provided in the embodiment of the present application, as Fig.13 As shown, in S605, when receiving the keyboard rewrite message body uploaded by the client, the server generates rewrite secure keyboard rendering parameters based on the keyboard rewrite message body and sends them to the client, including:

[0218] In step S1301, the server parses the keyboard rewrite message body through a decryption algorithm to obtain keyboard rewrite interaction data;

[0219] In step S1302, the server converts the keyboard rewriting interaction data into a keyboard operation event;

[0220] In step S1303, the server redraws the keyboard video stream data and the secure keyboard rendering data based on the keyboard operation event to obtain rewritten secure keyboard rendering parameters.

[0221] In an embodiment of the present application, when the server receives a keyboard rewrite message body uploaded by the client, the server can parse the keyboard rewrite message body and convert the parsed keyboard rewrite interaction data into a keyboard operation event. Based on the keyboard operation event, the keyboard video stream data and the secure keyboard rendering data are redrawn to obtain the rewritten secure keyboard rendering parameters. Then, the rewritten secure keyboard rendering parameters can be sent to the client so that the client can perform secure keyboard rendering based on the rewritten secure keyboard rendering parameters to obtain the redrawn secure keyboard displayed on the client's display page, so as to better meet the user's personalized needs for the secure keyboard.

[0222] Specifically, Fig.14 As shown, when the server receives the keyboard rewrite message body uploaded by the client, the server can parse the keyboard rewrite message body to obtain the keyboard rewrite interaction data. Specifically, the server can parse the keyboard rewrite interaction data through a decryption algorithm (for example, an SM2 decryption algorithm) to obtain the parsed keyboard rewrite interaction data, so that the server can convert the parsed keyboard rewrite interaction data into keyboard operation events, for example, clicking a certain key, switching the keyboard, confirming, deleting, and retracting the keyboard. Then, based on the keyboard operation event, the server can redraw the keyboard image and organize it to obtain the corresponding video stream data, as well as adjust the custom rendering parameters and redraw the image to obtain the rewrite security keyboard rendering parameters, and send them to the client (such as Fig.14 The sent data is re-rendered as indicated).

[0223] The following is a detailed description of the generation device of the security keyboard in this application. Fig. 20 , Fig. 20 This is a schematic diagram of an embodiment of a device for generating a secure keyboard in an embodiment of the present application. The device for generating a secure keyboard 20 includes:

[0224] The processing unit 201 is configured to, when receiving a secure keyboard call request carrying a session identifier sent by a client, send secure keyboard rendering parameters corresponding to the session identifier to the client;

[0225] The processing unit 201 is further configured to decode the secure keyboard rendering parameters to obtain keyboard stream data when the client receives the secure keyboard rendering parameters;

[0226] The display unit 202 is used for the client to render a secure keyboard based on the keyboard stream data, obtain an initial secure keyboard, and display the initial secure keyboard on a display page of the client;

[0227] The processing unit 201 is further configured to, when the client receives an interaction event corresponding to the initial secure keyboard, transmit the interaction event to the event receiving layer, and perform keyboard rewriting on the interaction event through the event receiving layer to obtain keyboard rewriting interaction data;

[0228] The processing unit 201 is further configured to generate a rewrite safe keyboard rendering parameter based on the keyboard rewrite interaction data when receiving the keyboard rewrite interaction data uploaded by the client, and send the rewrite safe keyboard rendering parameter to the client;

[0229] The processing unit 201 is also used for the client to decode the rewrite secure keyboard rendering parameters to obtain the rewrite keyboard stream data;

[0230] The display unit 202 is also used for the client to render a secure keyboard based on the rewritten keyboard stream data, obtain a redrawn secure keyboard, and display the redrawn secure keyboard on a display page of the client.

[0231] Optionally, in the above Fig. 20 On the basis of the corresponding embodiment, in another embodiment of the device for generating a secure keyboard provided in the embodiment of the present application, the processing unit 201 can be specifically used for:

[0232] When receiving a secure keyboard call request carrying a session identifier from a client, the server sends secure keyboard rendering parameters corresponding to the session identifier to the client through a streaming protocol, wherein the secure keyboard rendering parameters at least include a keyboard diagram, a keyboard color, and a key spacing;

[0233] The processing unit 201 may be specifically configured to: when the client receives the secure keyboard rendering parameters through the streaming media protocol, the client decodes the secure keyboard rendering parameters to obtain keyboard stream data.

[0234] Optionally, in the above Fig. 20 On the basis of the corresponding embodiment, in another embodiment of the device for generating a secure keyboard provided in the embodiment of the present application, the processing unit 201 can be specifically used for:

[0235] The client reads a frame of video data through the streaming media protocol and converts the data into objects to obtain keyboard video stream data;

[0236] The client decodes the secure keyboard rendering parameter according to the decoding algorithm corresponding to the session identifier to obtain the decoded secure keyboard rendering parameter.

[0237] Optionally, in the above Fig. 20 On the basis of the corresponding embodiment, in another embodiment of the device for generating a secure keyboard provided in the embodiment of the present application,

[0238] The acquisition unit 203 is used to acquire the position of the player of the client and initialize the secure keyboard drawing window based on the position of the player, wherein the player is used to support the operating environment of the secure keyboard, and the player includes a view layer and an event receiving layer, and the view layer is located at an upper level of the event receiving layer;

[0239] The display unit 202 can be specifically used to: initialize the secure keyboard drawing window, draw and render the video stream data and the decoded secure keyboard rendering parameters, obtain the initial secure keyboard and the keyboard style corresponding to the initial secure keyboard, and display the initial secure keyboard on the display page of the client, wherein the keyboard style includes at least the key spacing, keyboard color and key text.

[0240] Optionally, in the above Fig. 20 On the basis of the corresponding embodiment, in another embodiment of the device for generating a secure keyboard provided in the embodiment of the present application, the processing unit 201 can be specifically used for:

[0241] Adjust the style of the initial secure keyboard based on the keyboard style and generate interaction events;

[0242] The client passes the interaction event from the view layer corresponding to the initial security keyboard to the event receiving layer through the event hub;

[0243] Respond to the interaction event through the event receiving layer, and obtain the interaction behavior coordinates and interaction behavior path corresponding to the interaction behavior on the display page based on the interaction event;

[0244] Encapsulate the interaction behavior coordinates and interaction behavior path into a keyboard rewrite message body and upload it to the server through the streaming media protocol;

[0245] The processing unit 201 may be specifically configured to: upon receiving a keyboard rewrite message body uploaded by a client, the server generates rewrite security keyboard rendering parameters based on the keyboard rewrite message body, and sends the parameters to the client.

[0246] Optionally, in the above Fig. 20On the basis of the corresponding embodiment, in another embodiment of the device for generating a secure keyboard provided in the embodiment of the present application,

[0247] The storage unit 204 is used to convert the decoded secure keyboard rendering parameters into data according to a dictionary format and store the converted data in the client.

[0248] Optionally, in the above Fig. 20 On the basis of the corresponding embodiment, in another embodiment of the device for generating a secure keyboard provided in the embodiment of the present application,

[0249] The processing unit 201 is further used to construct a keyboard mirroring platform for the server, and modify the network protocol corresponding to each server node in the keyboard mirroring platform to a long link;

[0250] The processing unit 201 is further used to establish a communication protocol between the client and the server corresponding to each server node based on the long link, wherein the communication protocol is a streaming media protocol.

[0251] Optionally, in the above Fig. 20 On the basis of the corresponding embodiment, in another embodiment of the device for generating a secure keyboard provided in the embodiment of the present application,

[0252] The acquisition unit 203 is further used to acquire the network behavior currently running in the background of the client, wherein the network behavior includes one or more of refreshing, downloading, uploading or storing;

[0253] The acquisition unit 203 is further used to acquire the operating environment quality value corresponding to the client based on the current background network behavior;

[0254] The processing unit 201 is further configured to determine the network behavior to be suspended from the network behaviors currently running in the background based on the environment quality threshold range that the running environment quality value complies with, and perform a suspension operation on the network behavior to be suspended.

[0255] Optionally, in the above Fig. 20 On the basis of the corresponding embodiment, in another embodiment of the device for generating a secure keyboard provided in the embodiment of the present application,

[0256] The acquisition unit 203 is further configured to, when the server receives a session establishment request sent by the client, acquire the encryption identifier corresponding to the client based on the session request;

[0257] The processing unit 201 is also used for the server to establish a session authentication between the server and the client based on the encryption identifier.

[0258] Optionally, in the above Fig. 20On the basis of the corresponding embodiment, in another embodiment of the device for generating a secure keyboard provided in the embodiment of the present application, the processing unit 201 can be specifically used for:

[0259] The server decrypts the encrypted identifier. If the decryption result is successful, a session identifier is generated and sent to the client for binding.

[0260] If the decryption result is a decryption failure, a session establishment failure prompt is sent to the client.

[0261] Optionally, in the above Fig. 20 On the basis of the corresponding embodiment, in another embodiment of the device for generating a secure keyboard provided in the embodiment of the present application, the acquisition unit 203 can be specifically used for:

[0262] When the server receives the session establishment request sent by the client, the server obtains the object identity based on the session request, and sends the object identity authentication data corresponding to the object identity, the public key corresponding to the object identity authentication data, and the keyboard type supported by the current object identity to the client;

[0263] The client randomly generates a key and encrypts the key based on the public key in the object authentication data to obtain an encrypted identifier;

[0264] The processing unit 201 may be specifically used for:

[0265] The server decrypts the encrypted identifier using the private key corresponding to the object identity authentication data. If the decryption result is successful, a session identifier is generated.

[0266] The server establishes a mapping relationship between the session identifier and the key, and sends the session identifier after the mapping relationship is established to the client, so that the client binds the session identifier with the object identifier.

[0267] Optionally, in the above Fig. 20 On the basis of the corresponding embodiment, in another embodiment of the device for generating a secure keyboard provided in the embodiment of the present application, the processing unit 201 can be specifically used for:

[0268] The server parses the keyboard rewrite message body through a decryption algorithm to obtain the keyboard rewrite interaction data;

[0269] The server converts the keyboard rewrite interaction data into keyboard operation events;

[0270] The server redraws the keyboard video stream data and the secure keyboard rendering data based on the keyboard operation event to obtain the rewritten secure keyboard rendering parameters.

[0271] On the other hand, the present application provides another schematic diagram of a computer device, such as Fig.21 As shown, Fig.21 3 is a schematic diagram of a computer device structure provided in an embodiment of the present application. The computer device 300 may have relatively large differences due to different configurations or performances, and may include one or more central processing units (CPU) 310 (for example, one or more processors) and a memory 320, and one or more storage media 330 (for example, one or more mass storage devices) storing application programs 331 or data 332. Among them, the memory 320 and the storage medium 330 may be temporary storage or permanent storage. The program stored in the storage medium 330 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the computer device 300. Furthermore, the central processing unit 310 may be configured to communicate with the storage medium 330 to execute a series of instruction operations in the storage medium 330 on the computer device 300.

[0272] The computer device 300 may also include one or more power supplies 340, one or more wired or wireless network interfaces 350, one or more input and output interfaces 360, and / or one or more operating systems 333, such as Windows Server 2000, Windows Server 2003, Windows Server 2003E, Windows Server 2003R ... TM , Mac OS X TM , Unix TM ,Linux TM , FreeBSD TM etc.

[0273] The computer device 300 is also used to execute the following steps: Figures 2 to 13 The steps in the corresponding embodiments.

[0274] Another aspect of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the following Figures 2 to 13 The illustrated embodiments describe steps in a method.

[0275] Another aspect of the present application provides a computer program product comprising a computer program, which, when executed by a processor, implements the following Figures 2 to 13 The illustrated embodiments describe steps in a method.

[0276] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0277] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0278] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0279] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0280] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

Claims

1. A method for generating a secure keyboard, characterized in that: include: When receiving a secure keyboard call request carrying a session identifier sent by a client, the server sends secure keyboard rendering parameters corresponding to the session identifier to the client; When the client receives the secure keyboard rendering parameters, the client renders based on the secure keyboard rendering parameters to obtain an initial secure keyboard, and displays the initial secure keyboard on a display page of the client; When the client receives an interaction event of the target object with respect to the initial secure keyboard, the interaction event is transmitted to an event receiving layer in the client, and the interaction event is keyboard rewritten by the event receiving layer to obtain keyboard rewriting interaction data; When receiving the keyboard rewriting interaction data uploaded by the client, the server generates a rewriting safety keyboard rendering parameter based on the keyboard rewriting interaction data, and sends it to the client; The client performs secure keyboard rendering on the rewritten secure keyboard rendering parameters to obtain a redrawn secure keyboard, and displays the redrawn secure keyboard on a display page of the client.

2. The method according to claim 1, characterized in that: When receiving the secure keyboard call request carrying the session identifier sent by the client, the server sends the secure keyboard rendering parameters corresponding to the session identifier to the client, including: When receiving a secure keyboard call request carrying a session identifier from a client, the server sends secure keyboard rendering parameters corresponding to the session identifier to the client through a streaming protocol, wherein the secure keyboard rendering parameters at least include a keyboard diagram, a keyboard color, and a key spacing; When the client receives the secure keyboard rendering parameter, the client renders based on the secure keyboard rendering parameter to obtain an initial secure keyboard, and displays the initial secure keyboard on a display page of the client, including: When the client receives the secure keyboard rendering parameters through the streaming protocol, the client renders based on the secure keyboard rendering parameters to obtain an initial secure keyboard, and displays the initial secure keyboard on a display page of the client.

3. The method according to claim 2, characterized in that When the client receives the secure keyboard rendering parameter through the streaming protocol, the client renders an initial secure keyboard based on the secure keyboard rendering parameter, and displays the initial secure keyboard on a display page of the client, including: The client reads at least one frame of video data through the streaming media protocol and converts the data into an object to obtain keyboard video stream data; The client decodes the secure keyboard rendering parameter according to a decoding algorithm corresponding to the session identifier to obtain a decoded secure keyboard rendering parameter; The client performs secure keyboard rendering based on the keyboard video stream data and the decoded secure keyboard rendering parameters to obtain the initial secure keyboard, and displays the initial secure keyboard on a display page of the client.

4. The method according to claim 3, characterized in that The client reads at least one frame of video data through the streaming media protocol to convert the data into a data object, and before obtaining the video stream data, the method further includes: Acquire the position of the player of the client, and initialize the secure keyboard drawing window based on the position of the player, wherein the player is used to support the operating environment of the secure keyboard, the player includes a view layer and the event receiving layer, and the view layer is located at an upper level of the event receiving layer; The client performs secure keyboard rendering based on the keyboard video stream data and the decoded secure keyboard rendering parameters to obtain the initial secure keyboard, and displays the initial secure keyboard on a display page of the client, including: The video stream data and the decoded secure keyboard rendering parameters are image-drawn and rendered through the initialized secure keyboard drawing window to obtain an initial secure keyboard and a keyboard style corresponding to the initial secure keyboard, and the initial secure keyboard is displayed on a display page of the client, wherein the keyboard style includes at least key spacing, keyboard color, and key text.

5. The method according to claim 4, characterized in that The keyboard rewriting interaction data includes interaction behavior coordinates and interaction behavior paths; when the client receives an interaction event of the target object for the initial secure keyboard, the interaction event is transmitted to the event receiving layer in the client, and the interaction event is keyboard rewritten by the event receiving layer to obtain keyboard rewriting interaction data, including: Adjusting the style of the initial secure keyboard based on the keyboard style, and generating the interaction event; The client transmits the interactive event from the view layer corresponding to the initial security keyboard to the event receiving layer through the event hub; Responding to the interaction event through the event receiving layer, and acquiring the interaction behavior coordinates and the interaction behavior path corresponding to the interaction behavior on the display page based on the interaction event; Encapsulating the interaction behavior coordinates and the interaction behavior path into a keyboard rewriting message body, and uploading it to the server through the streaming media protocol; When receiving the keyboard rewriting interaction data uploaded by the client, the server generates a rewriting safety keyboard rendering parameter based on the keyboard rewriting interaction data and sends it to the client, including: When receiving the keyboard rewrite message body uploaded by the client, the server generates rewrite security keyboard rendering parameters based on the keyboard rewrite message body and sends them to the client.

6. The method according to claim 3, characterized in that After the client decodes the secure keyboard rendering parameter according to the decoding algorithm corresponding to the session identifier to obtain the decoded secure keyboard rendering parameter, the method further includes: The decoded secure keyboard rendering parameters are converted into data according to a dictionary format, and the converted data is stored in the client.

7. The method according to claim 2, characterized in that When receiving a secure keyboard call request carrying a session identifier sent by a client, the server sends secure keyboard rendering parameters corresponding to the session identifier to the client through a streaming media protocol, and the method further includes: Constructing a keyboard mirroring platform for the server, and modifying the network protocol corresponding to each server node in the keyboard mirroring platform to a long link; Based on the long link, a communication protocol is established between the client corresponding to each server node and the server, wherein the communication protocol is a streaming media protocol.

8. The method according to claim 7, characterized in that After establishing a communication protocol between the client corresponding to each server node and the server based on the long link, the method further includes: Acquire the network behavior currently running in the background of the client, wherein the network behavior includes one or more of refreshing, downloading, uploading or storing; Based on the network behavior currently running in the background, obtaining a running environment quality value corresponding to the client; Based on the environment quality threshold range that the running environment quality value complies with, the network behavior to be suspended is determined from the network behavior currently running in the background, and a suspension operation is performed on the network behavior to be suspended.

9. The method according to claim 1, characterized in that: When receiving a secure keyboard call request carrying a session identifier sent by a client, the server sends secure keyboard rendering parameters corresponding to the session identifier to the client, and the method further includes: When the server receives the session establishment request sent by the client, the server obtains the encryption identifier corresponding to the client based on the session request; The server establishes a session authentication between the server and the client based on the encryption identifier.

10. The method according to claim 9, characterized in that The server establishes session authentication between the server and the client based on the encryption identifier, including: The server performs a decryption operation on the encrypted identifier, and if the decryption result is successful, generates a session identifier, and sends the session identifier to the client for binding; If the decryption result is a decryption failure, a session establishment failure prompt is sent to the client.

11. The method according to claim 10, characterized in that When the server receives the session establishment request sent by the client, the server obtains the encryption identifier corresponding to the client based on the session request, including: When the server receives the session establishment request sent by the client, the server obtains the object identity based on the session request, and sends the object identity verification data corresponding to the object identity, the public key corresponding to the object identity verification data, and the keyboard type supported by the current object identity to the client; The client randomly generates a key, and encrypts the key based on the public key in the object identity authentication data to obtain the encrypted identifier; The server performs a decryption operation on the encrypted identifier, and if the decryption result is successful, generates a session identifier, and sends the session identifier to the client for binding, including: The server decrypts the encrypted identifier using the private key corresponding to the object identity authentication data, and generates a session identifier if the decryption result is successful; The server establishes a mapping relationship between the session identifier and the key, and sends the session identifier after the mapping relationship is established to the client, so that the client binds the session identifier with the object identifier.

12. The method according to claim 5, characterized in that When receiving the keyboard rewrite message body uploaded by the client, the server generates rewrite security keyboard rendering parameters based on the keyboard rewrite message body and sends them to the client, including: The server parses the keyboard rewrite message body through the decryption algorithm to obtain keyboard rewrite interaction data; The server converts the keyboard rewriting interaction data into a keyboard operation event; The server redraws the keyboard video stream data and the secure keyboard rendering data based on the keyboard operation event to obtain the rewritten secure keyboard rendering parameters.

13. A device for generating a secure keyboard, characterized in that: include: The processing unit is configured to, when receiving a secure keyboard call request carrying a session identifier sent by a client, cause the server to send secure keyboard rendering parameters corresponding to the session identifier to the client; A display unit, configured to, when the client receives the secure keyboard rendering parameter, render the client based on the secure keyboard rendering parameter to obtain an initial secure keyboard, and display the initial secure keyboard on a display page of the client; The processing unit is further configured to, when the client receives an interaction event of the target object with respect to the initial secure keyboard, transmit the interaction event to an event receiving layer in the client, and perform keyboard rewriting on the interaction event through the event receiving layer to obtain keyboard rewriting interaction data; The processing unit is further configured to, when receiving the keyboard rewriting interaction data uploaded by the client, generate a rewriting safety keyboard rendering parameter based on the keyboard rewriting interaction data by the server, and send the rewriting safety keyboard rendering parameter to the client; The display unit is also used for the client to perform secure keyboard rendering on the rewritten secure keyboard rendering parameters to obtain a redrawn secure keyboard, and display the redrawn secure keyboard on a display page of the client.

14. A computer device comprising a memory, a processor and a bus system, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 12 are implemented; The bus system is used to connect the memory and the processor so that the memory and the processor can communicate with each other.

15. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 12 are implemented.

16. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 12 are implemented.