Data collaborative input method, device and electronic device

Abstract

The present invention provides a data collaborative input method, device, and electronic device, relating to the field of human-computer interaction technology, and applied to XR devices. The method comprises: determining a current usage scenario; determining a data transmission mode that matches the current usage scenario based on the current usage scenario; wherein the data transmission mode includes a data transmission method and a data encryption method; sending a data transmission request to a target device; wherein the data transmission request includes the data transmission mode; and receiving target data transmitted by the target device according to the data transmission mode. Through the method provided by the present invention, a data transmission mode that matches the current usage scenario is determined based on the current usage scenario, and combined with the matching encryption level, data can be efficiently and securely transmitted between the XR device and the target mobile device, thereby improving the user experience.

Description

Technical Field

[0001] The present invention relates to the field of human-computer interaction technology, and in particular to a data collaborative input method, device and electronic equipment. Background Art

[0002] With the rapid development of XR (Extended Reality) technology, XR devices are increasingly being used in consumer electronics. When users interact with virtual information through XR devices, they often need to input text on mobile devices (such as phones) to enable cross-device collaboration, such as entering payment passwords in shopping scenarios or sending messages in social apps. However, due to size and power consumption limitations, XR devices lack built-in efficient input modules and must rely on mobile devices as an input medium. This makes data transmission efficiency, security, and adaptability between the two devices core technical bottlenecks that affect the user experience.

[0003] The current mainstream solutions use static transmission protocols and fixed encryption strategies, which cannot effectively adapt to usage scenarios, resulting in waste of resources, low transmission efficiency, and poor user experience.

[0004] How to achieve dynamic adaptation of data transmission to various scenarios is a technical problem that needs to be solved at present. Summary of the Invention

[0005] The present invention provides a data collaborative input method, device and electronic device to solve the defects in the prior art.

[0006] The present invention provides a data collaborative input method, which is applied to an XR device and includes the following steps:

[0007] Determine the current usage scenario;

[0008] Determine, based on the current usage scenario, a data transmission mode that matches the current usage scenario; wherein the data transmission mode includes: a data transmission mode and a data encryption mode;

[0009] Sending a data transmission request to a target device; wherein the data transmission request includes: the data transmission mode;

[0010] Receive target data transmitted by the target device according to the data transmission mode.

[0011] According to a data collaborative input method provided by the present invention, determining the current usage scenario includes:

[0012] Detect the element type in the target area where the user is looking;

[0013] When the time the user gazes at the target area is greater than a preset time threshold, the current usage scenario is determined according to the element type of the target area.

[0014] According to a data collaborative input method provided by the present invention, determining a data transmission mode matching the current usage scenario according to the current usage scenario includes:

[0015] Determining a data encryption level for the current usage scenario based on the current usage scenario;

[0016] Determine the corresponding data encryption method according to the data encryption level of the current usage scenario.

[0017] According to a data collaborative input method provided by the present invention, the data transmission request further includes: a keyboard type;

[0018] After determining the current usage scenario, the method further includes:

[0019] A keyboard type that matches the current usage scenario is determined based on a preset keyboard template corresponding to the current usage scenario or historical user behavior data under the current usage scenario.

[0020] According to a data collaborative input method provided by the present invention, the data transmission mode further includes: at least two data transmission modes and priorities of the at least two data transmission modes;

[0021] The determining, according to the current usage scenario, a data transmission mode matching the current usage scenario includes:

[0022] Determining, according to the current usage scenario, at least two data transmission modes matching the current usage scenario and priorities of the at least two data transmission modes;

[0023] The at least two data transmission modes are switched according to the priority.

[0024] According to a data collaborative input method provided by the present invention, after receiving the target data transmitted by the target device according to the data transmission mode, the method further includes:

[0025] The target data is parsed, virtual information is generated based on the user input text obtained by the parsing, and the virtual information is displayed in the target area of ​​the target device.

[0026] The present invention provides a data collaborative input method, which is applied to a target device and includes the following steps:

[0027] Receive a data transmission request sent by the XR device, where the data transmission request includes a data transmission mode determined by the XR device according to a current application scenario, where the data transmission mode includes a data transmission method and a data encryption method;

[0028] The target data is transmitted to the XR device according to the data transmission mode.

[0029] The present invention also provides a data collaborative input device, which is applied to XR equipment and includes the following modules:

[0030] A first determination module is used to determine the current usage scenario;

[0031] A second determining module is configured to determine a data transmission mode that matches the current usage scenario according to the current usage scenario; wherein the data transmission mode includes: a data transmission mode and a data encryption mode;

[0032] A request module, configured to send a data transmission request to a target device; wherein the data transmission request includes: the data transmission mode;

[0033] The receiving module is configured to receive target data transmitted by the target device according to the data transmission mode.

[0034] The present invention also provides a data collaborative input device, which is applied to a target device and includes the following modules:

[0035] A receiving module, configured to receive a data transmission request sent by an XR device, wherein the data transmission request includes a data transmission mode determined by the XR device according to a current usage scenario, wherein the data transmission mode includes a data transmission method and a data encryption method;

[0036] The transmission module is used to transmit the target data to the XR device according to the data transmission mode.

[0037] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, any one of the above-described collaborative data input methods is implemented.

[0038] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-described collaborative data input methods.

[0039] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any of the above-mentioned collaborative data input methods.

[0040] The present invention provides a data collaborative input method, device, and electronic device, which are applied to XR devices, by determining the current usage scenario; determining a data transmission mode that matches the current usage scenario based on the current usage scenario; wherein the data transmission mode includes: a data transmission method and a data encryption method; sending a data transmission request to a target device; wherein the data transmission request includes: the data transmission mode; and receiving target data transmitted by the target device according to the data transmission mode. It can be seen that the present invention determines a data transmission mode that matches the current usage scenario based on the current usage scenario, and combines the matching encryption level to enable data to be efficiently and securely transmitted between the XR device and the target mobile device, thereby improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction is given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0042] Figure 1 It is a system architecture diagram of the data collaborative input method provided by the present invention.

[0043] Figure 2 This is one of the flow charts of the data collaborative input method provided by the present invention.

[0044] Figure 3 This is an application scenario diagram of the data collaborative input method provided by the present invention.

[0045] Figure 4 It is a scene diagram of the data collaborative input method provided by the present invention in one embodiment.

[0046] Figure 5 This is the second flow chart of the data collaborative input method provided by the present invention.

[0047] Figure 6 This is one of the structural diagrams of the data collaborative input device provided by the present invention.

[0048] Figure 7 This is the second structural diagram of the data collaborative input device provided by the present invention.

[0049] Figure 8 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0050] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0051] The following combination Figures 1-8 A data collaborative input method, device and electronic device of the present invention are described.

[0052] Before describing the method of the present invention in detail, we first provide a schematic explanation of the terminology used in this invention. XR (Extended Reality) refers to the use of computers to combine the real and virtual worlds to create a virtual environment that allows for human-computer interaction. This is also a general term for various technologies, including AR, VR, and MR.

[0053] MR (Mixed Reality): Mixed reality technology refers to an interactive environment that integrates the physical world and digital objects.

[0054] BLE (Bluetooth Low Energy): Bluetooth low energy protocol, using the 2.4GHz ISM band, with a typical power consumption of 0.01-0.5W.

[0055] Dynamic protocol stack: refers to an intelligent decision-making system that dynamically selects the BLE / Wi-Fi / Socket transmission channel based on signal strength (RSSI), latency, and data load size.

[0056] Contextual Security Model: An assessment system that automatically matches encryption strategies based on the sensitivity of input scenarios (payment / social / navigation), including a three-level risk assessment matrix.

[0057] Differential transmission: An optimized algorithm that sends only changed data rather than complete data packets, and uses the 0x01 / 0x00 flag bits to identify data differences.

[0058] Focus retention detection: MR glasses use the eye tracking module to determine whether the user's gaze on the input box exceeds the threshold (default 800ms±50ms) as a trigger mechanism.

[0059] Figure 1 This is a system architecture diagram of the data collaborative input method provided by the present invention, such as Figure 1As shown, the system architecture provided by the present invention includes a mobile device 102, a transmission channel 104 and an XR device 106. The transmission channel 104 is used to provide a medium for a communication link between the mobile device 102 and the XR device 106, and supports dynamic switching of three modes: WIFI local area network, Bluetooth BLE and server Socket.

[0060] A user can use a mobile device 102 to interact with an XR device 106 via a transmission channel 104 to receive or send messages. Various communication client applications can be installed on the mobile device 102, such as video applications, live broadcast applications, instant messaging tools, email clients, social platform software, etc.

[0061] Mobile device 102 can be either hardware or software. If the mobile device 102 is hardware, it can be any electronic device with a display screen, including but not limited to smartphones, tablet computers, e-book readers, laptop computers, and the like. If the mobile device 102 is software, it can be installed in any of the electronic devices listed above. It can be implemented as multiple software programs or software modules (e.g., multiple software programs or software modules used to provide distributed services) or as a single software program or software module. The XR device 106 can be a VR device, an AR device, or a MR device. These are not specifically limited here.

[0062] Figure 2 This is one of the flow charts of the data collaborative input method provided by the present invention, such as Figure 2 As shown, applied to an XR device, the method includes the following:

[0063] Step 100: Determine the current usage scenario.

[0064] It should be noted that this embodiment is implemented using an XR device, and the following explanation will use MR glasses as an example. MR glasses use a built-in eye tracking module (sampling rate ≥ 120Hz) to detect focus retention and capture the coordinates of the user's gaze focus in real time. For example, if the user gazes at an input box for 800ms or actively triggers the action, the current usage scenario is determined.

[0065] Current usage scenarios can be either high-frequency short text input or high-frequency long text input. For example, chatting and information retrieval correspond to high-frequency short text input scenarios, while social media and content creation correspond to high-frequency long text input scenarios. Displacement parameters can also be used to further determine whether the user is in a mobile scenario.

[0066] Step 200: Determine a data transmission mode that matches the current usage scenario based on the current usage scenario; wherein the data transmission mode includes: a data transmission method and a data encryption method.

[0067] This embodiment defines the core process of collaborative input between MR glasses and target devices (taking mobile terminals as an example): by dynamically identifying the current usage scenario and matching the optimal data transmission mode (data transmission method + data encryption method), efficient and secure data interaction is achieved.

[0068] Optionally, for scenarios with high-frequency input of short texts, the data transmission mode includes: using efficient encoding methods to reduce the amount of data transmitted in a single BLE transmission; for scenarios with high-frequency input of long texts, to ensure real-time data transmission, the data transmission mode includes: using real-time transmission methods (such as inputting 3 words and transmitting them at the same time) to avoid lag caused by one-time transmission of long texts.

[0069] Furthermore, the data encryption level of the current usage scenario can be determined, and the corresponding data encryption method can be determined according to the data encryption level.

[0070] Step 300: Send a data transmission request to a target device; wherein the data transmission request includes: the data transmission mode.

[0071] Specifically, the MR glasses may further include a keyboard type request (such as a numeric keyboard or an alphabetic keyboard), whether it is a high-frequency input, etc. according to the currently determined data transmission mode, encapsulate it into an instruction packet unit8list, and send a data transmission request to the mobile terminal based on the instruction packet.

[0072] Step 400: Receive target data transmitted by the target device according to the data transmission mode.

[0073] Specifically, the mobile terminal receives the MR glasses end instruction package unit8list, and parses the corresponding json data according to the corresponding encryption and decryption key. The content of the json data may include data encryption level, data transmission method priority, whether it is high frequency, keyboard type, etc.

[0074] Figure 3 This is an application scenario diagram of the data collaborative input method provided by the present invention. Figure 3 , the data collaborative input method provided by the present invention is specifically described.

[0075] The MR glasses detect eye tracking module and find that the user has been looking at the comment box for more than 800ms. The parsed input box label is <comment>, determined to be a "social long text scenario".

[0076] The data transmission mode is further determined based on the scenario. Among them, the data transmission method prefers WIFI local area network (long texts require high bandwidth), and the data encryption method is medium encryption level (AES-256, because it contains user privacy content).

[0077] Further, based on the user's historical behavior, it is predicted that a full keyboard is required and a command package is generated.

[0078] After receiving the command, the mobile phone pops up a full keyboard and the user enters "The weather is really nice today". Every three characters entered are encrypted and transmitted to the MR glasses via WIFI (differential transmission technology).

[0079] The MR glasses process the decrypted data packet and render it in real time to the comment box. During this process, if the Wi-Fi signal weakens (RSSI < -85dBm), it automatically switches to BLE transmission for the remaining content. If the network is disconnected after sending "Today's weather," the unsent characters are temporarily stored in the phone's local SQLite file. Once the network is restored, the word "really good" is prioritized for synchronization to the MR glasses.

[0080] Rendering effect: The MR glasses display the full comment "What a nice weather today" and simultaneously display it in the preview area on the mobile phone.

[0081] The above is a description of the steps of the data collaborative input method provided by the present invention. From the description of the above steps, it can be seen that the data collaborative input method provided by the present invention is applied to XR devices, by determining the current usage scenario; according to the current usage scenario, determining the data transmission mode that matches the current usage scenario; wherein the data transmission mode includes: data transmission mode and data encryption mode; sending a data transmission request to the target device; wherein the data transmission request includes: the data transmission mode; receiving the target data transmitted by the target device according to the data transmission mode. It can be seen that the present invention determines the data transmission mode that matches the current usage scenario according to the current usage scenario, combined with the matching encryption level, so that data can be efficiently and securely transmitted between the XR device and the target mobile device, thereby improving the user experience.

[0082] Based on the above embodiment, in this embodiment, step 100 of determining the current usage scenario includes:

[0083] Step 110: Detect the element type of the target area that the user is gazing at.

[0084] Specifically, the target area may be, for example, an input box, a comment box, etc., and the type of input that the user wants to make, such as a number or text, is determined according to the element type of the target area.

[0085] Step 120: When the time the user gazes at the target area is greater than a preset time threshold, determine the current usage scenario according to the element type of the target area.

[0086] In one embodiment, when the time the user gazes at the input box is greater than a preset time threshold (for example, 800ms), based on the first response position of the input box cursor, the MR glasses analyze and obtain the current usage scenario based on the current screen image element data or the currently used application, including: by capturing the current input box system keyboard mode or the data analyzed in the above embodiments, confirming the short text high-frequency input scenario or the long text high-frequency input scenario; for example, chatting, information retrieval, etc. correspond to short text high-frequency input scenarios; social media, content creation, etc. correspond to long text high-frequency input scenarios.

[0087] Furthermore, MR glasses can also detect displacement parameters to determine whether the user is in a mobile scenario and whether real-time monitoring of data transmission stability is necessary. If a mobile scenario is detected, the smart glasses are triggered to monitor data transmission stability in real time and determine the corresponding data transmission mode based on the data transmission stability.

[0088] Furthermore, the corresponding data transmission mode can be determined by combining mobile scenarios and high-frequency input scenarios. For example, when a user is walking and chatting, it is necessary to ensure both real-time data transmission and data transmission stability.

[0089] The data collaborative input method provided in this embodiment uses an eye tracking module to detect the type of UI element (such as a password box or a search box) that the user is gazing at, and accurately determines the scene based on the gaze duration, thereby avoiding manual operation and improving the naturalness of interaction.

[0090] Based on the above embodiment, in this embodiment, step 200 determines a data transmission mode matching the current usage scenario according to the current usage scenario, including:

[0091] Step 210: Determine the data encryption level of the current usage scenario according to the current usage scenario.

[0092] Step 220: Determine a corresponding data encryption method according to the data encryption level of the current usage scenario.

[0093] It should be noted that data encryption levels include: high encryption level, medium encryption level and low encryption level.

[0094] Step 220 specifically includes:

[0095] Step 221: When it is determined that the data encryption level of the current usage scenario is a high encryption level, the data encryption mode is determined to be advanced encryption.

[0096] Step 222: When it is determined that the data encryption level of the current usage scenario is the medium encryption level, the data encryption mode is determined to be intermediate encryption.

[0097] Step 223: When it is determined that the data encryption level of the current usage scenario is a low encryption level, the data encryption mode is determined to be low-level encryption.

[0098] Specifically, the data encryption level of the current usage scenario is first determined, and the data encryption method is determined according to the data encryption level.

[0099] For example, for payment, login and password pages or apps, they correspond to high encryption levels and use advanced encryption methods; for navigation, scheduling pages or apps, they correspond to medium encryption levels and use intermediate encryption methods; for information retrieval pages or apps, they correspond to low encryption levels and use low-level encryption methods.

[0100] It should be noted that SM4 is a national secret algorithm that emphasizes hardware implementation efficiency. It is suitable for embedded systems and IoT devices and is mainly used in security fields such as finance and government affairs. Two-way authentication means that the two communicating parties (such as client and server) verify each other's identities to ensure mutual trust and prevent man-in-the-middle attacks. AES-256 is an international high-intensity encryption with high security, suitable for top security needs such as military and government secrets. AES-128 is an international general standard with fast speed and sufficient security, and is widely used in HTTPS, WIFI, and daily data encryption.

[0101] Furthermore, advanced encryption can adopt SM4+ two-way authentication, which can be applied to financial scenarios, for example; intermediate encryption can adopt AES-256 encryption, for example, for scenarios where private data is sent; low-level encryption can adopt AES-128 or no encryption, for example, for scenarios where ordinary text is sent.

[0102] The data collaborative input method provided in this embodiment divides encryption into three levels: high, medium, and low according to the sensitivity of the scenario, and dynamically selects encryption algorithms (such as SM4 / AES-256 / AES-128) to achieve dynamic adaptation of encryption methods and application scenarios.

[0103] Based on the above embodiment, in this embodiment, the data transmission request further includes: keyboard type.

[0104] After determining the current usage scenario in step 100, the method further includes:

[0105] A keyboard type that matches the current usage scenario is determined based on a preset keyboard template corresponding to the current usage scenario or historical user behavior data under the current usage scenario.

[0106] Specifically, to improve user experience, after determining the usage scenario, the MR glasses can also send a keyboard type request to the mobile device based on the usage scenario.

[0107] Optionally, a keyboard type request can be sent to the mobile device based on the commonly used keyboard type corresponding to the usage scenario or the user's historical usage habits in the usage scenario, thereby reducing the waiting time when the user enters data on the mobile device. For example, if the usage habit in the scenario is text input, a request for an alphabetic keyboard will be sent; if the usage habit is numeric input, a request for a numeric keyboard will be sent, and so on.

[0108] Furthermore, the priority of the transmission mode can be determined by determining whether the current input scenario is high-frequency. For example, if the current scenario is high-frequency input, Bluetooth BLE is less effective than Wi-Fi LAN. When determining the priority of the transmission mode, Wi-Fi LAN has a higher priority than Bluetooth BLE.

[0109] Figure 4 FIG. 1 is a schematic diagram of a scenario in an embodiment of the data collaborative input method provided by the present invention, such as Figure 4 As shown, the mobile terminal receives the MR glasses end instruction packet unit8list, and parses the corresponding json data according to the corresponding encryption and decryption key. The content of the json data may include data encryption level, data transmission mode priority, whether it is high frequency, keyboard type, etc. Figure 4 As shown in the following example, "keyboard type": "num" indicates that the keyboard type is numeric keys; "encryption level": 2 indicates that the data encryption level is level 2, which is medium encryption; "transmission priority": ["BLE","WiFi", "5G"] indicates the priority of data transmission mode, with BLE taking priority, WiFi taking second place, and 5G transmission taking last place; "high frequency":"" indicates that the high frequency function is not enabled; "keyboard scenario": "" indicates that the keyboard usage scenario is unspecified; "preferkeyboard": "abbreviation" indicates that the abbreviation input method is preferred.

[0110] In the data collaborative input method provided in this embodiment, the XR device requests a matching keyboard type from the mobile terminal based on the scenario or user's historical behavior, thereby reducing the time spent on manual keyboard switching and improving the user experience.

[0111] Based on the above embodiment, in this embodiment, the data transmission mode further includes: at least two data transmission modes and priorities of the at least two data transmission modes.

[0112] In step 200, determining a data transmission mode that matches the current usage scenario according to the current usage scenario includes:

[0113] Determining, according to the current usage scenario, at least two data transmission modes matching the current usage scenario and priorities of the at least two data transmission modes;

[0114] The at least two data transmission modes are switched according to the priority.

[0115] Specifically, the mobile terminal updates the keyboard type that pops up according to the corresponding content. When the user enters data, data transmission is performed according to the current data transmission mode, including the data encryption level and the data transmission method priority. Data transmission can be performed only according to the determined data transmission priority, or only according to the data transmission stability information. When it is determined that there are at least two data transmission methods according to the data transmission stability information, the data transmission method with a higher priority is selected for data transmission.

[0116] Specifically, the BLE / Wi-Fi / Socket three-mode switching engine triggers real-time monitoring of data transmission stability based on the mobile scenario, and automatically switches the transmission mode based on the real-time monitored data transmission stability. For example:

[0117] 1. Dynamically select the transmission mode based on the RSSI value. For example, when the BLE signal is <-85dBm, it is judged as a weak signal, and the current BLE connection is terminated and automatically switched to the Wi-Fi LAN.

[0118] 2. Monitor Wi-Fi status. When there is no Wi-Fi and BLE is unavailable, it will automatically switch to the server socket and transmit data via 4G / 5G.

[0119] 3. If the above three methods fail, temporarily store the user input content in local SQLite and give priority to transmitting unconfirmed data after synchronization recovery;

[0120] 4. The mobile app uses a priority queue to process requests (response time < 100ms). For example, high-priority processing requests include real-time user input and heartbeat packets, while low-priority processing requests include log reporting and non-critical data.

[0121] 5. Send long texts in batches (256 bytes per packet) through differential transmission technology, that is, generate difference blocks for long texts and only send the changed parts to avoid repeated transmission.

[0122] The data collaborative input method provided in this embodiment realizes cross-device interaction by dynamically adapting the data transmission mode of the scene, thereby improving the efficiency, stability, security and user experience of cross-device interaction.

[0123] Based on the above embodiment, in this embodiment, after step 400 receives the target data transmitted by the target device according to the data transmission mode, the method further includes:

[0124] The target data is parsed, virtual information is generated based on the user input text obtained by the parsing, and the virtual information is displayed in the target area of ​​the target device.

[0125] Specifically, MR glasses obtain data and perform corresponding decryption according to different data transmission methods. The parsed information is displayed in real time in the content box that the user needs to input. At the same time, the mobile terminal keyboard uses context-aware input method and LSTM (Long Short-Term Memory) to predict the next input field type, realizing intelligent mapping of input type-keyboard layout-encryption level.

[0126] Furthermore, during the user input process, the MR glasses can monitor the user's gaze focus in real time to determine the current keyboard layout based on the focus. For example, if the user's gaze focus is detected on the "password box" and the gaze duration exceeds a certain threshold, a corresponding keyboard layout request is sent to the mobile terminal, causing the mobile device to automatically pop up the secure keyboard. If the user's gaze focus is detected on "numbers" or "characters" (achieved through high-precision gaze tracking technology) and the gaze duration exceeds a certain threshold, a corresponding keyboard layout request is sent to the mobile terminal, causing the mobile terminal to automatically pop up the numeric keypad or symbol keyboard.

[0127] In the data collaborative input method provided in this embodiment, the XR device parses the target data to generate virtual information (such as a 3D projection of the input text) and synchronously displays it on the mobile terminal screen, thereby enhancing the real-time feedback of the collaborative input.

[0128] Figure 5 This is the second flow chart of the data collaborative input method provided by the present invention, such as Figure 5 As shown, applied to the target device, the method includes the following:

[0129] Step 510: Receive a data transmission request sent by the XR device, where the data transmission request includes a data transmission mode determined by the XR device according to a current usage scenario, where the data transmission mode includes a data transmission method and a data encryption method.

[0130] It should be noted that the target device is the subject of this embodiment, and the following explanation will use a mobile terminal as an example. XR devices, such as MR glasses, use built-in eye tracking modules (sampling rate ≥ 120Hz) to detect focus retention and capture the coordinates of the user's gaze focus in real time. For example, if the user gazes at an input box for 800ms or actively triggers the action, the current usage scenario is determined.

[0131] Current usage scenarios can be either high-frequency short text input or high-frequency long text input. For example, chatting and information retrieval correspond to high-frequency short text input scenarios, while social media and content creation correspond to high-frequency long text input scenarios. Displacement parameters can also be used to further determine whether the user is in a mobile scenario.

[0132] This embodiment defines the core process of collaborative input with a mobile terminal and MR glasses: by dynamically identifying the current usage scenario and matching the optimal data transmission mode (data transmission method + data encryption method), efficient and secure data interaction is achieved.

[0133] Step 520: Transmit the target data to the XR device according to the data transmission mode.

[0134] The data collaborative input method provided by the present invention is applied to a target device. The method receives a data transmission request from an XR device, the data transmission request including a data transmission mode determined by the XR device based on the current usage scenario, the data transmission mode including a data transmission method and a data encryption method. The method then transmits target data to the XR device in accordance with the data transmission mode. This method determines a data transmission mode that matches the current usage scenario based on the current usage scenario, and combines this with a matching encryption level to enable efficient and secure data transmission between the XR device and the target mobile device, thereby enhancing the user experience.

[0135] Based on the above embodiment, in this embodiment, the data transmission mode further includes: at least two data transmission modes and priorities of the at least two data transmission modes.

[0136] Step 520 transmits the target data to the XR device according to the data transmission mode, including:

[0137] Step 521: Determine a target data transmission mode according to the priorities of the at least two data transmission modes and / or the data transmission stability information generated by real-time monitoring.

[0138] Step 522: Transmit the target data to the XR device according to the target data transmission mode.

[0139] Specifically, when the user inputs data, data transmission is performed according to the current data transmission mode, including the data encryption level and the data transmission method priority. Data transmission can be performed only according to the determined data transmission priority, or only according to the data transmission stability information. Or, when it is determined that there are at least two data transmission methods according to the data transmission stability information, the data transmission method with a higher priority is selected for data transmission.

[0140] Specifically, the BLE / Wi-Fi / Socket three-mode switching engine triggers real-time monitoring of data transmission stability based on the mobile scenario, and automatically switches the transmission mode based on the real-time monitored data transmission stability. For example:

[0141] 1. Dynamically select the transmission mode based on the RSSI value. For example, when the BLE signal is <-85dBm, it is judged as a weak signal, and the current BLE connection is terminated and automatically switched to the Wi-Fi LAN.

[0142] 2. Monitor Wi-Fi status. When there is no Wi-Fi and BLE is unavailable, it will automatically switch to the server socket and transmit data via 4G / 5G.

[0143] 3. If the above three methods fail, temporarily store the user input content in local SQLite and give priority to transmitting unconfirmed data after synchronization recovery;

[0144] 4. The mobile app uses a priority queue to process requests (response time < 100ms). For example, high-priority processing requests include real-time user input and heartbeat packets, while low-priority processing requests include log reporting and non-critical data.

[0145] 5. Send long texts in batches (256 bytes per packet) through differential transmission technology, that is, generate difference blocks for long texts and only send the changed parts to avoid repeated transmission.

[0146] The data collaborative input method provided in this embodiment realizes cross-device interaction by dynamically adapting the data transmission mode of the scene, thereby improving the efficiency, stability, security and user experience of cross-device interaction.

[0147] The data collaborative input device provided by the present invention is described below. The data collaborative input device described below and the data collaborative input method described above can be referenced to each other.

[0148] Figure 6 This is one of the structural diagrams of the data collaborative input device provided by the present invention. Figure 6 As shown, the data collaborative input device provided by the present invention is applied to XR equipment, and the device includes:

[0149] A first determining module 601 is used to determine the current usage scenario;

[0150] A second determining module 602 is configured to determine a data transmission mode that matches the current usage scenario based on the current usage scenario; wherein the data transmission mode includes: a data transmission mode and a data encryption mode;

[0151] The request module 603 is configured to send a data transmission request to the target device; wherein the data transmission request includes: the data transmission mode;

[0152] The first receiving module 604 is configured to receive target data transmitted by the target device according to the data transmission mode.

[0153] The data collaborative input device provided by the present invention is applied to XR devices, by determining the current usage scenario; determining a data transmission mode that matches the current usage scenario based on the current usage scenario; wherein the data transmission mode includes: a data transmission method and a data encryption method; sending a data transmission request to a target device; wherein the data transmission request includes: the data transmission mode; and receiving target data transmitted by the target device according to the data transmission mode. It can be seen that the present invention determines a data transmission mode that matches the current usage scenario based on the current usage scenario, and combines it with the matching encryption level to enable efficient and secure data transmission between the XR device and the target mobile device, thereby improving the user experience.

[0154] Based on the above embodiment, in this embodiment, the first determining module 601 is specifically configured to:

[0155] Detect the element type in the target area where the user is looking;

[0156] When the time the user gazes at the target area is greater than a preset time threshold, the current usage scenario is determined according to the element type of the target area.

[0157] Based on the above embodiment, in this embodiment, the second determining module 602 is specifically configured to:

[0158] Determining a data encryption level for the current usage scenario based on the current usage scenario;

[0159] Determine the corresponding data encryption method according to the data encryption level of the current usage scenario.

[0160] Based on the above embodiment, in this embodiment, the data transmission request further includes: keyboard type;

[0161] The apparatus further includes a third determining module, specifically configured to:

[0162] After the current usage scenario is determined, a keyboard type that matches the current usage scenario is determined based on a preset keyboard template corresponding to the current usage scenario or historical user behavior data under the current usage scenario.

[0163] Based on the above embodiment, in this embodiment, the data transmission mode further includes: at least two data transmission modes and priorities of the at least two data transmission modes;

[0164] The second determining module 602 is specifically configured to:

[0165] Determining, according to the current usage scenario, at least two data transmission modes matching the current usage scenario and priorities of the at least two data transmission modes;

[0166] The at least two data transmission modes are switched according to the priority.

[0167] Based on the above embodiment, in this embodiment, the device further includes a display module, which is specifically configured to:

[0168] After receiving the target data transmitted by the target device according to the data transmission mode, the target data is parsed, virtual information is generated based on the user input text obtained by parsing, and the virtual information is displayed in the target area of ​​the target device.

[0169] Figure 7 This is the second structural diagram of the data collaborative input device provided by the present invention, such as Figure 7 As shown, the data collaborative input device provided by the present invention is applied to a target device, and the device includes:

[0170] A second receiving module 701 is configured to receive a data transmission request sent by an XR device, where the data transmission request includes a data transmission mode determined by the XR device according to a current usage scenario, where the data transmission mode includes a data transmission method and a data encryption method.

[0171] The transmission module 702 is configured to transmit target data to the XR device according to the data transmission mode.

[0172] The data collaborative input device provided by the present invention is applied to a target device and receives a data transmission request from an XR device. The data transmission request includes a data transmission mode determined by the XR device based on the current usage scenario, and the data transmission mode includes a data transmission method and a data encryption method. The target data is then transmitted to the XR device according to the data transmission mode. Therefore, the present invention determines a data transmission mode that matches the current usage scenario based on the current usage scenario, and combines this with a matching encryption level to enable efficient and secure data transmission between the XR device and the target mobile device, thereby enhancing the user experience.

[0173] Figure 8 An example of a physical structure diagram of an electronic device is shown below. Figure 8 As shown, the electronic device may be a robot or other electronic device, and may include: a processor 810, a communications interface 820, a memory 830, and a communication bus 840. The processor 810, the communications interface 820, and the memory 830 communicate with each other via the communication bus 840. The processor 810 may call logic instructions in the memory 830 to execute a data collaborative input method applied to an XR device, the method including:

[0174] Determine the current usage scenario;

[0175] Determine, based on the current usage scenario, a data transmission mode that matches the current usage scenario; wherein the data transmission mode includes: a data transmission mode and a data encryption mode;

[0176] Sending a data transmission request to a target device; wherein the data transmission request includes: the data transmission mode;

[0177] Receive target data transmitted by the target device according to the data transmission mode.

[0178] Furthermore, the logic instructions in the aforementioned memory 830 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product, stored in a storage medium, includes instructions for causing a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in at least one embodiment of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0179] In another aspect, the present invention further provides a computer program product, comprising a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can perform the data collaborative input method provided by the above methods, which is applied to an XR device. The method comprises:

[0180] Determine the current usage scenario;

[0181] Determine, based on the current usage scenario, a data transmission mode that matches the current usage scenario; wherein the data transmission mode includes: a data transmission mode and a data encryption mode;

[0182] Sending a data transmission request to a target device; wherein the data transmission request includes: the data transmission mode;

[0183] Receive target data transmitted by the target device according to the data transmission mode.

[0184] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the computer program is implemented to perform the data collaborative input method provided by the above methods, and is applied to an XR device. The method includes:

[0185] Determine the current usage scenario;

[0186] Determine, based on the current usage scenario, a data transmission mode that matches the current usage scenario; wherein the data transmission mode includes: a data transmission mode and a data encryption mode;

[0187] Sending a data transmission request to a target device; wherein the data transmission request includes: the data transmission mode;

[0188] Receive target data transmitted by the target device according to the data transmission mode.

[0189] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, i.e., they may be located in one location or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of the present embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0190] Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a necessary general-purpose hardware platform, or of course, hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (such as a personal computer, server, or network device) to execute the methods described in each embodiment or certain portions of the embodiments.

[0191] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.< / comment>

Claims

1. A collaborative data input method, characterized in that: Applied to an XR device, the method includes: Determine a current usage scenario; wherein the current usage scenario is a short text high-frequency input scenario or a long text high-frequency input scenario; Determine, based on the current usage scenario, a data transmission mode that matches the current usage scenario; wherein the data transmission mode includes: a data transmission mode and a data encryption mode; Sending a data transmission request to a target device; wherein the data transmission request includes the data transmission mode; Receiving target data transmitted by the target device according to the data transmission mode; and determining the current usage scenario includes: Detect the element type in the target area where the user is looking; When the time for which the user gazes at the target area is greater than a preset time threshold, determining the current usage scenario according to the element type of the target area; After receiving the target data transmitted by the target device according to the data transmission mode, the method further includes: The target data is parsed, virtual information is generated based on the user input text obtained by the parsing, and the virtual information is displayed in the target area of ​​the target device.

2. The collaborative data input method according to claim 1, wherein: The determining, according to the current usage scenario, a data transmission mode matching the current usage scenario includes: Determining a data encryption level for the current usage scenario based on the current usage scenario; Determine the corresponding data encryption method according to the data encryption level of the current usage scenario.

3. The collaborative data input method according to claim 2, wherein: The data transmission request also includes: keyboard type; After determining the current usage scenario, the method further includes: A keyboard type that matches the current usage scenario is determined based on a preset keyboard template corresponding to the current usage scenario or historical user behavior data under the current usage scenario.

4. The data collaborative input method according to claim 2, characterized in that: The data transmission mode includes: at least two data transmission modes and priorities of the at least two data transmission modes; The determining, according to the current usage scenario, a data transmission mode matching the current usage scenario includes: Determining, according to the current usage scenario, at least two data transmission modes matching the current usage scenario and priorities of the at least two data transmission modes; The at least two data transmission modes are switched according to the priority.

5. A collaborative data input method, characterized in that: Applied to a target device, the method includes: Receiving a data transmission request sent by an XR device, the data transmission request including a data transmission mode determined by the XR device based on a current usage scenario, the data transmission mode including a data transmission method and a data encryption method; wherein the current usage scenario is: detecting, by the XR device, an element type of a target area at which a user is gazing, and, if the user gazes at the target area for a period greater than a preset time threshold, determining, based on the element type of the target area, that the current usage scenario is a short text high-frequency input scenario or a long text high-frequency input scenario; The target data is transmitted to the XR device according to the data transmission mode, so that the XR device parses the target data, generates virtual information based on the parsed user input text, and displays the virtual information in the target area of ​​the target device.

6. A data collaborative input device, characterized in that: Applied to XR equipment, the apparatus includes: A first determination module is configured to determine a current usage scenario, wherein the current usage scenario is a short text high-frequency input scenario or a long text high-frequency input scenario; A second determining module is configured to determine a data transmission mode that matches the current usage scenario according to the current usage scenario; wherein the data transmission mode includes: a data transmission mode and a data encryption mode; A request module, configured to send a data transmission request to a target device; wherein the data transmission request includes: the data transmission mode; A receiving module, configured to receive target data transmitted by the target device according to the data transmission mode; The first determining module is specifically configured to: Detect the element type in the target area where the user is looking; When the time for which the user gazes at the target area is greater than a preset time threshold, determining the current usage scenario according to the element type of the target area; The device also includes a display module, specifically configured to: After receiving the target data transmitted by the target device according to the data transmission mode, the target data is parsed, virtual information is generated based on the user input text obtained by parsing, and the virtual information is displayed in the target area of ​​the target device.

7. A data collaborative input device, characterized in that: Applied to a target device, the apparatus comprises: a receiving module, configured to receive a data transmission request sent by an XR device, the data transmission request including a data transmission mode determined by the XR device based on a current usage scenario, the data transmission mode including a data transmission method and a data encryption method; wherein the current usage scenario is: detecting, by the XR device, the type of element in a target area at which a user is gazing, and, if the user gazes at the target area for a period greater than a preset time threshold, determining, based on the type of element in the target area, whether the current usage scenario is a high-frequency short text input scenario or a high-frequency long text input scenario; A transmission module is used to transmit target data to the XR device according to the data transmission mode, so that the XR device parses the target data, generates virtual information based on the user input text obtained by parsing, and displays the virtual information in the target area of ​​the target device.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the data collaborative input method according to any one of claims 1 to 5 is implemented.

Images