Notification message bidirectional interaction method and system based on intelligent wearable device

Abstract

The invention relates to the technical field of message interaction, and discloses a notification message bidirectional interaction method and system based on intelligent wearable equipment, and the method comprises the steps: monitoring a message notification sent by proxy middleware through an application tool after the application tool is installed at a mobile phone end, and carrying out the message processing of the message notification, so as to determine a response message of a wearable equipment end; transmitting the response message to the proxy middleware so as to forward the response message to the wearable device end through the proxy middleware; displaying the response message at the wearable device end to obtain a message display result, and receiving a message uploaded by a user at the wearable device end; the message receiving result is subjected to message packaging, the packaged message is returned to the proxy middleware, and the proxy middleware transmits the packaged message to the mobile phone side; and carrying out message response on the packaged message in the mobile phone terminal. According to the invention, the simplicity and the protection of sensitive information can be improved on the premise of ensuring the consistency of data transmission.

Description

Technical Field

[0001] The present invention relates to a method and system for two-way interaction of notification messages based on intelligent wearable devices, belonging to the technical field of message interaction. Background Art

[0002] Currently, the notification function of current intelligent wearable devices (such as smart watches) usually only supports one-way message reception. Users need to rely on the mobile phone voice assistant to complete the reply operation. In the prior art, message notifications are pushed to the watch through the Android Notification interface, but there is a lack of direct interaction ability. For example, after receiving a notification message, the watch user needs to wake up and open the mobile phone voice assistant to reply to the mobile phone message, resulting in cumbersome operations, poor real-time performance, and inconvenience. Existing devices only support one-way notification pushing and cannot achieve message context awareness and active response. Users need to frequently operate the mobile phone to complete the reply, which violates the "convenience" design concept of wearable devices. Continuous message synchronization significantly reduces the device battery life.

[0003] Currently, the data transmission methods include the Binder / AIDL method and the PendingIntent method. For the Binder / AIDL method, the Binder transaction supports atomic operations and can ensure the consistency of data transmission. However, the disadvantage is that a binding relationship needs to be established between the two parties in advance. For the PendingIntent method, there is no need to develop an additional cross-device communication layer for data transmission. In addition, if the watch and the mobile phone directly interact through the Binder / AIDL method or the PendingIntent method, the watch application will over-apply for sensitive permissions of the mobile phone, such as sending text messages and accessing locations. Therefore, the message interaction in the prior art lacks simplicity and protection of sensitive information while ensuring the consistency of data transmission. Summary of the Invention

[0004] The present invention provides a method and system for two-way interaction of notification messages based on intelligent wearable devices, and its main purpose is to improve simplicity and protection of sensitive information while ensuring the consistency of data transmission.

[0005] To achieve the above object, a method for two-way interaction of notification messages based on intelligent wearable devices provided by the present invention includes:

[0006] After installing an application tool on the mobile phone side, listen to the message notifications sent by the proxy middleware through the application tool, and process the message notifications to determine the response message on the wearable device side;

[0007] Transmit the response message to the proxy middleware, so as to forward the response message to the wearable device side through the proxy middleware;

[0008] On the wearable device side, display the response message to obtain a message display result. Based on the message display result, receive the user-uploaded message on the wearable device side to obtain a message reception result;

[0009] Perform message encapsulation on the message reception result to obtain an encapsulated message, and return the encapsulated message to the proxy middleware. The proxy middleware transmits the encapsulated message to the mobile phone side;

[0010] Perform a message response on the encapsulated message in the mobile phone side to complete the two-way message interaction between the mobile phone side and the wearable device side.

[0011] Optionally, the listening for the message notification sent by the proxy middleware through the application tool includes:

[0012] After enabling the permission for the proxy middleware to access notifications, retrieve the message to be verified through the information callback function in the application tool;

[0013] Transmit the message to be verified to the listening service in the application tool through the information transfer function in the application tool;

[0014] Judge whether the message to be verified comes from the proxy middleware through the listening service;

[0015] When the message to be verified comes from the proxy middleware, verify whether the message to be verified is a message notification through the information verification function in the application tool;

[0016] When the message to be verified is a message notification, complete the process of listening for the message notification sent by the proxy middleware through the application tool.

[0017] Optionally, the message processing of the message notification to determine the response message on the wearable device side includes:

[0018] Extract keywords from the message notification to obtain the extracted keywords;

[0019] Identify the chat scenario on the wearable device side according to the extracted keywords;

[0020] Query the chat intention and environmental status regarding the extracted keywords from the mobile phone side;

[0021] According to the chat scenario, the chat intention, and the environmental status, use the following formula to calculate the fitness score corresponding to the field display mode of the message notification:

[0022] R score =α(1 - N q) + β(1 - E b ) + γS a + δT + I u + ζ(1 - L)

[0023] where R score represents the fitness score, and α, β, γ, δ, ζ represent weight parameters, N q represents the network state, E b represents the device power, S a represents the system mode, T represents the time information, I u represents the user intention urgency determined based on historical click records, application switching frequencies, and voice commands, and L represents the positioning scenario complexity determined based on spatial information and status information;

[0024] According to the fitness score, use the field display method to perform message display on the message notification to obtain a display message, so as to complete the process of message processing on the message notification based on the display message;

[0025] Among them, the extracted keywords include time information, spatial information, and status information, the chat intention includes historical click records, application switching frequencies, and voice commands, and the environmental state includes network state, device power, and system mode.

[0026] Optionally, determining the response message of the wearable device side includes:

[0027] Determine the input message regarding the wearable device side;

[0028] Determine the protocol identifier of the input message;

[0029] Calculate the sender byte length corresponding to the input message and query the sender information corresponding to the input message;

[0030] Calculate the message byte length of the input message;

[0031] Concatenate the protocol identifier, the sender byte length, the byte array corresponding to the sender information, the message byte length, and the byte array corresponding to the input message to obtain a concatenated character;

[0032] Generate the check code of the concatenated character; concatenate the concatenated character and the check code to obtain the response message.

[0033] Optionally, forwarding the response message to the wearable device side through the proxy middleware includes:

[0034] Perform message compression on the response message to obtain a compressed message;

[0035] Fragment the compressed message to obtain fragmented messages;

[0036] Generate fragmented sequence numbers and session numbers for the fragmented messages;

[0037] Independently encrypt each fragmented message in the fragmented messages to obtain encrypted fragments;

[0038] Concatenate the fragmented sequence numbers, the session numbers, and the encrypted fragments to obtain concatenated fragments;

[0039] Detect the battery power of the mobile phone corresponding to the proxy middleware;

[0040] When the signal mode between the proxy middleware and the wearable device is a strong signal mode, based on the battery power of the mobile phone, set the first transmission power, high frequency band, and short connection interval of the concatenated fragments;

[0041] Use the first transmission power, the high frequency band, and the short connection interval to perform batch transmission of the fragmented concatenated fragments to complete the process of forwarding the response message to the wearable device;

[0042] When the signal mode between the proxy middleware and the wearable device is a weak signal mode, based on the battery power of the mobile phone, set the second transmission power, low frequency band, and long connection interval of the concatenated fragments;

[0043] Use the second transmission power, the low frequency band, and the long connection interval to perform slow transmission of the fragmented concatenated fragments to obtain slow transmission fragments;

[0044] Based on the number of times of segment movement of the wearable device, set the number of timeout retransmissions for the fragments lost in the middle during the slow transmission fragments;

[0045] According to the number of timeout retransmissions, perform timeout retransmission of the fragments lost in the middle to obtain timeout retransmission fragments;

[0046] Use the slow transmission fragments and the timeout retransmission fragments to complete the process of forwarding the response message to the wearable device.

[0047] Optionally, the receiving of the user-uploaded message by the wearable device based on the message display result to obtain a message receiving result includes:

[0048] Based on the message display result, collect the user voice message in the user-uploaded message through the microphone array in the wearable device;

[0049] Generate a quick text reply corresponding to the user-uploaded message according to the screen click trajectory in the user-uploaded message;

[0050] Take the text input messages in the user voice message, the quick text reply, and the user uploaded message as the message reception result.

[0051] Optionally, the message encapsulation of the message reception result to obtain an encapsulated message includes:

[0052] Divide the message reception result into a message content and a message operation type;

[0053] Encapsulate the message content and the message operation type into a bound message;

[0054] Perform a backhaul trigger operation on the bound message to obtain an encapsulated message.

[0055] Optionally, the returning the encapsulated message to the proxy middleware includes:

[0056] Perform cross-device communication binding between the wearable device side and the proxy middleware to obtain a cross-device communication binding result;

[0057] Based on the cross-device communication binding result, encapsulate the encapsulated message into a cross-device data packet;

[0058] Transmit the cross-device data packet to the proxy middleware to complete the process of returning the encapsulated message to the proxy middleware.

[0059] Optionally, the proxy middleware transmits the encapsulated message to the mobile phone side, including:

[0060] After enabling the permission to access notifications of the proxy middleware, create an additional notification for the encapsulated message in the proxy middleware;

[0061] Set a dedicated identifier for the additional notification regarding the proxy middleware;

[0062] Mark the notification source of the additional notification;

[0063] Initiate a message notification to the mobile phone side through the additional notification, the dedicated identifier, and the notification source to complete the process of the proxy middleware transmitting the encapsulated message to the mobile phone side.

[0064] To solve the above problems, the present invention also provides a notification message two-way interaction system based on a smart wearable device, and the system includes:

[0065] A message determination module, configured to, after installing an application tool on the mobile phone side, listen to the message notification sent by the proxy middleware through the application tool, perform message processing on the message notification to determine the response message of the wearable device side;

[0066] A message forwarding module, configured to transmit the response message to the proxy middleware, so as to forward the response message to the wearable device end through the proxy middleware;

[0067] A message receiving module, configured to perform message display on the response message at the wearable device end to obtain a message display result, and based on the message display result, receive a user upload message at the wearable device end to obtain a message receiving result;

[0068] A message transmission module, configured to perform message encapsulation on the message receiving result to obtain an encapsulated message, and return the encapsulated message to the proxy middleware, and the proxy middleware transmits the encapsulated message to the mobile phone end;

[0069] A message response module, configured to perform message response on the encapsulated message in the mobile phone end to complete the two-way message interaction between the mobile phone end and the wearable device end.

[0070] Compared with the problems described in the background art, in the embodiments of the present invention, the application tool listens for the message notifications sent by the proxy middleware. The main point is that compared with the prior art such as Binder / AIDL which requires prior service binding, the proxy middleware and the mobile application do not need to establish a direct inter-process communication (such as binding a Service) or a network connection (such as a long Socket connection). They only rely on the Android system notification channel to transfer data. Further, in the embodiments of the present invention, the message notifications are processed to dynamically extract key information from the message notifications based on the data on the mobile side and the data on the watch side and display it to the user on the mobile side, thereby improving the efficiency of message interaction between the watch and the mobile phone. Further, in the embodiments of the present invention, the response message of the wearable device is determined to receive the chat information input by the user on the mobile side, and the CMF protocol is used to encapsulate the chat data. Encapsulating the data can reduce the risk of code errors. In the embodiments of the present invention, the response message is transmitted to the proxy middleware to achieve the transmission of mobile-side information to the proxy middleware without establishing a direct inter-process communication between the proxy middleware and the mobile application, improving the simplicity of data transmission. In addition, when the proxy middleware triggers a mobile-side operation through a PendingIntent, the mobile side executes with the permissions of the proxy middleware instead of the watch directly operating, avoiding excessive application of sensitive permissions by the watch application. Further, in the embodiments of the present invention, the proxy middleware forwards the response message to the wearable device side to dynamically adjust the transmission strategy according to the changes in the mobile phone battery power and the watch positioning data, improving the stability and accuracy of the transmission. In the embodiments of the present invention, the message reception result is encapsulated by encapsulating the message reception result in the form of an Intent, which is convenient for the mobile side to operate after receiving the data encapsulated in the form of an Intent. This is because the data transmission relationship between the mobile side and the proxy middleware is a data transmission relationship without a pre-binding relationship, that is, a relationship transmitted through data in the form of an Intent. Further, in the embodiments of the present invention, the encapsulated message is returned to the proxy middleware to achieve the safe and efficient transfer of the watch-side Intent to the mobile-side proxy middleware by combining the inter-process communication capabilities of Binder / AIDL and the cross-device transmission protocol. Further, in the embodiments of the present invention, the proxy middleware transmits the encapsulated message to the mobile side to achieve the transmission of proxy middleware information to the mobile side without establishing a direct inter-process communication between the proxy middleware and the mobile application, thereby improving the simplicity of data transmission. Therefore, the notification message two-way interaction method and system based on intelligent wearable devices provided by the embodiments of the present invention can improve simplicity and the protection of sensitive information while ensuring the consistency of data transmission. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] Figure 1Schematic flowchart of a notification message two-way interaction method based on a smart wearable device provided by an embodiment of the present invention;

[0072] Figure 2 Schematic diagram of modules of a system for implementing the notification message two-way interaction based on a smart wearable device provided by an embodiment of the present invention.

[0073] The implementation, functional features, and advantages of the present invention will be further described in conjunction with embodiments with reference to the accompanying drawings. Detailed implementation manners

[0074] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0075] An embodiment of the present application provides a notification message two-way interaction method based on a smart wearable device. The execution subject of the notification message two-way interaction method based on a smart wearable device includes, but is not limited to, at least one of electronic devices such as a server, a terminal, etc. that can be configured to execute the method provided by the embodiment of the present application. In other words, the notification message two-way interaction method based on a smart wearable device can be executed by software or hardware installed on a terminal device or a server device. The server includes, but is not limited to: a single server, a server cluster, a cloud server, or a cloud server cluster, etc.

[0076] Embodiment 1:

[0077] Refer to Figure 1 As shown, it is a schematic flowchart of a notification message two-way interaction method based on a smart wearable device provided by an embodiment of the present invention. In this embodiment, the notification message two-way interaction method based on a smart wearable device includes:

[0078] S1. After installing an application tool on the mobile phone, listen for message notifications sent by the proxy middleware through the application tool, and process the message notifications to determine a response message from the wearable device side.

[0079] In the embodiments of the present invention, the mobile phone end refers to the Android mobile phone end, the application tool refers to a tool integrating multifunctional components, and the multifunctional components include an information callback function component, an information transmission function component, a monitoring service component, and an information verification function component. The information callback function component refers to the code with the onNotificationPosted() function. The information transmission function component refers to the code that can convert a message into a StatusBarNotification object and then transmit it to the monitoring service. The monitoring service component refers to the code with the StatusBarNotification.getPackageName() function. The information verification function component refers to the code that checks the extras field of a notification to confirm whether it is the target message to be processed. The onNotificationPosted() mainly realizes passive monitoring of external notifications through the observer mode, without the application tool actively polling external notifications. That is, when a new notification is sent by the proxy middleware, onNotificationPosted() is immediately triggered to ensure low latency of information processing. For example, the proxy middleware forwards a watch message → triggers a callback → the application tool synchronously receives the message notification. The StatusBarNotification object is mainly used to obtain the complete metadata of a notification. The complete metadata includes basic information such as the package name (getPackageName()), notification ID (getId()), and posting time (getPostTime()), content data such as the title, text, icon, and priority (parsing extras through the Notification object), and extended fields such as custom key-value pairs (such as the device ID and encryption identifier embedded by the proxy middleware). The

[0080] StatusBarNotification.getPackageName() is mainly used to obtain the package name (getPackageName()) in the complete metadata. In addition, GPS location information of the watch end and sensor data such as accelerometers are inserted into the extras.

[0081] Optionally, the process of installing the application tool on the mobile phone end means that when the multifunctional components corresponding to the application tool are not integrated in the Android mobile phone, the missing function components are added to the Android mobile phone. When the multifunctional components corresponding to the application tool are integrated in the Android mobile phone, there is no need to additionally add duplicate function components.

[0082] Further, in the embodiment of the present invention, the application tool listens for the message notifications sent by the proxy middleware. The main point is that compared with the prior art such as Binder / AIDL which requires prior service binding, the proxy middleware and the mobile application do not need to establish a direct inter-process communication (such as binding a Service) or a network connection (such as a long Socket connection), but only rely on the Android system notification channel to transfer data.

[0083] Among them, the proxy middleware refers to a tool configured in the mobile phone.

[0084] In an embodiment of the present invention, the process of listening for the message notifications sent by the proxy middleware through the application tool includes: after enabling the permission for the application tool to access notifications of the proxy middleware, invoking the message to be verified through the information callback function in the application tool; transferring the message to be verified to the listening service in the application tool through the information transfer function in the application tool; determining whether the message to be verified comes from the proxy middleware through the listening service; when the message to be verified comes from the proxy middleware, verifying whether the message to be verified is a message notification through the information verification function in the application tool; and when the message to be verified is a message notification, completing the process of listening for the message notifications sent by the proxy middleware through the application tool.

[0085] Among them, the permission for the application tool to access notifications refers to the permission for the proxy middleware to allow the application tool to access the message notifications published by the proxy middleware. The message to be verified refers to the system notifications published by the proxy middleware, which mainly refers to the data packets sent from the watch end. This data contains multiple metadata, and these metadata are the complete metadata obtained by the StatusBarNotification object.

[0086] Optionally, when integrating the proxy middleware into the Android mobile phone, the process of enabling the permission for the application tool to access notifications of the proxy middleware requires the proxy middleware to guide the user to manually enable the notification listening permission of the proxy middleware on the mobile phone end. For example, the user enters the mobile phone settings, selects the application, finds the proxy middleware, and enables the "Allow notification access" permission.

[0087] Optionally, the process of retrieving the message to be verified sent by the proxy middleware through the information callback function in the application tool means that the NotificationListenerService registered on the mobile device automatically triggers the onNotificationPosted() callback when the system notification is issued; the process of passing the message to be verified to the listening service in the application tool through the information transmission function in the application tool means that the system passes the notification sent by the proxy middleware to the listening service as a StatusBarNotification object; the process of determining whether the message to be verified comes from the proxy middleware through the listening service means that the mobile device identifies whether the notification source is the proxy middleware (such as com.proxy.middleware) through StatusBarNotification.getPackageName(); the process of verifying whether the message to be verified is a message notification through the information verification function in the application tool means that the extras field of the notification is checked to confirm whether it is the target message to be processed (such as isFromWatch = true).

[0088] Furthermore, in the embodiment of the present invention, message processing is performed on the message notification to dynamically extract key information in the message notification based on the data on the mobile device side and the data on the smartwatch side and display it to the user on the mobile device side, thereby improving the efficiency of message interaction between the smartwatch and the mobile device.

[0089] In one embodiment of the present invention, the message processing of the message notification to determine the response message on the wearable device side includes: extracting keywords from the message notification to obtain the extracted keywords; identifying the chat scenario on the wearable device side according to the extracted keywords; querying the chat intention and environmental status regarding the extracted keywords from the mobile device side; calculating the fitness score corresponding to the field display mode of the message notification according to the chat scenario, the chat intention, and the environmental status by using the following formula:

[0090] R score = α(1 - N q ) + β(1 - E b ) + γS a + δT + I u + ζ(1 - L)

[0091] where R score represents the fitness score, α, β, γ, δ, ζ represent weight parameters, N q represents the network status, E b represents the device battery power, S a represents the system mode, T represents the time information, I uIndicates the user intent urgency determined based on historical click records, application switching frequency, and voice commands, and L represents the positioning scenario complexity determined based on spatial information and status information;

[0092] According to the fitness score, use the field display method to display the message notification to obtain a displayed message, so as to complete the process of message processing for the message notification based on the displayed message; wherein, the extracted keywords include time information, spatial information, and status information, the chat intent includes historical click records, application switching frequency, and voice commands, and the environmental status includes network status, device battery power, and system mode.

[0093] Among them, the extracted keywords refer to the content data such as the release time, title, text, icon, priority (parsing extras through the Notification object), etc. included in the message notification. The chat scenario refers to the scenario where the watch is located determined by the release time, GPS positioning information in extras, and sensor data such as accelerometers. For example, when the GPS positioning information is the location of the gym and the release time is used to determine that the user is in the historical fitness period, and the accelerometer shows that the user is running, it is determined that the user wearing the watch is in the fitness scenario. The status information refers to sensor data such as accelerometers. The historical click record refers to the record of the user on the mobile phone side clicking on the notification message in the historical period. The application switching frequency refers to the number of application switches per unit time. The keyword and intonation analysis of the voice command voice content refers to that the network status refers to the state of the network, such as network type (WIFI, 5G network, etc.), network signal strength. The device battery power refers to the remaining battery power of the mobile phone. The system mode refers to modes such as do not disturb, driving, sleep, and flight. The field display method refers to the method of selecting target information from the message notification and displaying the target information in the mobile phone notification bar. The core lies in how to select target information from the message notification. The selectable algorithms include regular matching, semantic analysis enhancement, and shielding non-urgent notifications. Semantic analysis enhancement refers to the process of analyzing the semantics of the watch-side chat information included in the message notification to increase the accuracy of the semantics. For example, the original message (such as the text input by the user, the content of voice-to-text conversion) is initially cleaned, irrelevant characters (such as special symbols, repeated spaces) are removed, and key fragments (such as names, phone numbers, etc. entities) are extracted. The process of shielding non-urgent notifications refers to only displaying urgent messages in the notification bar. Regarding the symbol N q , 0 indicates WIFI, 1 indicates 5G network, S a and N q Similarly, it is also represented by data such as 0, 1, 2, 3, etc., I u The calculation formula is as follows:

[0094] I u= α′C e + γ′F s + δ′K v + ′V t

[0095] where C e represents the ratio of the number of clicks on emergency notifications to the total number of notifications received, F s represents the ratio of the number of switches to the switching time period, K v represents the emergency degree of semantic keywords (0 - 1, mapped by a predefined dictionary), and the semantics are from the words the user is speaking into the mobile phone at the current moment, which are not the chat messages input by the user into the mobile phone, V t represents the intonation emergency degree, and it is judged whether the user's intonation is urgent based on the variance of the energy peak value of the spectrogram. α′, γ′, δ′, ′ represent weights;

[0096] The calculation formula of L is as follows:

[0097] L = ω1(1 - g) + ω2A

[0098] where ω1, ω2 represent weights, g represents the positioning method, such as g is 1 for GNSS positioning, g is 0.5 for WIFI positioning, g is 0.2 for base station positioning, and A represents the exercise intensity, for example, the sum of the partial derivatives of the accelerometer value and the partial derivative of the angle sensor data.

[0099] Optionally, the process of performing message display on the message notification using the field display method according to the fitness score refers to obtaining the field display methods corresponding to different intervals when the fitness score belongs to different intervals, and performing message display on the message notification through the field display method. For example, there are a total of three intervals, which are the numerical intervals corresponding to regular matching, semantic analysis enhancement, and shielding non - emergency notifications respectively. If the fitness score belongs to the numerical interval corresponding to regular matching, then the field display method of regular matching is selected to perform message display on the message notification.

[0100] Furthermore, the embodiment of the present invention determines the response message at the wearable device end to receive the chat messages input by the user on the mobile phone side, and encapsulates the chat data using the CMF protocol. Encapsulating the data can reduce the risk of code errors.

[0101] Among them, the wearable device end refers to a smart watch, and the response message refers to the chat response made by the user to the watch end after viewing the message notification on the mobile phone.

[0102] In an embodiment of the present invention, determining the response message at the wearable device end includes: determining an input message regarding the wearable device end; determining a protocol identifier of the input message; calculating a sender byte length corresponding to the input message and querying sender information corresponding to the input message; calculating a message byte length of the input message; concatenating the protocol identifier, the sender byte length, a byte array corresponding to the sender information, the message byte length, and a byte array corresponding to the input message to obtain a concatenated character; generating a checksum of the concatenated character; and concatenating the concatenated character and the checksum to obtain a response message.

[0103] Among them, the response message is as follows:

[0104] [Header][Sender_Len][Sender][Content_Len][Content][CRC32], where Header represents the protocol identifier, Sender_Len represents the byte length of the sender information, Sender represents the sender information, Content_Len represents the message byte length of the input message, Content represents the input message, and CRC32 represents a checksum calculated based on [Header][Sender_Len][Sender][Content_Len][Content].

[0105] S2. Transmit the response message to the proxy middleware so as to forward the response message to the wearable device end through the proxy middleware.

[0106] In the embodiment of the present invention, by transmitting the response message to the proxy middleware, information on the mobile phone side can be transmitted to the proxy middleware without establishing direct inter-process communication between the proxy middleware and the mobile phone side application, improving the simplicity of data transmission. In addition, when the proxy middleware triggers a mobile phone side operation through PendingIntent, the mobile phone side executes with the permission of the proxy middleware instead of the watch directly operating, avoiding the watch application from overly applying for sensitive permissions.

[0107] Optionally, transmitting the response message to the proxy middleware includes: the mobile phone side encapsulates the message into the extras of an Intent, specifically injecting the reply content (text, binary data, etc.) into the extras of a preset PendingIntent, identifying the semantic type of the reply by modifying the action or adding an additional tag (such as REPLY_TYPE = QUICK), and sending an ordered broadcast. The proxy middleware registers a static broadcast receiver, and the static broadcast receiver receives the ordered broadcast.

[0108] Further, in the embodiment of the present invention, the proxy middleware is used to forward the response message to the wearable device side, so as to adapt to the changes in the mobile phone battery power and the watch positioning data to dynamically adjust the transmission strategy, and improve the stability and accuracy of the transmission.

[0109] In an embodiment of the present invention, the process of forwarding the response message to the wearable device side through the proxy middleware includes: compressing the response message to obtain a compressed message; fragmenting the compressed message to obtain fragmented messages; generating fragment sequence numbers and session numbers for the fragmented messages; independently encrypting each fragmented message in the fragmented messages to obtain encrypted fragments; concatenating the fragment sequence numbers, the session numbers and the encrypted fragments to obtain concatenated fragments; detecting the mobile phone battery power corresponding to the proxy middleware; when the signal mode between the proxy middleware and the wearable device side is a strong signal mode, setting a first transmission power, a high frequency band and a short connection interval for the concatenated fragments based on the mobile phone battery power; using the first transmission power, the high frequency band and the short connection interval to perform batch transmission of the fragments on the concatenated fragments to complete the process of forwarding the response message to the wearable device side; when the signal mode between the proxy middleware and the wearable device side is a weak signal mode, setting a second transmission power, a low frequency band and a long connection interval for the concatenated fragments based on the mobile phone battery power; using the second transmission power, the low frequency band and the long connection interval to perform slow transmission of the fragments on the concatenated fragments to obtain slow transmission fragments; setting the timeout retransmission times for the lost fragments in the middle of the slow transmission fragments based on the number of times of movement of the wearable device side during the period; performing timeout retransmission on the lost fragments in the middle according to the timeout retransmission times to obtain timeout retransmission fragments; using the slow transmission fragments and the timeout retransmission fragments to complete the process of forwarding the response message to the wearable device side.

[0110] Optionally, the process of message compression for the response message to obtain a compressed message refers to GZIP compression of message data (such as voice) to reduce traffic consumption; the process of message fragmentation for the compressed message to obtain fragmented messages refers to fragmenting the data using the BLE fragmentation transmission strategy (MTU = 247 bytes); the process of generating the fragment sequence number of the fragmented message and the session ID refers to using redundant fragment coding (such as Fountain Code), appending the dynamically generated fragment sequence number + session ID, allowing 30% of the fragments to be lost and still enabling complete data recombination; the process of independently encrypting each fragmented message in the fragmented message to obtain encrypted fragments refers to independently encrypting each fragment using AES-CCM; the process of setting the first transmission power, high-frequency band, and short connection interval of the spliced fragments based on the mobile phone battery when the signal between the proxy middleware and the wearable device is in a strong signal mode refers to actively reducing its own transmission power when the mobile phone battery < 20% to save power, using the 5GHz band, setting the connection interval to 7.5ms, and batch-transmitting the fragments to increase the throughput to 2Mbps; the process of setting the second transmission power, low-frequency band, and long connection interval of the spliced fragments based on the mobile phone battery when the signal between the proxy middleware and the wearable device is in a weak signal mode refers to actively reducing its own transmission power when the mobile phone battery < 20% to save power, switching to the 2.4GHz band, extending the connection interval to 4s, and enabling the fragment timeout retransmission mechanism (up to 3 times); the process of setting the timeout retransmission times of the lost fragments in the slow transmission fragments based on the number of times the wearable device moves during a period refers to combining the GPS positioning data sent from the watch end and automatically increasing the retransmission times when the user's movement speed is low (<1m / s), and using the stable environment to improve the success rate.

[0111] S3. Display the response message on the wearable device to obtain a message display result, and based on the message display result, receive the user upload message on the wearable device to obtain a message reception result.

[0112] Optionally, the process of displaying the response message on the wearable device to obtain a message display result is, for example: displaying the sender's nickname, message summary (truncated at 50 characters), and 3 quick reply buttons on the AMOLED screen. It should be noted that the quick reply buttons are quick replies pre-searched in the thesaurus and need to be implemented based on the dialogue model. For example, input the message sent from the mobile phone end into the dialogue model, output the corresponding reply text through the dialogue model, and generate these reply texts into quick reply buttons.

[0113] In an embodiment of the present invention, the message receiving of the user-uploaded message at the wearable device end based on the message display result to obtain a message receiving result includes: collecting the user voice message in the user-uploaded message through a microphone array in the wearable device end based on the message display result; generating a quick text reply corresponding to the user-uploaded message according to the screen click trajectory in the user-uploaded message; using the user voice message, the quick text reply, and the text input message in the user-uploaded message as the message receiving result.

[0114] Wherein, the text input message refers to the user's typing chat information, and the user-uploaded message refers to the operation data performed by the user on the watch, including data obtained by turning on the microphone for recording, data input by typing, and data input by clicking a shortcut key, etc.

[0115] Optionally, the process of collecting the user voice message in the user-uploaded message through the microphone array in the wearable device end refers to noise reduction of the received voice through a Beamforming microphone array; the process of generating a quick text reply corresponding to the user-uploaded message according to the screen click trajectory in the user-uploaded message refers to detecting the capacitance screen click trajectory to generate a quick reply (such as clicking the quick reply to send "OK", clicking the emoji to send "smile").

[0116] S4. Perform message encapsulation on the message receiving result to obtain an encapsulated message, and return the encapsulated message to the proxy middleware, and the proxy middleware transmits the encapsulated message to the mobile phone end.

[0117] In the embodiment of the present invention, the message receiving result is encapsulated in the form of Intent to facilitate the mobile phone end to perform operations after receiving the data encapsulated in the form of Intent, because the data transmission relationship between the mobile phone end and the proxy middleware is a data transmission relationship without a pre-bound relationship, that is, a relationship transmitted through data in the form of Intent.

[0118] In an embodiment of the present invention, the performing message encapsulation on the message receiving result to obtain an encapsulated message includes: dividing the message receiving result into message content and message operation type; encapsulating the message content and the message operation type into a bound message; performing a callback trigger operation on the bound message to obtain an encapsulated message.

[0119] Wherein, the message content refers to chat information, and the message operation type refers to data indicating the semantic type of the reply.

[0120] Optionally, the process of encapsulating the message content and the message operation type into a bound message refers to injecting the reply content (text, binary data, etc.) into the extras of the preset PendingIntent, and identifying the semantic type of the reply by modifying the action or adding an additional tag (such as REPLY_TYPE=QUICK). Further, the process of performing a callback trigger operation on the bound message to obtain the encapsulated message refers to the watch end calling the PendingIntent.send() method to trigger the Intent callback and obtain the data to be sent at the starting moment of the callback, which is the encapsulated message.

[0121] Further, in the embodiment of the present invention, by returning the encapsulated message to the proxy middleware, the secure and efficient transfer of the watch end Intent to the mobile phone end proxy middleware is realized by combining the inter-process communication capability of Binder / AIDL and the cross-device transmission protocol.

[0122] In one embodiment of the present invention, returning the encapsulated message to the proxy middleware includes: performing cross-device communication binding between the wearable device end and the proxy middleware to obtain a cross-device communication binding result; encapsulating the encapsulated message into a cross-device data packet based on the cross-device communication binding result; and transmitting the cross-device data packet to the proxy middleware to complete the process of returning the encapsulated message to the proxy middleware.

[0123] Optionally, the process of performing cross-device communication binding between the wearable device side and the proxy middleware to obtain a cross-device communication binding result refers to binding the wearable device side and the proxy middleware through a cross-device communication framework (such as Wearable Data Layer API); the process of encapsulating the encapsulated message into a cross-device data packet based on the cross-device communication binding result refers to encapsulating the Intent to be transmitted into a cross-device data packet, and the cross-device data packet format is Header|Session ID|Packet Index|AES-CCM Encrypted Payload|CRC32|, where Header represents a magic number identifier (such as 0xAA55AA55), Session ID represents a unique session identifier to prevent replay attacks, Packet Index represents a fragmentation index (fragmentation is required when MTU = 247), Payload represents the encrypted Binder / AIDL call data (including Bundle), and CRC32 represents a checksum for fragment integrity; the process of transmitting the cross-device data packet to the proxy middleware refers to performing data transmission between the watch and the proxy middleware through the inter-process communication capability of Binder / AIDL. AIDL is the Android Interface Definition Language, and Binder is the IPC framework of Android. Binder / AIDL is used for cross-process communication.

[0124] Further, in the embodiment of the present invention, the proxy middleware transmits the encapsulated message to the mobile phone side, so as to realize the transmission of the proxy middleware information to the mobile phone side without establishing a direct inter-process communication between the proxy middleware and the mobile phone side application, thereby improving the simplicity of data transmission.

[0125] In an embodiment of the present invention, the process of the proxy middleware transmitting the encapsulated message to the mobile phone side includes: after enabling the permission to access notifications of the proxy middleware, creating an additional notification of the encapsulated message in the proxy middleware; setting a dedicated identifier of the proxy middleware for the additional notification; marking the notification source of the additional notification; and initiating a message notification to the mobile phone side through the additional notification, the dedicated identifier, and the notification source, so as to complete the process of the proxy middleware transmitting the encapsulated message to the mobile phone side.

[0126] Among them, the additional notification refers to inserting a custom field into the extras of the notification to be released soon, such as is_from_proxy = true. is_from_proxy = true indicates that the source is the proxy middleware. It should be noted that the notification to be released soon refers to the notification composed of the encapsulated messages to be released by the proxy middleware. The dedicated identifier refers to the type (category) or tag of the notification. The category or tag is used as the dedicated identifier of the proxy middleware (such as proxy_message). The notification source refers to the notification source marked with a specific package name (packageName) (such as com.proxy.agent). com.proxy.agent represents the source of the proxy middleware.

[0127] S5. Perform a message response to the encapsulated message on the mobile device to complete the two-way message interaction between the mobile device and the wearable device.

[0128] Optionally, the process of performing a message response to the encapsulated message on the mobile device refers to the process of listening for the encapsulated message on the mobile device, presenting the message to the user, and receiving the message input by the user into the mobile device. The specific principle is similar to the above principle of listening for the message notification sent by the proxy middleware through the application tool and performing message processing on the message notification to determine the response message of the wearable device, and will not be elaborated further here.

[0129] Compared with the problems described in the background art, in the embodiments of the present invention, the application tool listens for the message notifications sent by the proxy middleware. The main difference is that compared with the prior art such as Binder / AIDL which requires pre-binding services, the proxy middleware and the mobile application do not need to establish a direct inter-process communication (such as binding a Service) or a network connection (such as a long Socket connection). Instead, it only relies on the Android system notification channel to transfer data. Further, in the embodiments of the present invention, the message notifications are processed to dynamically extract the key information in the message notifications based on the data on the mobile side and the data on the watch side and display it to the user on the mobile side, thereby improving the efficiency of message interaction between the watch and the mobile phone. Further, in the embodiments of the present invention, the response message of the wearable device is determined to receive the chat information input by the user on the mobile side, and the CMF protocol is used to encapsulate the chat data. Encapsulating the data can reduce the risk of code errors. In the embodiments of the present invention, the response message is transmitted to the proxy middleware to achieve the transmission of information from the mobile side to the proxy middleware without establishing a direct inter-process communication between the proxy middleware and the mobile application, improving the simplicity of data transmission. In addition, when the proxy middleware triggers a mobile operation through a PendingIntent, the mobile side executes with the permissions of the proxy middleware instead of the watch directly operating, avoiding the watch application from over-applying for sensitive permissions. Further, in the embodiments of the present invention, the proxy middleware forwards the response message to the wearable device side to dynamically adjust the transmission strategy according to the changes in the mobile phone battery and the watch positioning data, improving the stability and accuracy of the transmission. In the embodiments of the present invention, the message reception result is encapsulated by encapsulating the message reception result in the form of an Intent, which is convenient for the mobile side to operate after receiving the data encapsulated in the Intent form. This is because the relationship between the mobile side and the proxy middleware is a data transmission relationship without a pre-binding relationship, that is, a relationship transmitted through Intent-form data. Further, in the embodiments of the present invention, the encapsulated message is returned to the proxy middleware to achieve the safe and efficient transmission of the watch-side Intent to the mobile-side proxy middleware by combining the inter-process communication capabilities of Binder / AIDL and the cross-device transmission protocol. Further, in the embodiments of the present invention, the proxy middleware transmits the encapsulated message to the mobile side to achieve the transmission of the proxy middleware information to the mobile side without establishing a direct inter-process communication between the proxy middleware and the mobile application, thereby improving the simplicity of data transmission. Therefore, the notification message two-way interaction method and system based on the smart wearable device provided by the embodiments of the present invention can improve the simplicity and the protection of sensitive information while ensuring the consistency of data transmission.

[0130] Embodiment 2:

[0131] Such as Figure 2As shown, it is a functional module diagram of a notification message two-way interaction system based on a smart wearable device according to the present invention.

[0132] The notification message two-way interaction system 200 based on a smart wearable device according to the present invention can be installed in an electronic device. According to the functions implemented, the notification message two-way interaction system based on a smart wearable device may include a message determination module 201, a message forwarding module 202, a message receiving module 203, a message transmission module 204, and a message response module 205. The modules described in the present invention may also be referred to as units, which refer to a series of computer program segments that can be executed by a processor of an electronic device and can complete fixed functions, and are stored in the memory of the electronic device.

[0133] In the embodiments of the present invention, the functions of each module / unit are as follows:

[0134] The message determination module 201 is used to, after installing an application tool on the mobile phone side, listen to the message notifications sent by the proxy middleware through the application tool, process the message notifications to determine the response message on the wearable device side;

[0135] The message forwarding module 202 is used to transmit the response message to the proxy middleware, so as to forward the response message to the wearable device side through the proxy middleware;

[0136] The message receiving module 203 is used to display the response message on the wearable device side to obtain a message display result, and based on the message display result, receive the user upload message on the wearable device side to obtain a message receiving result;

[0137] The message transmission module 204 is used to encapsulate the message receiving result to obtain an encapsulated message, return the encapsulated message to the proxy middleware, and the proxy middleware transmits the encapsulated message to the mobile phone side;

[0138] The message response module 205 is used to perform a message response on the encapsulated message on the mobile phone side to complete the two-way message interaction between the mobile phone side and the wearable device side.

[0139] Specifically, each module in the notification message two-way interaction system 200 based on a smart wearable device in the embodiments of the present invention uses the same technical means as the Figure 1 notification message two-way interaction method based on a smart wearable device described above and can produce the same technical effects, which will not be elaborated here.

[0140] It is obvious to those skilled in the art that the present invention is not limited to the details of the above-described exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0141] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A notification message two-way interaction method based on a smart wearable device, characterized in that: The method comprises: After the application tool is installed on the mobile phone, the message notification sent by the proxy middleware is monitored by the application tool, and the message notification is processed to determine the response message of the wearable device; Transmitting the response message to the proxy middleware, so that the response message is forwarded to the wearable device end through the proxy middleware; Displaying the response message on the wearable device to obtain a message display result, and receiving the message uploaded by the user on the wearable device based on the message display result to obtain a message reception result; Encapsulating the message receiving result to obtain an encapsulated message, returning the encapsulated message to the proxy middleware, and the proxy middleware transmits the encapsulated message to the mobile phone end; The mobile phone terminal responds to the encapsulated message to complete the two-way message interaction between the mobile phone terminal and the wearable device terminal.

2. The notification message two-way interaction method based on a smart wearable device according to claim 1, characterized in that: The monitoring of the message notification sent by the proxy middleware by the application tool includes: After enabling the notification access permission of the proxy middleware, the message to be checked is retrieved through the information callback function in the application tool; Passing the message to be checked to a monitoring service in the application tool through an information transmission function in the application tool; Determine whether the message to be checked originates from the proxy middleware through the monitoring service; When the message to be checked originates from the proxy middleware, checking whether the message to be checked is a message notification by using an information checking function in the application tool; When the message to be checked is a message notification, the process of monitoring the message notification sent by the proxy middleware through the application tool is completed.

3. The notification message two-way interaction method based on a smart wearable device according to claim 1, characterized in that: The performing message processing on the message notification to determine a response message from the wearable device includes: Extracting keywords from the message notification to obtain extracted keywords; According to the extracted keywords, identifying the chat scene on the wearable device side; Querying the chat intention and environment status about the extracted keywords from the mobile phone terminal; According to the chat scenario, the chat intention and the environmental state, the fitness score corresponding to the field display mode of the message notification is calculated using the following formula: R score =α(1-N q )+β(1-E b )+γS a +δT+I u +ζ(1-L) Among them, R score represents the fitness score, α, β, γ, δ, ζ represent weight parameters, and N q Indicates the network status, E b Indicates the power of the device, S a Indicates system mode, T indicates time information, I u represents the urgency of user intention based on historical click records, application switching frequency, and voice commands, and L represents the complexity of the positioning scenario based on spatial information and state information; According to the fitness score, the message notification is displayed using the field display mode to obtain a display message, so as to complete the process of performing message processing on the message notification based on the display message; Among them, the extracted keywords include time information, space information and status information, the chat intention includes historical click records, application switching frequency and voice commands, and the environmental status includes network status, device power and system mode.

4. The notification message two-way interaction method based on a smart wearable device according to claim 1, characterized in that: The step of determining a response message from the wearable device includes: Determining an input message about the wearable device; determining a protocol identifier of the input message; Calculate the sender byte length corresponding to the input message and query the sender information corresponding to the input message; Calculate the message byte length of the input message; Concatenate the protocol identifier, the sender byte length, the byte array corresponding to the sender information, the message byte length and the byte array corresponding to the input message to obtain a concatenated character; Generate a check code of the spliced ​​characters; splice the spliced ​​characters and the check code to obtain a response message.

5. The notification message two-way interaction method based on a smart wearable device according to claim 1, characterized in that: The step of forwarding the response message to the wearable device through the proxy middleware includes: Compressing the response message to obtain a compressed message; Segmenting the compressed message to obtain segmented messages; Generate a fragment sequence number and a session number of the fragment message; Each fragment message in the fragment message is independently encrypted to obtain an encrypted fragment; Concatenate the fragment sequence number, the session number and the encrypted fragment to obtain a concatenated fragment; Detecting the battery level of the mobile phone corresponding to the proxy middleware; When the signal between the proxy middleware and the wearable device is in a strong signal mode, the first transmission power, high frequency band and short connection interval of the spliced ​​slice are set based on the power of the mobile phone; The spliced ​​fragments are batch-transmitted using the first transmission power, the high frequency band, and the short connection interval to complete the process of forwarding the response message to the wearable device end; When the signal between the proxy middleware and the wearable device is in weak signal mode, the second transmission power, low frequency band and long connection interval of the spliced ​​slice are set based on the power of the mobile phone; Performing slow transmission of the spliced ​​fragments by using the second transmission power, the low frequency band and the long connection interval to obtain slow transmission fragments; Based on the number of time period shifts on the wearable device, the number of timeout retransmissions of lost segments in the slow transmission segments is set; According to the timeout retransmission number, retransmit the lost fragments to obtain timeout retransmission fragments; The slow transmission segment and the timeout retransmission segment are used to complete the process of forwarding the response message to the wearable device.

6. The notification message two-way interaction method based on a smart wearable device according to claim 1, characterized in that: The receiving of the message uploaded by the user of the wearable device based on the message display result to obtain the message receiving result includes: Based on the message display result, collecting the user voice message in the message uploaded by the user through the microphone array in the wearable device end; Generating a quick text reply corresponding to the user uploaded message according to the screen click track in the user uploaded message; The user voice message, the quick text reply and the text input message in the user uploaded message are taken as the message reception result.

7. The notification message two-way interaction method based on a smart wearable device according to claim 1, characterized in that: The step of encapsulating the message receiving result to obtain an encapsulated message includes: Dividing the message reception result into message content and message operation type; Encapsulating the message content and the message operation type into a binding message; A return trigger operation is performed on the binding message to obtain an encapsulated message.

8. The notification message two-way interaction method based on a smart wearable device according to claim 1, characterized in that: The step of returning the encapsulated message to the proxy middleware comprises: Performing cross-device communication binding between the wearable device and the proxy middleware to obtain a cross-device communication binding result; Based on the cross-device communication binding result, encapsulating the encapsulated message into a cross-device data packet; The cross-device data packet is transmitted to the proxy middleware to complete the process of returning the encapsulated message to the proxy middleware.

9. The notification message two-way interaction method based on a smart wearable device according to claim 1, characterized in that: The transmitting of the encapsulated message to the mobile phone by the proxy middleware includes: After enabling the notification access permission of the proxy middleware, creating an additional notification of the encapsulated message in the proxy middleware; Setting the additional notification about the dedicated identifier of the proxy middleware; marking the notification source of the additional notification; A message notification is initiated to the mobile phone terminal through the additional notification, the dedicated identifier and the notification source, so as to complete the process of transmitting the encapsulated message to the mobile phone terminal by the proxy middleware.

10. A notification message two-way interactive system based on smart wearable devices, characterized in that: The system comprises: A message determination module is used to monitor the message notification sent by the proxy middleware through the application tool after the application tool is installed on the mobile phone, and perform message processing on the message notification to determine the response message of the wearable device; A message forwarding module, used for transmitting the response message to the proxy middleware, so as to forward the response message to the wearable device end through the proxy middleware; A message receiving module, used to display the response message on the wearable device end to obtain a message display result, and based on the message display result, receive the message uploaded by the user on the wearable device end to obtain a message reception result; A message transmission module, used for performing message encapsulation on the message receiving result to obtain an encapsulated message, returning the encapsulated message to the proxy middleware, and the proxy middleware transmits the encapsulated message to the mobile phone end; The message response module is used to respond to the encapsulated message in the mobile phone end to complete the two-way message interaction between the mobile phone end and the wearable device end.

Images