Screen lighting-up correction method for wearable device, device, and storage medium

Abstract

Embodiments of the present application provide a screen lighting-up correction method for a wearable device, a device, and a storage medium. The method comprises: collecting first acceleration data generated by a wearable device worn by a user along with a first action of the user; determining the category of the first action on the basis of the first acceleration data and an action recognition model, wherein the category of the first action is one of a set action, a non-set action, and a suspected set action; if the category of the first action is a suspected set action, detecting whether the user has triggered an operation of lighting up a screen of the wearable device within a set time; if it is detected that the user has triggered the operation of lighting up the screen of the wearable device within the set time, controlling the wearable device to light up the screen; and adjusting a model weight of the action recognition model on the basis of the first acceleration data. According to the present application, the learning of the action that a user lifts the wrist but has not triggered screen lighting-up is realized, so that the subsequent determination of the wrist lifting action is more accurate and rapid. The method has low time delay, and the user experience is good.

Description

Screen lighting correction method, device and storage medium for wearable device

[0001] Cross-references

[0002] This application refers to Chinese Patent Application No. 2024100561546, filed on January 15, 2024, entitled “Screen lighting correction method, device and storage medium for wearable device”, which is incorporated into this application by reference. Technical Field

[0003] The present application relates to the field of smart wearable technology, and in particular to a screen lighting correction method, device, and storage medium for a wearable device. Background Art

[0004] With the development of technology, smartwatches have become widely used in people's lives. They have multiple functions, such as displaying time, counting steps, and connecting to mobile phones to receive text messages or phone calls. In practice, to conserve power and ensure the battery life of smartwatches, they are generally kept in a dark state when not in use. When the user needs to check the current time or step count on the smartwatch screen, they can raise their wrist to automatically light up the screen and view the information displayed on the smartwatch screen.

[0005] However, in the existing technology, whether the user has raised their wrist is generally judged only by whether the current acceleration value of the smart watch or the acceleration change in a short period of time exceeds the set threshold. This method of judging by comparing a single variable has low accuracy and large delay. It is often the case that the smart watch fails to trigger the screen to light up due to misjudgment, resulting in a poor user experience. Summary of the Invention

[0006] Multiple aspects of the present application provide a screen lighting correction method, device and storage medium for a wearable device, which realizes the learning of the user's wrist-raising action without triggering the screen to light up, making the subsequent judgment of the wrist-raising action more accurate and fast, with low latency, and better user experience.

[0007] In a first aspect, an embodiment of the present application provides a method for calibrating screen lighting of a wearable device, the method comprising:

[0008] Collecting first acceleration data generated by a wearable device worn by a user in response to a first action of the user;

[0009] Determining a category of the first action based on the first acceleration data and a motion recognition model, wherein the category of the first action is one of the following: a set action, a non-set action, or a suspected set action;

[0010] If the category of the first action is a suspected set action, detecting whether the user triggers an operation of lighting up the screen of the wearable device within a set time;

[0011] If it is detected that the user triggers the operation of lighting up the screen of the wearable device within the set time, controlling the wearable device to light up the screen;

[0012] Based on the first acceleration data, a model weight of the action recognition model is adjusted.

[0013] In a second aspect, an embodiment of the present application provides a screen lighting correction device for a wearable device, the device comprising:

[0014] The acquisition module is used to acquire first acceleration data generated by the wearable device worn by the user in response to the user's first action.

[0015] A determination module is configured to determine a category of the first action based on the first acceleration data and an action recognition model, wherein the category of the first action is one of the following: a set action, a non-set action, or a suspected set action.

[0016] The detection module is used to detect whether the user triggers the operation of lighting up the screen of the wearable device within a set time if the category of the first action is a suspected set action.

[0017] The control module is used to control the wearable device to light up the screen if it is detected that the user has triggered an operation to light up the screen of the wearable device within a set time.

[0018] An adjustment module is used to adjust the model weight of the action recognition model based on the first acceleration data.

[0019] In a third aspect, an embodiment of the present application also provides a wearable device, which includes a memory for storing computer program instructions and a processor for executing program instructions, wherein when the computer program instructions are executed by the processor, the terminal device is triggered to execute the screen lighting correction method of the above-mentioned wearable device.

[0020] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, which includes a stored program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute the screen lighting correction method of the above-mentioned wearable device.

[0021] In the solution provided in the embodiment of the present application, the first acceleration data generated by the wearable device worn by the user in response to the user's first action is collected, and the category of the first action can be determined based on the first acceleration data and the action recognition model. If the category of the first action is determined to be a suspected set action, it is detected whether the user triggers the operation of lighting up the screen of the wearable device within the set time, and it is determined whether the user participates in the operation. If it is detected that the user triggers the operation of lighting up the screen of the wearable device within the set time, it can be considered that the first action is a set action. At this time, the wearable device is controlled to light up the screen, and the model weight of the action recognition model is adjusted based on the first acceleration data. In short, the present application realizes the learning of the action of the user raising the wrist but not triggering the screen to light up, so that the subsequent judgment of the wrist raising action is more accurate, fast, with low latency, and a better user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0023] FIG1 is a flow chart of a screen lighting correction method for a wearable device provided in an embodiment of the present application;

[0024] FIG2 is a schematic diagram of the structure of a smart watch provided in an embodiment of the present application;

[0025] FIG3 is a specific example diagram of a screen lighting correction method for a wearable device provided in an embodiment of the present application;

[0026] FIG4 is another specific example diagram of a screen lighting correction method for a wearable device provided in an embodiment of the present application;

[0027] FIG5a is a specific example diagram of the action recognition model provided in an embodiment of the present application;

[0028] FIG5 b is a specific example diagram of the first adjusted action recognition model provided in an embodiment of the present application;

[0029] FIG6 is a diagram illustrating an example of weight adjustment of an action recognition model provided in an embodiment of the present application;

[0030] FIG7 is a schematic structural diagram of a screen lighting correction device for a wearable device provided in an embodiment of the present application;

[0031] FIG8 is a schematic structural diagram of a wearable device provided in an embodiment of the present application;

[0032] FIG9 is a schematic diagram of the software structure of a wearable device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0033] The technical solutions in the embodiments of the present application will be described below in conjunction with the accompanying drawings in the embodiments of the present application. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in this article is merely a description of the association relationship of associated objects, indicating that three relationships can exist, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0034] In the following, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the quantity of the technical features indicated. Therefore, a feature specified as "first," "second," or "third" may explicitly or implicitly include one or more of the features.

[0035] With the development of science and technology, smart watches have been widely used in people's lives. In actual applications, in order to save power and ensure the battery life of smart watches, they are generally kept in a screen-off state when not in use. When the user needs to check the current time, step count, and other information on the smart watch screen, they can automatically light up the screen by raising their wrist to view the information displayed on the smart watch screen. However, in the existing technology, whether the user has raised their wrist is generally determined only by whether the current acceleration value of the smart watch or the acceleration change in a short period of time exceeds a set threshold. This method of judging by comparing a single variable has low accuracy and large delays. It is often the case that the smart watch fails to trigger the screen to light up due to misjudgment, resulting in a poor user experience.

[0036] In response to the above problems, the embodiments of the present application provide a solution. The basic idea is: adding a trained neural network model (i.e., the action recognition model described below) to the smart watch end, using the action recognition model to identify the user's actions, determining that the actions may be wrist-raising actions, and combining the user's active screen-lighting operation and the modification of the model weights of the action recognition model to achieve learning of the user's wrist-raising actions that do not trigger the screen-lighting, so that subsequent judgments on wrist-raising actions are more accurate, faster, with lower latency, and better user experience.

[0037] FIG1 is a flowchart of a screen lighting correction method for a wearable device provided in an embodiment of the present application. The method may be executed by a wearable device and specifically includes the following steps:

[0038] 101. Collect first acceleration data generated by a wearable device worn by a user in response to a first action of the user.

[0039] Among them, the wearable device can be a smart watch, a smart bracelet, and other smart wearable devices with a screen light function, which are not listed here one by one.

[0040] 102. Determine a category of the first action based on the first acceleration data and the action recognition model, wherein the category of the first action is one of the following: a set action, a non-set action, or a suspected set action.

[0041] 103. If the category of the first action is a suspected set action, detect whether the user triggers an operation of lighting up the screen of the wearable device within a set time.

[0042] 104. If it is detected that the user triggers an operation of lighting up the screen of the wearable device within the set time, the wearable device is controlled to light up the screen.

[0043] 105. Adjust a model weight of the motion recognition model based on the first acceleration data.

[0044] In practical applications, a neural network model is pre-trained on a server based on pre-collected training samples and corresponding labels to generate an action recognition model. The training samples can be acceleration data generated by a user performing multiple actions while wearing a wearable device, and the labels can be action labels corresponding to the acceleration data, which reflect the type of action.

[0045] For ease of understanding, the following describes the solution of this application in detail using a smartwatch as an example:

[0046] As shown in Figure 2, the smartwatch includes: a system-on-chip (SoC) 20, an accelerometer 21 electrically connected to the SoC 20, a screen 22, and buttons 23. The accelerometer 21 can be a three-axis accelerometer, and the motion recognition model is deployed on the SoC 20. In specific implementation, when a user wears the wearable device and performs a first action, the accelerometer 21 will collect the first acceleration data generated by the wearable device in response to the user's first action, and send the first acceleration data to the motion recognition model deployed on the SoC 20. Based on the output of the motion recognition model, the category of the first action is determined. The specific category determination method is as follows:

[0047] The first acceleration data is input into the action recognition model to obtain a first action prediction value output by the action recognition model.

[0048] If the first action prediction value is greater than the first set threshold, the category of the first action is determined to be: a set action.

[0049] If the first action prediction value is between the first set threshold and the second set threshold, the category of the first action is determined to be: suspected set action, wherein the second set threshold is less than the first set threshold.

[0050] If the first action prediction value is less than the second set threshold, the category of the first action is determined to be: non-set action.

[0051] In a specific implementation, for example, assuming that the action recognition model is a 5-layer fully connected neural network, specifically including: 1 input layer, 3 hidden layers (each hidden layer has 396 nodes) and 1 output layer (1 node), the sampling frequency of the accelerometer 21 is 100Hz, and the acceleration data within 1.5s is collected using the accelerometer 21. Based on the sampling results of the accelerometer 21, it can be determined that the input of the input layer is 450 nodes. The first action prediction value output by one node of the output layer is between -1 and 1, and the first set threshold can be 0.2, and the second set threshold can be -0.2.

[0052] Afterwards, based on the first action prediction value of the action recognition model and the first set threshold and the second set threshold, the category of the first action is determined: if the first action prediction value output by the action recognition model is 0.8, which is greater than the first set threshold 0.2, the category of the first action is determined to be: a set action. If the first action prediction value output by the action recognition model is 0.1, which is between the first set threshold 0.2 and the second set threshold -0.2, the category of the first action is determined to be: a suspected set action. If the first action prediction value output by the action recognition model is -0.8, which is less than the second set threshold -0.2, the category of the first action is determined to be: a non-set action. By setting the first set threshold and the second set threshold, and determining the category of the first action by comparing the first action prediction value with the first set threshold and the second set threshold, it can be ensured that the category of the first action is determined more quickly and accurately.

[0053] Following the above, after determining the category of the first action, if the category of the first action is: a setting action (ie, a wrist raising action), the system-level chip 20 directly sends a control signal to the screen 22 to light up the screen.

[0054] If the category of the first action is: non-set action (ie, not a wrist-raising action), it is ignored and no processing is performed.

[0055] If the first action is classified as a suspected set action (i.e., possibly a wrist-raising action), the system detects whether the user triggers the operation of lighting up the screen of the wearable device within a set time (e.g., 3s, 4s, etc., which can be determined according to actual conditions and is not limited here). If the user triggers the operation of lighting up the screen of the wearable device within the set time, the system-level chip 20 can have two options:

[0056] The first method is to assume that the user has determined that the suspected set action is a wrist-raising action. Subsequently, the model weight of the action recognition model can be adjusted based on the current acceleration data. For details, please refer to the flowchart shown in FIG3 .

[0057] The second method involves sending a prompt to the user and waiting for confirmation. Only after receiving the confirmation does the weights of the motion recognition model adjust based on the current acceleration data. The specific process is as follows: A prompt confirmation message is sent to confirm whether the first action is the set action; In response to the user confirming the prompt confirmation message, the weights of the motion recognition model are adjusted based on the first acceleration data. For details, see the flowchart in Figure 4.

[0058] In actual application, after the user triggers the operation of lighting up the screen of the wearable device, a prompt message will pop up on the screen 22, asking the user whether to record the current first action as the set action. At this time, there may be selection controls on the screen, such as "Yes" and "No". If the user clicks "Yes", it is equivalent to confirming the prompt message. At this time, based on the first acceleration data, the model weight of the action recognition model can be adjusted. Among them, the form of the "prompt message" is not limited here. It can be text in the form of a pop-up window or a voice prompt. It is not limited here. The user's confirmation operation for the "prompt message" is not limited to the above-mentioned "Yes" and "No" screen control form. It can also be confirmed by inputting a signal through the button 23 on the smart watch to confirm the prompt message. It can also be voice confirmation, or replaced with other confirmation words, such as "Confirm" and "Cancel", which are not listed here one by one.

[0059] Based on the above, the screen lighting correction method for a wearable device provided in an embodiment of the present application collects the first acceleration data generated by the wearable device worn by the user as the user performs the first action, and based on the first acceleration data and the action recognition model, the category of the first action can be determined. If it is determined that the category of the first action is a suspected set action, it is detected whether the user triggers the operation of lighting up the screen of the wearable device within the set time, and it is determined whether the user participates in the operation. If it is detected that the user triggers the operation of lighting up the screen of the wearable device within the set time, it can be considered that the first action is a set action. At this time, the wearable device is controlled to light up the screen, and the model weight of the action recognition model is adjusted based on the first acceleration data. In short, the present application realizes the learning of the action of the user raising the wrist but not triggering the screen to light up, so that the subsequent judgment of the wrist raising action is more accurate, fast, with low latency, and a better user experience.

[0060] The following describes the specific process of adjusting the model weights of the action recognition model:

[0061] As an implementation method, adjusting the model weight of the action recognition model based on the first acceleration data includes:

[0062] A first output node is added to the output layer of the motion recognition model, and the first output node is connected to each node in the previous network layer, and a set weighting coefficient is set on each connecting edge; based on the input of the first acceleration data, the first eigenvalue output by each node in the previous network layer is determined; according to the set weighting coefficient and the first eigenvalue, the model weight corresponding to the connecting edge between the first output node and each node in the previous network layer is determined to obtain a first adjusted motion recognition model.

[0063] For ease of understanding, the following examples are provided with reference to FIG5a and FIG5b:

[0064] Figure 5a shows the nodes corresponding to multiple "hidden layers" and one "output layer" in the action recognition model, where X1, X2, X3...X n represents the eigenvalue corresponding to each node in the last layer "hidden layer" (that is, the first eigenvalue mentioned above), and Y1 represents the first action prediction value output by the original output node corresponding to the "output layer".

[0065] In actual applications, after the first acceleration data corresponding to the collected first action is input into the action recognition model shown in Figure 5a, the action recognition model will output Y1. At this point, Y1 can be judged based on the first and second set thresholds. If Y1 is greater than the first set threshold, the user's first action is determined to be a set action, and the action recognition model is not adjusted. If Y1 is less than the second set threshold, the user's first action is determined to be a non-set action, and the action recognition model is not adjusted.

[0066] If Y1 is between the first set threshold and the second set threshold, the type of the user's first action is determined to be a suspected set action. At this point, a first output node is added to the output layer of the action recognition model. The first output node is connected to each node in the previous network layer, and each connecting edge is assigned a set weighting coefficient k. This set weighting coefficient k can be pre-set based on actual conditions and is not specifically limited here. As shown in Figure 5b, the action prediction value corresponding to this first output node is represented as Y2.

[0067] In a specific implementation, based on the set weight coefficient and the first eigenvalue, the model weight corresponding to the connection edge between the first output node and each node in the previous network layer can be determined. Specifically, for example, the weight corresponding to the connection edge between node X1 and the first output node is W1=X1×k, the weight corresponding to the connection edge between node X2 and the first output node is W2=X2×k... nThe weight W corresponding to the edge connecting the first output node n =Xn×k, and Y2=W1+W2...+W n It should be understood that when determining the model weights W1, W2, ..., W corresponding to the connection edges between the first output node and each node in the previous network layer, n , it means that the model weights of the action recognition model are adjusted, and what is obtained at this time is the first adjusted action recognition model, and its structure can be seen in Figure 5b.

[0068] Furthermore, if the user subsequently performs a second action, a third action, etc., the action recognition model will also adjust the model weight in real time based on the collected acceleration data. The following is an example of the user performing a second action to further illustrate the solution of this application. The step of adjusting the model weight of the action recognition model also includes:

[0069] collecting second acceleration data generated by the wearable device in response to a second action of the user;

[0070] Determining a category of the second action based on the second acceleration data and the first adjusted action recognition model. Specifically, based on the input of the second acceleration data, obtaining a second action prediction value output by an original output node in an output layer of the first adjusted action recognition model and a third action prediction value output by the first output node, and determining the category of the second action based on a maximum value between the second action prediction value and the third action prediction value;

[0071] If the second action is a suspected set action and it is detected that the user triggers the operation of lighting up the screen of the wearable device within the set time, the wearable device is controlled to light up the screen;

[0072] Creating a second output node in the output layer of the first adjusted action recognition model, wherein the second output node is connected to each node in the previous network layer, and a set weight coefficient is set on each connection edge;

[0073] Determine a second eigenvalue output by each node in the previous network layer based on the input of the second acceleration data;

[0074] According to the set weighting coefficient and the second eigenvalue, the model weights corresponding to the connecting edges between the second output node and each node in the previous network layer are determined to obtain a second adjusted action recognition model.

[0075] During specific implementation, the second acceleration data is input into the first adjusted action recognition model to obtain the second action prediction value (Y1 in Figure 5b) and the third action prediction value (Y2 in Figure 5b) output by the first adjusted action recognition model, and the maximum value between Y1 and Y2 is determined. Assuming that Y1 is 0.5 and Y2 is 0.8, the category of the second action is determined based on the larger Y2. Specifically, if Y2 is greater than the first set threshold, the type of the user's second action is determined to be: a set action. At this time, the first adjusted action recognition model is not adjusted. If Y2 is less than the second set threshold, the type of the user's second action is determined to be: a non-set action. At this time, the first adjusted action recognition model is not adjusted.

[0076] If Y2 is between the first set threshold and the second set threshold, the type of the user's second action is determined to be: suspected set action. At this time, a second output node is added to the output layer of the first adjusted action recognition model. The second output node is connected to each node in the previous network layer, and each connecting edge is set with a set weighting coefficient k. The set weighting coefficient k can be pre-set according to actual conditions and is not specifically limited here. The action prediction value corresponding to the second output node is represented as Y3 (not shown in Figure 5b).

[0077] Based on the input of the second acceleration data, the second eigenvalue output by each node in the previous network layer is determined. Based on the set weighting coefficient and the second eigenvalue, the model weight corresponding to the connecting edge between the second output node and each node in the previous network layer can be determined, thereby obtaining a second adjusted action recognition model. The method for determining the model weight can be referred to the above example and will not be repeated here. It should be understood that the second adjusted action recognition model is only a specific example. The model weight of the action recognition model can be further adjusted based on the user's third action, fourth action, etc. to generate a third adjusted action recognition model and a fourth adjusted action recognition model. These are not listed here one by one.

[0078] Based on the above, in the embodiment of the present application, as the user continues to move, output nodes can be continuously added to the output layer of the action recognition model, and the model weights corresponding to the connection edges between the newly added output nodes and the nodes in the previous network layer can be updated, thereby realizing real-time adjustment of the action recognition model, making subsequent judgments on set actions more accurate, fast, with lower latency, and better user experience.

[0079] As another implementation, adjusting the model weight of the action recognition model based on the first acceleration data includes:

[0080] A sample set is obtained, wherein the sample set includes characteristic values ​​of acceleration data of multiple actions, the characteristic values ​​correspond to each node in a network layer before an output layer in a motion recognition model, the acceleration data of the multiple actions include first acceleration data, and the action label value corresponding to the first acceleration data matches the set action; action prediction values ​​corresponding to each of the multiple actions obtained by processing the characteristic values ​​of the acceleration data of the multiple actions through the output layer are determined; prediction error values ​​corresponding to each of the multiple actions are determined according to the action prediction values ​​and action label values ​​corresponding to each of the multiple actions; and a model weight of the motion recognition model is adjusted according to the prediction error values ​​corresponding to each of the multiple actions.

[0081] For ease of understanding, the following examples are provided with reference to FIG5a and FIG6:

[0082] In actual applications, after the first acceleration data corresponding to the collected first action is input into the action recognition model shown in Figure 5a, the action recognition model will output Y1. At this point, Y1 can be judged based on the first and second set thresholds. If Y1 is greater than the first set threshold, the user's first action is determined to be a set action, and the action recognition model is not adjusted. If Y1 is less than the second set threshold, the user's first action is determined to be a non-set action, and the action recognition model is not adjusted.

[0083] If Y1 is between the first set threshold and the second set threshold, the type of the user's first action is determined to be a suspected set action. At this time, a sample set is obtained, which may include: the feature value of the first acceleration data corresponding to the user's first action (see "Current Sample" in Figure 6) and the feature values ​​of the acceleration data of multiple actions obtained in advance (see "Prefabricated Sample Set" in Figure 6).

[0084] The acceleration data of multiple actions including the first action are input into the action recognition model, and the characteristic values ​​of the acceleration data of the multiple actions are processed respectively through the output layer to obtain the action prediction values ​​corresponding to the multiple actions. For example, the multiple actions include: the first action, the second action and the third action. Then, for each action, the action recognition model will output an action prediction value. For example, the action prediction value corresponding to the first action is 0.7, the action prediction value corresponding to the second action is 0.8, and the action prediction value corresponding to the third action is 0.9.

[0085] Afterwards, the action label values ​​corresponding to the first action, the second action, and the third action are determined. Assuming that the action label values ​​corresponding to the first action, the second action, and the third action are all 1, then the prediction error values ​​corresponding to the first action, the second action, and the third action are as follows: the error value of the first action = 1-0.7 = 0.3; the error value of the second action = 1-0.8 = 0.2; the error value of the first action = 1-0.9 = 0.1. According to the above-determined prediction error values, the model weight of the action recognition model can be adjusted. The specific adjustment process of the model weight is as follows:

[0086] Determine the set weight coefficients on the connection edges between the output nodes in the output layer of the action recognition model and the nodes in the previous network layer of the output layer; determine the cumulative prediction error value corresponding to the target connection edge between the target node in the previous network layer and the output node; wherein the target node is any node in the previous network layer, and the cumulative prediction error value is the cumulative sum of the products of the feature values ​​output by the target node for multiple actions and the prediction error values ​​corresponding to the corresponding actions; determine the model weight corresponding to the target connection edge based on the set weighting system and the cumulative prediction error value.

[0087] In practical applications, first, the set weight coefficient k on the edge connecting the output node in the output layer of the action recognition model and each node in the previous network layer of the output layer is determined. The set weight coefficient k can be pre-set according to actual conditions and is not limited here.

[0088] After that, the cumulative prediction error value corresponding to the target connection edge between the target node and the output node in the previous network layer is determined, continuing with the first, second, and third actions mentioned above as an example:

[0089] The model weight W corresponding to the edge formed by the output node obtained by the action recognition model for the first action and the first node in the network layer before the output layer 11 ' is: the eigenvalue corresponding to the first node (X 11 )×error value (0.3)×set weighting coefficient (k).

[0090] The model weight W corresponding to the connection formed by the output node obtained by the action recognition model for the second action and the first node in the network layer before the output layer 21 ' is: the eigenvalue corresponding to the first node (X 21 )×error value (0.2)×set weighting coefficient (k).

[0091] The model weight W corresponding to the edge formed by the output node obtained by the action recognition model for the third action and the first node in the network layer before the output layer 31 ' is: the eigenvalue corresponding to the first node (X 31 )×error value (0.1)×set weighting coefficient (k).

[0092] Then, the model weight corresponding to the connection edge formed by the output node obtained by the final action recognition model and the first node in the network layer before the output layer is W1'=W1+W 11 '+W 21 '+W 31 ', where W1 is the model weight before modification corresponding to the connection edge formed between the output node and the first node in the network layer before the output layer.

[0093] Similarly, the model weights corresponding to the edges formed by the output nodes obtained by the action recognition model and the second, third, and fourth nodes in the network layer before the output layer can be calculated according to the above method, and finally W1', W2', W3'...W n ', and then complete the adjustment of the model weight of the action recognition model.

[0094] Based on the above, the embodiment of the present application obtains a sample set, and combines the action prediction values ​​and action label values ​​corresponding to multiple actions in the sample set to determine the prediction error values ​​corresponding to multiple actions, thereby increasing the number of sample training and ensuring the accuracy of subsequent adjustment of the model weights of the action recognition model based on the prediction error values, so that the adjusted action recognition model can be more accurate and quickly identify the user's actions, and the user experience is better.

[0095] In some of the processes described in the above embodiments and the accompanying drawings, multiple operations that appear in a specific order are included, but it should be clearly understood that these operations may not be executed in the order in which they appear in this article or may be executed in parallel. The serial numbers of the operations, such as 101, 102, etc., are only used to distinguish between different operations, and the serial numbers themselves do not represent any order of execution. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this article are used to distinguish different messages, devices, modules, etc., and do not represent a sequential order, nor do they limit "first" and "second" to being different types.

[0096] The following describes in detail one or more embodiments of the present application of the screen lighting correction device for wearable devices. Those skilled in the art will understand that these devices can be configured using commercially available hardware components through the steps taught in this solution.

[0097] Figure 7 is a structural diagram of a screen lighting correction device for a wearable device provided in an embodiment of the present application. As shown in Figure 7, the device includes: an acquisition module 71, a determination module 72, a detection module 73, a control module 74 and an adjustment module 75.

[0098] The acquisition module 71 is configured to acquire first acceleration data generated by a wearable device worn by a user in response to a first action of the user.

[0099] The determination module 72 is configured to determine a category of the first action based on the first acceleration data and an action recognition model, wherein the category of the first action is one of the following: a set action, a non-set action, or a suspected set action.

[0100] The detection module 73 is configured to detect whether the user triggers an operation of lighting up the screen of the wearable device within a set time if the category of the first action is a suspected set action.

[0101] The control module 74 is configured to control the wearable device to light up the screen if it is detected that the user has triggered an operation to light up the screen of the wearable device within a set time.

[0102] The adjustment module 75 is configured to adjust the model weight of the action recognition model based on the first acceleration data.

[0103] Optionally, the determination module 72 is specifically used to: input the first acceleration data into the action recognition model to obtain a first action prediction value output by the action recognition model; if the first action prediction value is greater than a first set threshold, the category of the first action is determined to be: a set action; if the first action prediction value is between the first set threshold and the second set threshold, the category of the first action is determined to be: a suspected set action, wherein the second set threshold is less than the first set threshold; if the first action prediction value is less than the second set threshold, the category of the first action is determined to be: a non-set action.

[0104] Optionally, the adjustment module 75 is specifically used to: issue a prompt confirmation message, wherein the prompt confirmation message is used to confirm whether the first action is the set action; in response to the user's confirmation operation triggered by the prompt confirmation message, adjust the model weight of the action recognition model based on the first acceleration data.

[0105] Optionally, the adjustment module 75 is further specifically used to: add a first output node in the output layer of the action recognition model, the first output node is respectively connected to each node in the previous network layer, and a set weighting coefficient is set on each connecting edge; based on the input of the first acceleration data, determine the first eigenvalue output by each node in the previous network layer; according to the set weighting coefficient and the first eigenvalue, determine the model weight corresponding to the connecting edge between the first output node and each node in the previous network layer to obtain a first adjusted action recognition model.

[0106] Optionally, the adjustment module 75 is further specifically used to: collect second acceleration data generated by the wearable device following the user's second action; determine the category of the second action based on the second acceleration data and the first adjusted action recognition model; if the category of the second action is the suspected set action, and it is detected that the user triggers the operation of lighting up the screen of the wearable device within the set time, then control the wearable device to light up the screen; create a second output node in the output layer of the first adjusted action recognition model, and the second output node is respectively connected to each node in the previous network layer, and each connecting edge is set with the set weighting coefficient; based on the input of the second acceleration data, determine the second eigenvalue output by each node in the previous network layer; according to the set weighting coefficient and the second eigenvalue, determine the model weight corresponding to the connecting edge between the second output node and each node in the previous network layer to obtain the second adjusted action recognition model.

[0107] Optionally, the adjustment module 75 is further specifically used to: obtain the second action prediction value output by the original output node in the output layer of the first adjusted action recognition model and the third action prediction value output by the first output node based on the input of the second acceleration data; determine the category of the second action based on the maximum value of the second action prediction value and the third action prediction value.

[0108] Optionally, the adjustment module 75 is further specifically used to: obtain a sample set, the sample set including characteristic values ​​of acceleration data of multiple actions, the characteristic values ​​corresponding to each node in the previous network layer of the output layer in the action recognition model, the acceleration data of the multiple actions including the first acceleration data, and the action label value corresponding to the first acceleration data matches the set action; determine the action prediction values ​​corresponding to each of the multiple actions obtained after processing the characteristic values ​​of the acceleration data of the multiple actions respectively through the output layer; determine the prediction error values ​​corresponding to each of the multiple actions based on the action prediction values ​​and action label values ​​corresponding to each of the multiple actions; and adjust the model weight of the action recognition model based on the prediction error values ​​corresponding to each of the multiple actions.

[0109] Optionally, the adjustment module 75 is further specifically used to: determine the set weighting coefficient on the connection edge between the output node in the output layer of the action recognition model and each node in the previous network layer of the output layer; determine the cumulative prediction error value corresponding to the target connection edge between the target node in the previous network layer and the output node; wherein the target node is any node in the previous network layer, and the cumulative prediction error value is the cumulative sum of the products of the feature values ​​output by the target node for the multiple actions and the prediction error values ​​corresponding to the corresponding actions; determine the model weight corresponding to the target connection edge based on the set weighting system and the cumulative prediction error value.

[0110] The device shown in FIG7 can execute the steps of the screen lighting correction method for the wearable device in the aforementioned embodiment. The detailed execution process and technical effects are described in the aforementioned embodiment and will not be repeated here.

[0111] An embodiment of the present application also provides a wearable device, comprising a memory for storing computer program instructions and a processor for executing the program instructions, wherein, when the computer program instructions are executed by the processor, the wearable device is triggered to execute the above-mentioned screen lighting correction method for the wearable device.

[0112] The terminal device may be a wearable smart bracelet, a watch, etc. The embodiments of the present application do not limit the specific technology and specific device form adopted by the wearable device.

[0113] In order to better understand the embodiments of the present application, the structure of the terminal device applicable to the embodiments of the present application is described below. Figure 8 shows a schematic diagram of the structure of a wearable device provided in an embodiment of the present application. The wearable device 10 shown in Figure 8 may include a processor 110, a memory 120, a universal serial bus (USB) interface 130, a power supply 140, a communication module 150, and a display screen 160.

[0114] It is understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the wearable device 10. In other embodiments of the present application, the wearable device 10 may include more or fewer components than shown in the figure, or combine certain components, or split certain components, or arrange the components differently. The illustrated components can be implemented in hardware, software, or a combination of software and hardware. The processor 110 may include one or more processing units, for example: the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a digital signal processor (DSP, baseband processor), etc. Among them, different processing units may be independent devices or integrated into one or more processors.

[0115] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

[0116] Processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. This memory can store instructions or data that have just been used or are being recycled by processor 110. If processor 110 needs to use the same instruction or data again, it can directly access the memory. This avoids duplicate accesses, reduces processor 110 latency, and thus improves system efficiency.

[0117] The power supply 140 supplies power to the wearable device 10 .

[0118] The communication module 150 can use any transceiver type device to provide the wearable device 10 with the following functions:

[0119] Wireless Local Area Networks (WLAN) (such as Wireless Fidelity (Wi-Fi) networks), Bluetooth (BT), Global Navigation Satellite System (GNSS), Frequency Modulation (FM), Near Field Communication (NFC), Infrared (IR) and other wireless communication solutions. The communication module 150 can be one or more devices that integrate at least one communication processing module. The communication module 150 receives electromagnetic waves via an antenna, frequency modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110. The communication module 150 can also receive signals to be sent from the processor 110, frequency modulate them, amplify them, and convert them into electromagnetic waves for radiation through the antenna.

[0120] In some embodiments, the antenna of the wearable device 10 is coupled to the communication module 150 so that the wearable device 10 can communicate with a network and other devices via wireless communication technologies. The wireless communication technologies may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology. The GNSS may include global positioning system (GPS), global navigation satellite system (GLONASS), Beidou navigation satellite system (BDS), quasi-zenith satellite system (QZSS) and / or satellite based augmentation system (SBAS).

[0121] Wearable device 10 implements display functionality through a GPU, display 160, and an application processor. The GPU is a microprocessor for image processing that connects display 160 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or modify display information.

[0122] The display screen 160 is used to display images, videos, etc. The display screen 160 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a MiniLED, a MicroLED, a Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the wearable device 10 may include one or N display screens 160, where N is a positive integer greater than 1.

[0123] The memory 120 can be used to store one or more computer programs, each of which includes instructions. The processor 110 can execute the instructions stored in the memory 120, thereby enabling the wearable device 10 to perform various functional applications and data processing. The memory 120 may include a program storage area and a data storage area. The program storage area may store an operating system.

[0124] The storage data area can store data created during the use of the wearable device 10. In addition, the memory 120 can

[0125] The processor 110 includes a high-speed random access memory and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. In some embodiments, the processor 110 can execute instructions stored in the memory 120 and / or instructions stored in the memory provided in the processor 110 to enable the wearable device 10 to perform various functional applications and data processing.

[0126] FIG9 is a software structure block diagram of the wearable device 10 provided in an embodiment of the present application.

[0127] A layered architecture divides software into several layers, each with distinct roles and responsibilities. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers: the application layer, the framework layer, the Android runtime, the hardware abstraction layer, and the driver layer.

[0128] The application layer can include a series of application packages.

[0129] As shown in FIG9 , the application package may include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, and short message.

[0130] The application framework layer provides an application programming interface (API) and programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0131] As shown in FIG9 , the framework layer may include a phone framework, a Bluetooth framework, an audio framework, and the like.

[0132] The phone framework is used to manage phone applications. It can obtain the display size, determine whether there is a status bar, lock the screen, take screenshots, etc.

[0133] The Bluetooth framework is used to provide Bluetooth functionality.

[0134] The audio framework is used to provide audio data.

[0135] The Android Runtime consists of the core library and the virtual machine. The Android runtime is responsible for scheduling and management of the Android system.

[0136] The core library consists of two parts: one is the function that needs to be called by the Java language, and the other is the Android core library.

[0137] The application and framework layers run in a virtual machine. The virtual machine executes Java files from the application and framework layers as binary files. The virtual machine manages object lifecycles, stack management, thread management, security and exception management, and garbage collection.

[0138] The hardware abstraction layer can include multiple functional modules, such as call manager, Bluetooth manager, audio manager, etc.

[0139] The call manager is used to manage call functions.

[0140] The Bluetooth manager is used to manage Bluetooth functions.

[0141] The audio manager supports playback and recording of a variety of commonly used audio and video formats, as well as static image files, etc. The audio manager can also support a variety of audio and video encoding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0142] The driver layer is the layer between hardware and software. The driver layer includes at least display driver, Bluetooth driver, audio driver, etc.

[0143] The following is an exemplary description of the workflow of the software and hardware of the terminal device 10.

[0144] After determining the category of the first action based on the collected first acceleration data and the action recognition model, the processor 110 detects whether the user triggers the operation of lighting up the display screen 160 of the wearable device within the set time. If the display screen 160 receives a touch operation, the corresponding hardware interrupt is sent to the driver layer. The driver layer processes the touch operation into an original input event (including touch coordinates, timestamp of the touch operation and other information). The original input event is stored in the driver layer. The framework layer obtains the original input event from the driver layer and identifies the control corresponding to the input event. For example, if the touch operation is a touch single-click operation and the control corresponding to the single-click operation is the control of the confirmation icon of the user to light up the wearable device, after identifying the control, the processor 110 controls the lighting of the display screen 160 and adjusts the model weight of the action recognition model based on the first acceleration data.

[0145] An embodiment of the present application further provides a computer-readable storage medium, which includes a stored program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute the screen lighting correction method for the wearable device.

[0146] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0147] The present application is described with reference to the flow chart and / or block diagram of the method, device (system), and computer program product according to the embodiment of the present application. It should be understood that each flow process and / or box in the flow chart and / or block diagram and the combination of the flow process and / or box in the flow chart and / or block diagram can be realized by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processing machine or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device for realizing the function specified in one flow chart flow or multiple flows and / or one box or multiple boxes of the block diagram.

[0148] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction device that implements the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0149] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.

[0150] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0151] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0152] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0153] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0154] The above content is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A method for calibrating the screen lighting of a wearable device, characterized in that, Including: Collecting first acceleration data generated by a wearable device worn by a user along with a first action of the user; Determining the category of the first action based on the first acceleration data and an action recognition model, where the category of the first action is one of the following: a set action, a non-set action, a suspected set action; If the category of the first action is a suspected set action, then detecting whether the user triggers an operation to light up the screen of the wearable device within a set time; If it is detected that the user triggers an operation to light up the screen of the wearable device within the set time, then controlling the wearable device to light up the screen; Adjusting the model weights of the action recognition model based on the first acceleration data.

2. The method according to claim 1, wherein The determining the category of the first action based on the first acceleration data and the action recognition model includes: Inputting the first acceleration data into the action recognition model to obtain a first action prediction value output by the action recognition model; If the first action prediction value is greater than a first set threshold, then determining that the category of the first action is: a set action; If the first action prediction value is between the first set threshold and a second set threshold, then determining that the category of the first action is: a suspected set action, where the second set threshold is less than the first set threshold; If the first action prediction value is less than the second set threshold, then determining that the category of the first action is: a non-set action.

3. The method according to claim 1, wherein The adjusting the model weights of the action recognition model based on the first acceleration data includes: Sending a prompt confirmation message, where the prompt confirmation message is used to confirm whether the first action is the set action; In response to a confirmation operation triggered by the user for the prompt confirmation message, adjusting the model weights of the action recognition model based on the first acceleration data.

4. The method according to claim 1, wherein The adjusting the model weights of the action recognition model based on the first acceleration data includes: Adding a first output node to the output layer of the action recognition model, where the first output node is respectively connected to each node in the previous network layer, and a set weighting coefficient is set on each connection edge; Based on the input of the first acceleration data, determining first eigenvalue outputs by each node in the previous network layer; According to the set weighting coefficient and the first eigenvalue, determining the model weights corresponding to the connection edges between the first output node and each node in the previous network layer to obtain a first adjusted action recognition model.

5. The method according to claim 4, wherein It further includes: Collecting second acceleration data generated by the wearable device along with a second action of the user; Determining the category of the second action based on the second acceleration data and the first adjusted action recognition model; If the category of the second action is the suspected set action and it is detected that the user triggers an operation to light up the screen of the wearable device within the set time, then controlling the wearable device to light up the screen; Creating a second output node in the output layer of the first adjusted action recognition model, where the second output node is respectively connected to each node in the previous network layer, and the set weighting coefficient is set on each connection edge; Based on the input of the second acceleration data, determine the second eigenvalue output by each node in the previous network layer; According to the set weighting coefficient and the second eigenvalue, determine the model weights corresponding to the connection edges between the second output node and each node in the previous network layer, so as to obtain a second adjusted action recognition model.

6. The method according to claim 5, wherein The determining the category of the second action based on the second acceleration data and the first adjusted action recognition model includes: Based on the input of the second acceleration data, obtain the second action prediction value output by the original output node in the output layer of the first adjusted action recognition model, and the third action prediction value output by the first output node; Based on the maximum value of the second action prediction value and the third action prediction value, determine the category of the second action.

7. The method according to claim 1, wherein The adjusting the model weights of the action recognition model based on the first acceleration data includes: Obtain a sample set, where the sample set includes the eigenvalue of the acceleration data of multiple actions, the eigenvalue corresponds to each node in the previous network layer of the output layer in the action recognition model, the acceleration data of the multiple actions includes the first acceleration data, and the action label value corresponding to the first acceleration data matches the set action; Determine the action prediction value corresponding to each of the multiple actions obtained by processing the eigenvalue of the acceleration data of the multiple actions through the output layer respectively; According to the action prediction value and the action label value corresponding to each of the multiple actions, determine the prediction error value corresponding to each of the multiple actions; Adjust the model weights of the action recognition model according to the prediction error value corresponding to each of the multiple actions.

8. The method according to claim 7, characterized in that, The adjusting the model weights of the action recognition model according to the prediction error value corresponding to each of the multiple actions includes: Determine the set weighting coefficient on the connection edge between the output node in the output layer of the action recognition model and each node in the previous network layer of the output layer; Determine the cumulative prediction error value corresponding to the target connection edge between the target node and the output node in the previous network layer; wherein, the target node is any node in the previous network layer, and the cumulative prediction error value is the sum of the products of the eigenvalue output by the target node for each of the multiple actions and the prediction error value corresponding to the corresponding action; According to the set weighting coefficient and the cumulative prediction error value, determine the model weight corresponding to the target connection edge.

9. A wearable device, characterized in that, Including: A memory, a processor, and a communication interface; wherein, an executable code is stored on the memory, and when the executable code is executed by the processor, the processor executes the screen lighting correction method of the wearable device according to any one of claims 1 to 8.

10. A non-transitory machine-readable storage medium, characterized in that, An executable code is stored on the non-transitory machine-readable storage medium, and when the executable code is executed by the processor of the electronic device, the processor executes the screen lighting correction method of the wearable device according to any one of claims 1 to 8.

11. A computer program product, characterized in that, Including a computer program, when the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images