Music recommendation method and device based on wearable equipment, medium and equipment

Abstract

According to the music recommendation method and device based on the wearable device, the medium and the equipment provided by the invention, when the scene data sent by the wearable device connected with the terminal is received, the representation vector used for representing the instant state and historical preference of the user is generated based on the scene data and the user behavior recorded in the local behavior log of the terminal; and then personalized music conforming to the current representation vector is provided for the user. Personalized music is provided for the user based on the fused multi-terminal information, and the music recommendation accuracy of the user is improved.

Description

Technical Field

[0001] This specification relates to the field of computers, and in particular to methods, apparatuses, media, and devices for music recommendation based on wearable devices. Background Technology

[0002] Modern music platforms typically possess massive libraries of tens of millions or even hundreds of millions of songs. Faced with such a vast selection, users who want to update their playlists or find new songs experience significant time commitments, whether searching or listening to each song individually, easily leading to "selection paralysis." To improve user experience, providing music recommendation services is crucial for music platforms.

[0003] In existing technologies, music platforms typically provide recommendation services based on music stored locally on wearable devices or terminals, such as the user's playback history and music metadata. However, relying solely on locally stored music limits music resources and cannot meet users' needs for discovering new music. Summary of the Invention

[0004] This specification aims to at least partially address one of the technical problems in the related art.

[0005] Therefore, the first objective of this specification is to propose a music recommendation method based on wearable devices.

[0006] The second objective of this specification is to propose a music recommendation method based on wearable devices.

[0007] The third objective of this specification is to propose a music recommendation system based on wearable devices.

[0008] The fourth objective of this specification is to provide an electronic device.

[0009] The fifth objective of this specification is to provide a computer-readable storage medium.

[0010] The sixth objective of this specification is to provide a computer program product.

[0011] To achieve the above objectives, a first aspect of this specification proposes a music recommendation method based on a wearable device, the method being applied to a terminal and comprising:

[0012] Acquire contextual data sent by a wearable device, wherein the contextual data includes at least one contextual vector and a timestamp corresponding to each contextual vector;

[0013] Obtain candidate music vectors that match the context vector;

[0014] Based on the timestamp corresponding to each of the local context vectors, determine the user behavior corresponding to the context vector in the behavior log stored locally;

[0015] Based on the context data, each context vector stored locally, and the corresponding user behavior, a user representation vector is determined.

[0016] Based on the user representation vector, the target music recommended for the user is determined from the candidate music vector.

[0017] To achieve the above objectives, a second aspect of this specification provides a music recommendation method based on a wearable device, the method being applied to the wearable device and comprising:

[0018] Based on the data collected by the first sensor, the user's current situation is determined;

[0019] Based on the user's current situation and the data collected by the first sensor, a situation vector is generated, wherein the situation vector includes the identifier of the user's current situation, the confidence level of the user's current situation, the data collected by the first sensor, and situation metadata.

[0020] The context vector and the timestamps corresponding to the data collected by the first sensor are sent to the terminal.

[0021] To achieve the above objectives, a third aspect of this specification provides a music recommendation system based on wearable devices. The system includes a terminal and a wearable device, comprising:

[0022] The terminal is used to execute the method described above;

[0023] The wearable device is used to perform the method described above.

[0024] To achieve the above objectives, a fourth aspect of this specification provides an electronic device, including: a processor, and a memory communicatively connected to the processor;

[0025] The memory stores computer-executed instructions;

[0026] The processor executes computer execution instructions stored in the memory to implement the method described in the first aspect embodiment above.

[0027] To achieve the above objectives, a fifth aspect of this specification provides a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, are used to implement the method described in the first aspect of the specification.

[0028] To achieve the above objectives, a sixth aspect of this specification provides a computer program product including a computer program that, when executed by a processor, implements the method described in the first aspect of the specification.

[0029] The music recommendation method, apparatus, medium, and device based on wearable devices provided in this specification, upon receiving contextual data sent by a wearable device connected to a terminal, generate a representation vector representing the user's current state and historical preferences based on the contextual data and user behavior recorded in the terminal's local behavior log. Then, it provides the user with personalized music that matches the current representation vector. By providing personalized music to users based on fused multi-terminal information, the accuracy of music recommendations is improved.

[0030] Additional aspects and advantages of this specification will be set forth in part in the description which follows, and in part will be obvious from the description or may be learned by practice of this specification. Attached Figure Description

[0031] The above and / or additional aspects and advantages of this specification will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, in which:

[0032] Figure 1 A schematic flowchart illustrating the music recommendation method based on wearable devices provided in the embodiments of this specification;

[0033] Figure 2 A schematic flowchart illustrating a fusion method provided in an embodiment of this specification;

[0034] Figure 3 A schematic diagram illustrating the process of a model training method provided in an embodiment of this specification;

[0035] Figure 4 This is a flowchart illustrating a music recommendation method based on wearable devices provided in an embodiment of this specification. The method is applied to wearable devices.

[0036] Figure 5 A flowchart illustrating a data acquisition method provided in an embodiment of this specification;

[0037] Figure 6 A schematic diagram of the structure of a music recommendation device based on a wearable device provided in the embodiments of this specification;

[0038] Figure 7 This is a schematic diagram of the structure of a music recommendation device based on a wearable device provided in the embodiments of this specification. Detailed Implementation

[0039] The embodiments of this specification are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this specification, and should not be construed as limiting this specification.

[0040] The following description, with reference to the accompanying drawings, outlines a method, apparatus, medium, and device for recommending music based on wearable devices, according to embodiments of this specification.

[0041] With the development of computer technology, digital music has gradually become the mainstream form of music existence and dissemination today. Digital music refers to music processed, stored, and distributed using digital technology. Therefore, software or services that provide storage, transmission, playback, and management for digital music—music platforms—have emerged. Modern music platforms typically possess massive libraries of tens of millions or even hundreds of millions of songs. Faced with such a vast selection, when users want to update their playlists or find new songs, whether searching or listening to each song individually, it consumes a significant amount of time, easily leading to "selection paralysis." To improve user experience, providing music recommendation services is particularly important for music platforms.

[0042] Currently, music recommendation services for users can be mainly divided into three methods. First, based on music stored locally on wearable devices or terminals, such as playback history and music metadata, music recommendation services can be provided. However, recommending music based solely on local music resources is limited and cannot meet users' needs for discovering new music. Second, centralized cloud-based recommendations are possible. Terminals or wearable devices can use the software development kit (SDK) of the locally installed music platform to upload user listening history, preferences, and other data to a cloud server. The cloud server then uses collaborative filtering, content filtering, or hybrid algorithms to generate recommendations. However, uploading user data to the cloud for analysis poses a risk of privacy breaches, and users have limited control over their personal data. Third, recommendation algorithms can be deployed on wearable devices. Sensors installed on these devices can collect user data and infer the user's emotions, activity levels, and other contextual states to generate recommendations tailored to that user's situation. For example, when a user is exercising, upbeat music can be recommended, while when the user is resting, soothing music can be recommended. However, wearable devices have limited resources, such as computing power and memory, making it difficult to run complex algorithms, which makes it difficult for music platforms to provide recommendations that meet user needs.

[0043] Based on this, embodiments of this specification propose a music recommendation method based on wearable devices, which is applied to a terminal. It should be noted that in one or more embodiments of this specification, the specific type of terminal is not limited. It can be a mobile terminal, such as a mobile phone, tablet, or laptop, or a fixed terminal, such as a server or desktop computer. However, wearable devices are generally connected to mobile terminals; therefore, this specification will use a mobile terminal as an example to describe the music recommendation method based on wearable devices. Furthermore, a wearable device refers to a device equipped with at least one sensor and worn by a user, such as a smartwatch, bracelet, Bluetooth headset, or Bluetooth glasses; this specification does not limit this. The wearable device can interact with the terminal and collect user data. Figure 1 As shown, Figure 1 A flowchart illustrating the music recommendation method based on wearable devices provided in the embodiments of this specification includes:

[0044] Step 100: Obtain contextual data sent by the wearable device, wherein the contextual data includes at least one contextual vector and a timestamp corresponding to each contextual vector.

[0045] To provide personalized music recommendations, the terminal can first acquire contextual data sent by the wearable device when providing music recommendation services, and then analyze the user's current context based on the contextual data. The contextual data sent by the wearable device includes at least one contextual vector and a timestamp corresponding to each contextual vector.

[0046] It should be noted that, in one or more embodiments of this specification, there is no limitation on when the terminal acquires contextual data sent by the wearable device. It may acquire the data transmission signal sent by the wearable device in response to a data transmission signal, or it may actively send a contextual data acquisition request to the wearable device in response to a user's operation on the terminal, determining that the user has a need for music recommendations, thereby acquiring the contextual data sent by the wearable device. Alternatively, it may acquire the latest user contextual data from the wearable device based on analysis of periodically acquired historical contextual data indicating a change in the user's context. It may also acquire the user's contextual data from the wearable device whenever any of the above situations occur. Of course, acquiring contextual data sent by the wearable device can also begin under other circumstances. This specification does not impose any limitations on this.

[0047] In addition, there is no restriction on the specific method by which the wearable device transmits the collected context data to the terminal. The specific transmission method depends on the connection method between the wearable device and the terminal. If the two are connected by a data cable, the context data is transmitted to the terminal through the data cable. If the wearable device and the terminal communicate via a secure short-range communication protocol, such as Bluetooth Low Energy, Wi-Fi, etc., then the context data can be transmitted based on the short-range communication protocol such as Bluetooth or Wi-Fi.

[0048] In addition, in one or more embodiments of this specification, the context data refers to a data collection that includes one or more context vectors and the time stamps corresponding to the context vectors. Among them, the context vector includes the user's situation, the confidence level of the situation, the collected user data, and context metadata. The user data includes the average speed, average heart rate, average step frequency, movement amplitude, etc. of the user's movement, which can be set according to actual needs or the user's situation. The context metadata includes meta-information such as the duration of the situation and the geographical location. The context vector is generated by the wearable device based on the collected user data. Of course, it can also be generated by the terminal based on the user data collected by the wearable device. This specification does not limit this.

[0049] Step 101: Obtain candidate music vectors that match the context vector.

[0050] After obtaining the context data, the terminal can determine candidate music vectors that match the context data based on the context data, so as to further screen the candidate music vectors subsequently, thereby obtaining music recommendations for the user.

[0051] It should be noted that in one or more embodiments of this specification, there is no restriction on the specific method by which the terminal matches the context vector and the candidate music vector. It can be by calculating the vector similarity and taking the music vectors with a similarity reaching a preset value as the candidate music vectors that match, or it can be by using a pre-trained classification model to determine the music vectors corresponding to the situations included in the context data as the candidate music vectors. It can also be based on a preset rule to determine the candidate music vectors corresponding to each situation according to the range of beats per minute (BPM) corresponding to each situation. For example, music with BPM > 130 corresponds to the movement situation, music with 80 < BPM ≤ 130 corresponds to the stationary situation, and music with BPM ≤ 80 corresponds to the sleep situation. Since there are many optional methods, this specification does not limit this and can be set according to actual needs.

[0052] Furthermore, the aforementioned candidate music vectors represent the multi-dimensional features of music in a numerical and structured vector form. They describe at least one attribute of the music, such as style, rhythm, energy level, lyric thematic embedding, and pre-labeled applicable contextual tags, using numbers or labels. For example, [100, 0.9, pop] indicates that the music's BPM = 100, positive emotional intensity = 0.9, and the music style is pop. Another example is [100, 0.9, 0.7, 0.3, pop], which indicates that the music's BPM = 100, positive emotional intensity = 0.9, volume intensity = 0.7, rhythmic complexity = 0.3, and the music style is pop. Since there are many possible methods, they are not listed here; settings can be made according to actual needs. The candidate music vectors can be pre-determined by the music platform during music upload.

[0053] Step 102: Based on the timestamp corresponding to each context vector in local storage, determine the user behavior in the behavior log in local storage that corresponds to the context vector.

[0054] To further filter candidate music vectors, the terminal can analyze the user's historical preferences based on each locally stored context vector and user behavior logs. This allows for further filtering of candidate music vectors based on the user's historical preferences.

[0055] Specifically, for each context vector stored locally, and its corresponding timestamp, user behavior within a preset time range is determined from the locally stored behavior log for each timestamp. This user behavior includes actions such as playing / pausing songs, listening to the full duration, fast-forwarding / rewinding, skipping, adding to playlists, removing from playlists, liking / disliking, favorited / unfavorited, sharing, searching, and creating / modifying playlists. Each user behavior corresponds to a local timestamp and associated metadata, including song ID, artist ID, album ID, user ID, and session ID.

[0056] By using the timestamps of context vectors and user behaviors, the time alignment between user behaviors and each context vector is achieved, allowing the filtering of user behaviors corresponding to each context vector. This identifies context vectors and user behaviors that have a time alignment relationship.

[0057] Step 103: Determine the user representation vector based on the context data, each context vector stored locally, and the corresponding user behavior.

[0058] After determining the context vectors and user behaviors that have time alignment relationships, the terminal can determine the user representation vector based on the context data, each of the context vectors stored locally and the corresponding user behaviors, and obtain the user representation vector that combines the user's immediate context and historical preferences.

[0059] Specifically, the terminal generates a user first preference vector based on each context vector stored locally and the corresponding user behavior, using sequence modeling techniques to represent the user's stable music taste and long-term interests. For example, Word2Vec / Item2Vec can be applied to the song ID sequence, or more complex recurrent neural networks (RNN / LSTM / GRU) or Transformer encoders can be used to model sequences containing song features and user behavior types to determine the user first preference vector.

[0060] Alternatively, the contextual data, each locally stored contextual vector, and the corresponding user behavior can be directly encoded and concatenated to determine the user representation vector. For example, text can be concatenated based on each locally stored contextual vector and the corresponding user behavior to generate text. After generating the text, the semantic encoding of the text can be determined, and this semantic encoding can be used as the user preference vector. Of course, since there are many methods for determining user preference vectors, this manual will not list them all; you can set them according to your actual needs.

[0061] Then, the first contextual data in the contextual data sent by the wearable device is fused with the user's first preference vector, and the fusion result is used as the user representation vector.

[0062] It should be noted that, in this embodiment, the specific method used to fuse the first contextual data in the contextual data sent by the wearable device with the user's first preference vector is not limited. A fixed-weighted fusion method can be used, or the first contextual data in the contextual data sent by the wearable device can be directly concatenated with the user's first preference vector. Many methods are possible, and they will not be listed here. Other methods will be described in detail later.

[0063] Step 104: Based on the user representation vector, determine the target music recommended for the user from the candidate music vectors.

[0064] Once the user representation vector is determined, the terminal can further filter the candidate music vectors based on the determined user representation vector.

[0065] Specifically, the server can calculate the similarity between the user representation vector and each candidate music vector, and then determine the music recommended to the user based on the similarity scores. It should be noted that in one or more embodiments of this specification, the specific method used by the terminal to calculate the similarity is not limited. It can calculate at least one of the cosine similarity and vector similarity between the user representation vector and the candidate music vectors, and then output a preset number of music corresponding to the candidate music vectors in descending order of similarity. Alternatively, it can output all music corresponding to candidate music vectors with similarity scores greater than a preset value at once. Alternatively, based on the music corresponding to the candidate music vectors determined by similarity, it can select music that is actively played on the wearable device, such as listening to it completely, repeating it, or marking it as a favorite, and prioritize recommend it for the user to choose and play. Since there are many recommendation methods, this specification does not list them all; the method can be set according to actual needs.

[0066] In this embodiment, upon receiving contextual data from a wearable device connected to the terminal, a representation vector representing the user's current state and historical preferences is generated based on the contextual data and user behavior recorded in the terminal's local behavior log. Then, personalized music matching the current representation vector is provided to the user. By providing personalized music based on fused multi-terminal information, the accuracy of music recommendations is improved.

[0067] In step 103 above, when determining the user representation vector, the fixed weighting or simple vector concatenation method, which integrates contextual and preference features, makes it difficult to effectively model the complex and dynamically changing interaction between the two, potentially leading to recommendation results that deviate from the user's real-time needs. Therefore, as... Figure 2 As shown, Figure 2 This is a flowchart illustrating a fusion method provided in an embodiment of this specification.

[0068] Step 200: Determine candidate music tags associated with the context identifiers in the context data.

[0069] Step 201: Based on the candidate music tags, update the first preference vector to obtain a second preference vector, wherein the element value in the second preference vector corresponding to the candidate music tags is greater than the element value in the first preference vector corresponding to the candidate music tags.

[0070] Step 202: Determine the first weight corresponding to the context data and the second weight corresponding to the second preference vector.

[0071] Step 203: Based on the first weight and the second weight, the second preference vector and the contextual data are weighted and fused to obtain the user representation vector.

[0072] Because fixed weighting or simple vector concatenation methods for fusing contextual and preference features are insufficient for effectively modeling the complex and dynamically changing interaction between the two, this server can dynamically fuse the first preference vector and contextual data to address this issue. Specifically, it increases the attention score of context-related dimensions in the contextual data within the user's first preference vector, thereby generating a second preference vector. Then, the second preference vector is weighted and fused with the contextual data to obtain the user representation.

[0073] When updating the first preference vector, an attention mechanism can be used to update it based on the determined candidate music tags. Specifically, the attention mechanism dynamically focuses on and weights the preferences or dimensions most relevant to the contextual data from the first preference vector, thereby updating the first preference vector and determining the second preference vector. This involves first determining candidate music tags associated with the contextual identifiers in the contextual data. These candidate music tags can be determined based on each candidate music vector; for example, the tag for a candidate music vector with a BPM > 130 corresponds to sports music or fast-paced music. The method for determining candidate music vectors is described above and will not be repeated here.

[0074] Then, after determining the second preference vector, the terminal can use a gating mechanism to calculate the first weight corresponding to the contextual data and the second weight corresponding to the second preference vector. Based on the first weight and the second weight, the second preference vector and the contextual data are weighted and fused to obtain the user representation vector.

[0075] Specifically, a gating unit is used to calculate a dynamic weight W. context =σ(W c ·V context +U c ·V preference ′+b c V fused =w context ⊙V context +(1-w context )⊙V preference ′, where V context This is contextual data. V preference ′ is the second preference vector. W c U c It is a learnable parameter matrix, b c These are bias vectors, which together determine the calculation of the gating weights. σ is the sigmoid activation function, ensuring the dynamic weights w are... context The output value is in the range (0,1), intuitively reflecting the extent to which the current recommendation should be dominated by real-time contextual information. ⊙ represents the element-wise product. V fusedThis represents the user's representation vector, which more accurately reflects the user's real-time music preferences at a specific time and in a specific scenario.

[0076] In this embodiment, by updating the first preference vector based on contextual data, a second preference vector that more accurately reflects the user's preferences is generated, namely, the real and real-time music preferences in a specific time and scenario.

[0077] Furthermore, when recommending music to users based on user representation vectors, the terminal can also recommend music using a pre-trained recommendation model. Specifically, the user representation vector and each candidate music vector are input into the trained recommendation model to obtain the target music output by the model. The recommendation model is determined based on the aggregated model parameters sent by the server. This recommendation model can be a classification model or a dual-tower model. Because dual-tower models, especially optimized dual-tower neural networks, are more lightweight and efficient in recommendation tasks, this specification uses a dual-tower model as an example to describe this music recommendation method.

[0078] Specifically, the user representation vector is input into the user tower, and the candidate music vector is input into the item tower. The similarity between the user representation vector and each candidate music vector is calculated. Based on the similarity, the dual-tower model determines the music recommended to the user.

[0079] Furthermore, to protect user privacy data, the terminal can also employ federated learning algorithms, such as... Figure 3 As shown, Figure 3 This is a schematic diagram illustrating the process of a model training method provided in an embodiment of this specification.

[0080] Step 300: When the update condition of the recommendation model is met, determine at least one user representation vector and the corresponding music vector obtained within a preset time before the current moment. The music vector includes playback duration and music identifier.

[0081] Step 301: Use each user representation vector and the corresponding music vector as training samples to update the local recommendation model to be updated.

[0082] Step 302: Send the updated recommendation model parameters to the server and obtain the aggregated model parameters returned by the server.

[0083] Step 303: Update the recommendation model based on the aggregated model parameters.

[0084] To ensure user data privacy, the recommendation model can be trained using a federated learning algorithm. When the update condition of the recommendation model is met, at least one user representation vector and its corresponding music vector, acquired within a preset time period before the current moment, are determined. The music vector includes playback duration and music identifier. Then, each user representation vector and its corresponding music vector are used as training samples to update the local recommendation model. The timestamps of the user representation vector and its corresponding music vector are aligned; that is, the difference between their timestamps does not exceed a preset difference. This process is repeated for each user representation vector and its corresponding music vector.

[0085] Specifically, when training the recommendation model locally on the terminal, the terminal can use the user representation vector V fused The main input is processed through several layers of fully connected networks, residual connections, or miniaturized self-attention modules, ultimately outputting a fixed-dimensional user representation vector U. user Then, the music vector corresponding to each user's representation vector is input into the item tower, and through a network structure of several layers, the music vector I is output. item .

[0086] By calculating U user and I item Similarity is used to predict interactions. Common similarity calculation methods include dot product, cosine similarity, or concatenating two vectors and feeding them into a small multilayer perceptron to predict user preference scores for items or interaction probabilities.

[0087] Then, to guide the recommendation model to better understand and utilize contextual data and improve the performance of core recommendation tasks, a composite loss function can be designed and adopted for multi-task learning. That is, the local recommendation model is trained on the terminal at least once to minimize the loss function. To achieve this, a local recommendation model is trained. As an example, the loss function formula is:

[0088]

[0089] in, This represents the main loss term, used to optimize the prediction of user-item interactions. Its specific form depends on the nature of the recommendation task. For example, for implicit feedback, such as click / listen, negative log-likelihood loss or binary cross-entropy loss can be used. This represents the user's actual interaction label with the item; for example, click = 1, no click = 0, meaning the recommendation model predicts the probability that the user will interact with the music corresponding to the clicked music vector.

[0090] This is used to represent auxiliary loss terms, aiming to enhance the recommendation model's ability to encode and understand contextual information. For example, one could try using the representation output by the user tower or intermediate layer representations to predict the coarse-grained or fine-grained context label c encoded in the current context vector. true The loss function can be either mean squared error or cross-entropy loss, c predict This auxiliary task, which involves the recommendation model predicting context, helps extract more context-relevant user representations by explicitly requiring the model to focus on context, thus indirectly improving the performance of the main recommendation task.

[0091] λ ctx It is a hyperparameter used to balance the relative importance between the primary recommendation task and the auxiliary context prediction task.

[0092] After completing the local recommendation model training on each terminal, each terminal can send the updated recommendation model parameters to the music platform's server. The server then aggregates the parameters from each terminal, obtaining aggregated model parameters, and returns these parameters to each terminal. Each terminal then updates its local recommendation model based on these aggregated parameters. This achieves iterative optimization of the global recommendation model through federated learning.

[0093] It should be noted that before aggregation, the music platform's server can perform a preliminary check on the received model parameters, filtering out data packets with incorrect formats or corruption, and identifying abnormal updates that may be submitted by malicious terminals, have excessively large gradient values, or are significantly inconsistent with other updates, in order to prevent them from adversely affecting the global recommendation model. Updates identified as potentially malicious may be excluded from aggregation or have their aggregation weight reduced.

[0094] Then, in one or more embodiments of this specification, the specific method by which the server aggregates the parameters of the recommendation model sent by each terminal is not limited; for example, a federated averaging algorithm or its variants may be used as the core aggregation strategy. The server performs a weighted average operation on the model parameters uploaded by each client based on the amount of data used in local training. The updated global model parameters are calculated after the first round of aggregation as follows:

[0095]

[0096] Among them, W (t+1) This represents the parameters of the global recommendation model after aggregation in round t+1, i.e., the current round. This indicates that the k-th client, after local training, uses the previous round's global recommendation model W... (t) Uploaded local model parameters. K is the total number of terminals participating in this aggregation round. n kis the number of data samples used for local training by the k-th terminal, or other forms of contribution weight. N is the total number of data samples from all terminals participating in this round of aggregation, i.e.

[0097] Furthermore, to prevent any single terminal's model parameter updates from being directly accessed by the server or any other unauthorized entity during the aggregation process, the aggregation process is executed in an encrypted state, such as homomorphic encryption, secure multi-party computation, and privacy-preserving mechanisms. Taking homomorphic encryption for aggregating model parameters as an example, the server obtains the decrypted aggregated model update (e.g., an average gradient update ΔW). (t) Or the average model parameters W (t+1) Then, it is applied to the global model W from the previous round. (t) If the aggregation is of the model parameters of the recommendation model itself: That is, W in the formula of the previous section. (t+1) If the aggregation is the recommendation model update amount ΔW k : It is the update amount after weighted average.

[0098] Then, after storing the aggregated global model parameters, the server distributes the aggregated model parameters to each terminal, allowing each terminal to update the recommendation model locally.

[0099] In this embodiment, the federated algorithm protects users' privacy data while completing the parameter aggregation of the global recommendation model.

[0100] Furthermore, in one or more embodiments of this specification, the specific content of the update conditions for the recommendation model is not limited. The update can occur when the update cycle is reached, when the terminal is idle, charging, or connected to Wi-Fi, in response to a user-initiated update command, after a preset number of context vectors or user behaviors have been stored, or when at least two of the above conditions are met. Due to the large number of possible solutions, this specification does not list them all.

[0101] like Figure 4 As shown, Figure 4 This is a flowchart illustrating a music recommendation method based on wearable devices, provided in an embodiment of this specification. The method is applied to wearable devices.

[0102] Step 400: Determine the user's situation based on the data collected by the first sensor.

[0103] In this embodiment, the contextual data is collected and generated by the wearable device based on its own installed first sensor.

[0104] It should be noted that, in one or more embodiments of this specification, the type of data collected depends on the type of sensors installed on the wearable device. The collected data includes at least one of the following: user biological data, motion data, activity data, and geographic location information, as well as the timestamp of the collected user data. Specifically, the user's biological data can be a signal obtained by the wearable device using photoplethysmography (PPG) to collect the change in the user's blood volume over time, i.e., determined by the PPG signal. The wearable device can also extract physiological indicators such as the user's heart rate and heart rate variability based on the PPG signal. Motion data can be user body movement and posture data, activity intensity, cadence, and step count obtained by the wearable device based on inertial sensors, etc. The wearable device can also obtain the user's geographic location data based on the Global Positioning System (GPS), including but not limited to geographic coordinates, altitude, movement trajectory, and movement speed.

[0105] After collecting data, wearable devices can determine the user's situation based on the data. For example, the low-power activity recognition module built into the system on a chip (SoC) or coprocessor of modern wearable devices can directly output basic states such as "stationary", "walking", "running" and "in a car".

[0106] Alternatively, it can be based on the feature thresholding method, analyzing the user's situation. Specifically, this involves analyzing low-frequency accelerometer data, such as the X, Y, and Z axis readings of inertial sensors or the magnitude of their composite vectors. The time window is divided into short time windows. Within each window, one or more simple statistical features are calculated to determine at least one of the following: signal amplitude variance or standard deviation, signal energy, zero-crossing rate (ZCR), and activity count. The zero-crossing rate refers to the frequency with which the signal crosses its mean. This indicator can effectively distinguish between periodic motion, such as walking, and a steady state.

[0107] Then, if both variance and energy are below a preset low threshold, the user's situation is determined to be static. If variance or energy is above a preset high threshold, the user's situation is determined to be a significant movement trend. If it is between the two, or if the variance is moderate but the ZCR is low, the user's situation is determined to be a slight activity / non-specific movement. If N consecutive windows, for example, N=2 or 3, satisfy the conditions of the same state and are different from the previous state, then a state transition is confirmed.

[0108] Then, the wearable device can dynamically adjust the content of the collected data according to the user's situation. For example, if the user's situation is identified as a high-intensity exercise situation, target data is extracted from the data collected by the sensors. The target data includes PPG data-derived features, inertial data-derived features, and GPS data-derived features. Among them, PPG data-derived features include frame-by-frame heart rate, time-domain heart rate variability index, frequency-domain HRV index optimized for short time windows, and short-term heart rate recovery rate after exercise. Inertial data-derived features include cadence, stride estimation, vertical amplitude, ground contact time and airtime estimation, motion intensity quantification based on acceleration signal energy, and posture stability parameters. GPS data-derived features include instantaneous velocity, average velocity, rate of ascent, and motion trajectory smoothness.

[0109] If the user's situation is identified as one of focused work or sedentary activity, PPG data-derived features include HRV indicators related to the long-term regulatory state of the autonomic nervous system, such as VLF and ULF, and the stability of average heart rate. Inertial sensor data-derived features include micro-motion features, such as the frequency and amplitude of minute adjustments in body posture during prolonged sedentary periods, to indirectly assess the user's level of focus or anxiety. Furthermore, acoustic environmental features can be acquired, i.e., analyzing environmental audio signals and extracting non-semantic acoustic features such as classification of environmental noise levels (e.g., quiet, office noise, noisy street, human voice density estimation).

[0110] After extracting user data based on the user's context, it can be input into one or more highly optimized, low-latency master classifiers. These master classifiers aim to identify the core context category of the user with minimal computational overhead, such as high-intensity exercise, low-intensity activity, focused work, rest and relaxation, or commuting. The classifier can be a lightweight machine learning model, such as a quantized and pruned decision tree / random forest / gradient boosting tree, compact neural network, support vector machine, probabilistic model, etc. This manual does not impose restrictions; it can be configured according to actual needs.

[0111] Alternatively, it can be implemented based on expert model features. Each expert model is independently trained for various sub-contexts within its assigned core context, thus learning more subtle distinguishing features. For example, a "sports type segmentation" expert model might focus on distinguishing between "outdoor high-intensity interval running," "gym strength training bursts," and "swimming," while a "work status segmentation" expert model might focus on distinguishing between "deep coding work," "online meeting speaking," and "document reading." The expert model outputs the probability distribution P(S) of each fine-grained sub-context within its scope of responsibility. j |C i (Features), where S j It is the core scenario C iFor the j-th sub-context, Features are applicable to the feature set of this expert model. Similarly, the feature with the highest probability exceeding the preset confidence threshold θ is selected. fine The subcategories are used as the final fine-grained scenarios. It should be noted that if the confidence level of the classification result is insufficient at any level, a label indicating "uncertain" or "other" can be output, or the Top-N most likely scenarios and their probabilities can be provided for downstream modules to make decisions. This achieves fine-grained, high-precision identification of the user's current scenario.

[0112] Step 401: Generate a context vector based on the user's current context and the data collected by the first sensor, wherein the context vector includes the identifier of the user's current context, the confidence level of the user's current context, the data collected by the first sensor, and context metadata.

[0113] After collecting data through the first sensor and determining the user's situation, the wearable device can generate a situation vector. The generated situation vector includes at least the identifier of the user's situation, the confidence level of the user's situation, the data collected by the first sensor, and situation metadata.

[0114] It should be noted that, in one or more embodiments of this specification, when the wearable device generates the context vector, it may first preprocess the raw time-series data collected from each sensor, including but not limited to denoising, signal enhancement, and standardization. Subsequently, the preprocessed data is divided into time windows of predetermined duration, for example, each analysis window is 5 to 10 seconds. The overlap rate between windows can be set as needed to facilitate subsequent feature extraction.

[0115] When generating a context vector, the context vector can often be a numerical vector with a fixed maximum length or a variable effective length, and its content changes as the user's context changes.

[0116] Specifically, for the identification of a user's context, one-hot encoding or a more compact multi-hot encoding can be used to represent the identified core context categories. For a large number of categories, pre-trained low-dimensional category embeddings can be used. Specific bits or values ​​can also be included to represent the hierarchical relationships or paths between coarse-grained and fine-grained contexts.

[0117] The confidence level regarding the user's situation may include only the confidence level of the user's situation, or it may include all possible situations and their corresponding confidence levels. This specification does not impose any restrictions on this.

[0118] Based on the data collected by the first sensor, the wearable sensor can determine the following information based on the identified user context: If the context is "outdoor running," this area may include: [normalized average speed, normalized average heart rate, normalized cadence, ...]. If the context is "focused reading," it may include: [normalized amplitude variance / standard deviation, normalized micro-motion amplitude, whether noise-canceling headphones are being worn, ...]. Alternatively, it may be as described in step 400, which will not be repeated here.

[0119] For contextual metadata, auxiliary metadata may be included, such as the duration of the current context, the timestamp or intraday time period code of the context, and the geographic location type, such as home, office, or outdoors.

[0120] Additionally, it should be noted that the context vector has a maximum physical length limit. When dealing with simple contexts, only the necessary parts can be filled, while the rest can be set to zero or marked as invalid, thus forming a sparse or shorter effective vector. Alternatively, it can be divided into multiple reserved segments, each corresponding to a possible feature type (such as physiological, motor, or environmental segments). Based on the current context, relevant segments are selectively calculated and filled.

[0121] Furthermore, in one or more embodiments of this specification, the specific method used by the wearable device to generate the context vector based on the collected data is not limited. It can be generated by a simple data concatenation method, or it can be generated by a preset encoder that receives the context recognition results and all available features as input, and constructs the context vector based on its internal logic, such as a lookup table, rule set, or a small learnable transformation network. This specification does not impose any limitations on this; it can be set according to actual needs.

[0122] Step 402: Send the context vector and the timestamps corresponding to the data collected by the first sensor to the terminal.

[0123] After constructing the context vector, the wearable device can send the context vector and the timestamps corresponding to the data collected by the first sensor to the terminal, so that the terminal can update the local recommendation model or recommend music to the user based on the context data.

[0124] In this embodiment, the wearable device collects user data using its own sensors to analyze the user's context, thereby generating a context vector representing the user's immediate state and situation. This allows the terminal to update its local recommendation model or provide music recommendation services based on the context vector generated by the wearable device. The terminal connected to the wearable device acts as a node providing supplementary computing resources, overcoming the challenges faced by wearable devices in running complex recommendation tasks regarding computing power, storage space, and battery life.

[0125] In addition, to save energy consumption of wearable devices, this specification also provides an embodiment, such as... Figure 5 As shown, Figure 5 This is a flowchart illustrating a data acquisition method provided in an embodiment of this specification.

[0126] Step 500: Determine the user's situation based on the data collected by the second sensor, wherein the power consumption of the second sensor is less than that of the first sensor.

[0127] Step 501: When it is determined that the user's situation has changed, wake up the first sensor to obtain the data collected by the first sensor.

[0128] The wearable device can also collect data through a second sensor to determine the user's situation. The power consumption of the second sensor is less than that of the first sensor. Then, when the user's situation changes, the corresponding first sensor is activated according to the changed situation to obtain the data collected by the first sensor.

[0129] For example, if the user's situation changes from static to dynamic, the PPG and high-sampling-rate inertial measurement unit are activated first. Alternatively, if the wearable device detects a successful Bluetooth audio output connection, or if the user, in a specific situation such as shortly after exercise, habitually picks up their connected phone and performs specific actions, the wearable device may detect this through low-power communication with the phone, predicting the user's music listening needs and pre-activating all first sensors to complete the perception and context vector generation of the current situation, reducing potential delays in recommendation responses. If the second sensor detects a drastic short-term change in the user's physiological baseline, or if the accumulated amount / duration of raw activity data since the last data processing reaches a preset privacy processing threshold, the first sensors can be activated to efficiently transform the data into a more abstract and privacy-preserving context vector. The original detailed data is then immediately removed from the wearable device's temporary cache, thus achieving a privacy-preserving and storage-optimized strategy for real-time processing and transformation.

[0130] Additionally, when a sufficient number or diversity of context vectors representing recent user experiences are detected, and a window of opportunity for device status is met (e.g., the device is charging, connected to a Wi-Fi network, and not in a period of active user interaction), the first sensor is activated to prepare for federated learning contributions. Subsequently, this may trigger the local recommendation model to be trained and updated using the stored context vectors, or prepare a context vector data package for use in subsequent federated learning contribution processes.

[0131] It should be noted that in one or more embodiments of this specification, the preset range, preset time, etc., are set according to actual needs, and this specification does not limit them, nor does it limit the determination method.

[0132] Furthermore, if the wearable device receives a recommendation request but is not connected to the terminal, it can also retrieve music from the local music library that corresponds to the user's current situation and recommend it to the user.

[0133] To implement the above embodiments, this specification also proposes a music recommendation device based on wearable devices.

[0134] like Figure 6 As shown, Figure 6 This is a schematic diagram of the structure of a music recommendation device based on a wearable device provided in the embodiments of this specification.

[0135] like Figure 6 As shown, the music recommendation device 600 is applied to a terminal and includes:

[0136] The acquisition module 601 is used to acquire contextual data sent by the wearable device, wherein the contextual data includes at least one contextual vector and a timestamp corresponding to each contextual vector;

[0137] Matching module 602 is used to obtain candidate music vectors that match the context vector;

[0138] The determination module 603 is used to determine the user behavior corresponding to the context vector in the behavior log stored locally, based on the timestamp corresponding to each context vector stored locally.

[0139] The representation module 604 is used to determine the user representation vector based on the context data, each context vector stored locally, and the corresponding user behavior;

[0140] The first recommendation module 605 is used to determine the target music to recommend to the user from the candidate music vectors based on the user representation vector.

[0141] Furthermore, in one possible implementation of the embodiments of this specification, the representation module 604 is specifically used to determine the user's first preference vector based on each of the locally stored context vectors and the corresponding user behaviors; and to fuse the first preference vector and the context data to determine the user representation vector.

[0142] Further, in one possible implementation of an embodiment of this specification, the representation module 604 is specifically used to determine candidate music tags associated with context identifiers in the context data; update the first preference vector based on the candidate music tags to obtain a second preference vector, wherein the element value in the second preference vector corresponding to the candidate music tags is greater than the element value in the first preference vector corresponding to the candidate music tags; determine a first weight corresponding to the context data and a second weight corresponding to the second preference vector; and perform weighted fusion of the second preference vector and the context data based on the first weight and the second weight to obtain the user representation vector.

[0143] Furthermore, in one possible implementation of the embodiments of this specification, the first recommendation module 605 is specifically used to input the user representation vector and each candidate music vector into the trained recommendation model to obtain the target music output by the recommendation model, wherein the recommendation model is determined based on the aggregated model parameters sent by the server.

[0144] Furthermore, in one possible implementation of an embodiment of this specification, the apparatus further includes a training module 606, configured to, when the update condition of the recommendation model is met, determine at least one user representation vector and a corresponding music vector acquired within a preset time before the current moment, wherein the music vector includes playback duration and music identifier; use each user representation vector and the corresponding music vector as training samples to update the local recommendation model to be updated; send the parameters of the updated recommendation model to the server and obtain the aggregated model parameters returned by the server; and update the recommendation model according to the aggregated model parameters.

[0145] To implement the above embodiments, this specification also proposes a music recommendation device based on wearable devices.

[0146] like Figure 7 As shown, Figure 7 This is a schematic diagram of the structure of a music recommendation device based on a wearable device provided in the embodiments of this specification.

[0147] like Figure 7 As shown, the music recommendation device 700 is applied to wearable devices and includes:

[0148] The data acquisition module 701 is used to determine the user's current situation based on the data acquired by the first sensor.

[0149] The generation module 702 is used to generate a context vector based on the user's current context and the data collected by the first sensor, wherein the context vector includes the identifier of the user's current context, the confidence level of the user's current context, the data collected by the first sensor, and context metadata.

[0150] The sending module 703 is used to send the context vector and the timestamp corresponding to the data collected by the first sensor to the terminal.

[0151] Furthermore, in one possible implementation of the embodiments of this specification, the acquisition module 701 is specifically used to determine the user's situation based on the data acquired by the second sensor, wherein the power consumption of the second sensor is less than that of the first sensor; when it is determined that the user's situation has changed, the first sensor is woken up to obtain the data acquired by the first sensor.

[0152] Furthermore, in one possible implementation of the embodiments of this specification, the device further includes a second recommendation module 704, which is used to obtain music corresponding to the user's current situation from the local music library and recommend it to the user when a recommendation request is received and the device is not connected to the terminal.

[0153] It should be noted that the foregoing explanation of the embodiment of the music recommendation method based on wearable devices also applies to the music recommendation device based on wearable devices in this embodiment, and will not be repeated here.

[0154] In the embodiments of this specification, a three-party media channel architecture of caller-called party-intelligent agent is constructed to break through the traditional two-party call mode. By introducing an intelligent agent in the call phase to communicate with the caller on behalf of the called party, the intelligent agent is used to identify abnormal calls and control the hanging up, filling the gap in the traditional blacklist and whitelist mechanism in actively intercepting abnormal calls, so as to further reduce the probability of users being harassed by abnormal calls.

[0155] To implement the above embodiments, this specification also proposes an electronic device, including: a processor and a memory communicatively connected to the processor; the memory stores computer execution instructions; the processor executes the computer execution instructions stored in the memory to implement the method provided in the foregoing embodiments.

[0156] To implement the above embodiments, this specification also proposes a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the methods provided in the foregoing embodiments.

[0157] To implement the above embodiments, this specification also proposes a computer program product, including a computer program that, when executed by a processor, implements the methods provided in the foregoing embodiments.

[0158] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in this specification comply with relevant laws and regulations and do not violate public order and good morals.

[0159] It should be noted that personal information collected from users should be used for legitimate and reasonable purposes and should not be shared or sold outside of these legitimate uses. Furthermore, such collection / sharing should only be conducted after receiving the user's informed consent, including but not limited to notifying the user to read the user agreement / user notice and sign an agreement / authorization that includes authorization of relevant user information before the user uses the function. In addition, any necessary steps must be taken to protect and safeguard access to such personal information data and ensure that others with access to personal information data comply with their privacy policies and procedures.

[0160] This specification aims to enable terminals connected to wearable devices to act as nodes providing supplemental computing resources, overcoming the challenges wearable devices face in terms of computing power, storage space, and battery life when running complex recommendation tasks. Furthermore, by using timestamps contained in locally stored context vectors, it determines user behavior recorded in the terminal's local behavior logs, generates user representations, and recommends target music to users based on these representations, thereby providing personalized music recommendations and improving the user experience.

[0161] In the foregoing descriptions of the embodiments, the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to a specific feature, structure, material, or characteristic described in connection with that embodiment or example, which is included in at least one embodiment or example of this specification. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0162] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this specification, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0163] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this specification includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which the embodiments of this specification pertain.

[0164] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-including system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and programmable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0165] It should be understood that various parts of this specification can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any of the following techniques known in the art, or a combination thereof: discrete logic circuits having logic gates for implementing logical functions on data signals; application-specific integrated circuits (ASICs) having suitable combinational logic gates; programmable gate arrays (PGAs); field-programmable gate arrays (FPGAs); etc.

[0166] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0167] Furthermore, the functional units in the various embodiments of this specification can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0168] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of this specification have been shown and described above, it is to be understood that the above embodiments are exemplary and should not be construed as limiting this specification. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of this specification.

Claims

1. A music recommendation method based on wearable devices, characterized in that, The method is applied to a terminal and includes: Acquire contextual data sent by a wearable device, wherein the contextual data includes at least one contextual vector and a timestamp corresponding to each contextual vector; Obtain candidate music vectors that match the context vector; Based on the timestamp corresponding to each of the local context vectors, determine the user behavior corresponding to the context vector in the behavior log stored locally; Based on the context data, each context vector stored locally, and the corresponding user behavior, a user representation vector is determined. Based on the user representation vector, the target music recommended for the user is determined from the candidate music vector.

2. The method as described in claim 1, characterized in that, The step of determining the user representation vector based on the context data, each context vector stored locally, and the corresponding user behavior specifically includes: Based on each of the local context vectors and the corresponding user behaviors, the user's first preference vector is determined; The first preference vector and the contextual data are fused to determine the user representation vector.

3. The method as described in claim 2, characterized in that, The step of fusing the first preference vector and the contextual data to determine the user representation vector specifically includes: Identify candidate music tags that are associated with the context identifiers in the context data; Based on the candidate music tags, the first preference vector is updated to obtain a second preference vector, wherein the element value in the second preference vector corresponding to the candidate music tags is greater than the element value in the first preference vector corresponding to the candidate music tags; Determine the first weight corresponding to the context data and the second weight corresponding to the second preference vector; Based on the first weight and the second weight, the second preference vector and the contextual data are weighted and fused to obtain the user representation vector.

4. The method as described in claim 1, characterized in that, The step of determining the target music to recommend to the user from the candidate music vector based on the user representation vector specifically includes: The user representation vector and each candidate music vector are input into the trained recommendation model to obtain the target music output by the recommendation model, wherein the recommendation model is determined based on the aggregated model parameters sent by the server.

5. The method as described in claim 4, characterized in that, The method further includes: When the update condition of the recommendation model is met, at least one user representation vector and the corresponding music vector obtained within a preset time before the current moment are determined. The music vector includes the playback duration and the music identifier. Each user representation vector and its corresponding music vector are used as training samples to update the local recommendation model to be updated. The updated recommendation model parameters are sent to the server, and the aggregated model parameters returned by the server are obtained. The recommendation model is updated based on the aggregated model parameters.

6. A music recommendation method based on wearable devices, characterized in that, The method is applied to wearable devices and includes: Based on the data collected by the first sensor, the user's current situation is determined; Based on the user's current situation and the data collected by the first sensor, a situation vector is generated, wherein the situation vector includes the identifier of the user's current situation, the confidence level of the user's current situation, the data collected by the first sensor, and situation metadata. The context vector and the timestamps corresponding to the data collected by the first sensor are sent to the terminal.

7. The method as described in claim 6, characterized in that, Before acquiring the data collected by the first sensor, the method further includes: The user's situation is determined based on the data collected by the second sensor, wherein the power consumption of the second sensor is less than that of the first sensor; When a change in the user's situation is detected, the first sensor is activated to obtain data collected by the first sensor.

8. The method as described in claim 6, characterized in that, The method further includes: Upon receiving a recommendation request and without being connected to the terminal, the system retrieves music from the local music library that corresponds to the user's current situation and recommends it to the user.

9. A music recommendation system based on wearable devices, characterized in that, The system includes a terminal and wearable devices, including: The terminal is used to perform the method described in any one of claims 1-5; The wearable device is used to perform the method described in any one of claims 6-8.

10. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method of any one of claims 1-8.

Images