Action recognition method based on end-side cloud cooperation system and end-side cloud cooperation system

Abstract

The invention discloses an action recognition method and system based on an end-side cloud collaboration system, and the method comprises the steps: carrying out the parameter calibration of an adapter corresponding to a user according to the fine adjustment action data of the user and the action real label of each action, which are transmitted by a wearable device, when a model fine adjustment demand exists, adapter fine tuning parameters corresponding to the user are determined, the parameters are sent to the wearable device, the wearable device conducts parameter updating on the action recognition model according to the adapter fine tuning parameters, and the current action data of the user are recognized through the updated model. By means of the method, resource consumption is low, meanwhile, accurate modeling and rapid adaptation of personalized action features of the user are achieved, and therefore the recognition accuracy and adaptability of the model on the individual user are improved. And meanwhile, the wearable equipment can efficiently identify the current action data of the user in real time based on the updated model, so that the practicability of the system and the user experience are enhanced.

Description

Technical Field

[0001] This application relates to the field of recognition technology, and in particular to an action recognition method and an edge-cloud collaborative system based on an edge-cloud collaborative system. Background Technology

[0002] In wearable devices based on sensors such as electromyography (EMG), accelerometers, and gyroscopes, motion recognition models often suffer from poor prediction performance due to individual user differences. Existing solutions typically achieve user adaptation by centrally updating model parameters in the cloud, but this approach has the following problems: 1) Performance drops sharply when users switch to other devices after overall model parameter changes; 2) User physical conditions or movement habits change over time, requiring frequent recalibration and impacting the user experience; 3) On-device devices are limited by computing resources and battery life, making it difficult to balance model adaptability with the burden on the device itself using traditional fine-tuning methods. Furthermore, while full parameter calibration can improve adaptability in the short term, it can easily lead to a decrease in model universality and interference from abnormal data.

[0003] The above content is only used to help understand the technical solution of this application and does not represent an admission that the above content is prior art. Summary of the Invention

[0004] The main purpose of this application is to provide an action recognition method and an edge-cloud collaborative system based on an edge-cloud collaborative system. The aim is to solve the technical problem of how to achieve seamless and continuous adaptation of the action recognition model to the user's long-term and short-term behavioral characteristics under limited edge resources, while avoiding the loss of model universality and decrease in stability caused by full parameter updates.

[0005] To achieve the above objectives, this application proposes an action recognition method based on an edge-cloud collaborative system. The edge-cloud collaborative system includes a cloud server, multiple terminal devices, and wearable devices corresponding to each terminal device. Each terminal device is communicatively connected to its corresponding wearable device, and each terminal device is communicatively connected to the cloud server.

[0006] An action recognition method based on an edge-cloud collaborative system is applied to terminal devices. The method includes:

[0007] When there is a need for model fine-tuning, acquire multiple fine-tuning action data of the user sent by the wearable device and the real action labels of each fine-tuning action data;

[0008] Based on each fine-tuning action data and the actual action label of each fine-tuning action data, the parameters of the corresponding adapter for the user are calibrated to determine the fine-tuning parameters of the corresponding adapter for the user.

[0009] The adapter fine-tuning parameters corresponding to the user are sent to the wearable device so that the wearable device can update the parameters of the action recognition model according to the adapter fine-tuning parameters corresponding to the user, and recognize the user's current action data through the updated action recognition model.

[0010] In one embodiment, the step of calibrating the parameters of the adapter corresponding to the user based on each fine-tuning action data and the actual action label of each fine-tuning action data, and determining the fine-tuning parameters of the adapter corresponding to the user, includes:

[0011] Each fine-tuning action data is input into the action recognition model to obtain the model prediction probability of each fine-tuning action data.

[0012] The first loss function is determined by calculating the cross-entropy loss based on the model prediction probability of each fine-tuned action data and the true action label of each fine-tuned action data.

[0013] The second loss function is determined by performing regularization calculations based on the current model parameters of the user's corresponding adapter.

[0014] The adapter parameters for each user are calibrated based on the first and second loss functions to determine the fine-tuning parameters of the adapter for each user.

[0015] In one embodiment, the model fine-tuning requirement includes at least one of the following:

[0016] The number of times a user can use the service is the preset number of times.

[0017] The number of times a user uses the service is not the preset number of times, and the time interval between user uses the service is greater than the preset time interval;

[0018] A model fine-tuning request was received from the wearable device.

[0019] Furthermore, to achieve the above objectives, this application also proposes a terminal device, which includes:

[0020] The acquisition module is used to acquire multiple fine-tuning action data of the user sent by the wearable device, as well as the real action labels of each fine-tuning action data, when there is a need for model fine-tuning.

[0021] The first calibration module is used to calibrate the parameters of the user's corresponding adapter based on each fine-tuning action data and the actual action label of each fine-tuning action data, and to determine the fine-tuning parameters of the user's corresponding adapter.

[0022] The first sending module is used to send the user's corresponding adapter fine-tuning parameters to the wearable device, so that the wearable device can update the parameters of the action recognition model according to the user's corresponding adapter fine-tuning parameters, and recognize the user's current action data through the updated action recognition model.

[0023] In addition, to achieve the above objectives, this application also proposes a terminal device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor. The computer program is configured to implement the steps of the action recognition method based on the edge-cloud collaborative system described above.

[0024] In addition, to achieve the above objectives, this application also proposes an action recognition method based on an edge-cloud collaborative system. The edge-cloud collaborative system includes a cloud server, multiple terminal devices and wearable devices corresponding to each terminal device. Each terminal device is communicatively connected to its corresponding wearable device and to the cloud server.

[0025] An action recognition method based on an edge-cloud collaborative system is applied to cloud servers. The method includes:

[0026] When the amount of target action data corresponding to a user exceeds the preset data amount threshold, the parameters of the adapter corresponding to the user are calibrated based on the user's target domain action dataset and source domain action dataset to determine the parameters of the target adapter corresponding to the user.

[0027] The target adapter parameters corresponding to the user are sent to the terminal device. The terminal device forwards the target adapter parameters corresponding to the user to the wearable device, so that the wearable device updates the parameters of the action recognition model according to the target adapter parameters corresponding to the user, and recognizes the user's current action data through the updated action recognition model.

[0028] In one embodiment, the step of calibrating the parameters of the user's corresponding adapter based on the user's target domain action dataset and source domain action dataset, and determining the parameters of the user's corresponding target adapter, includes:

[0029] Based on the user's target domain action dataset, determine multiple historical action data and action pseudo-labels for each historical action data;

[0030] Based on the source domain sample labels corresponding to each sample action data in the source domain action dataset and the action pseudo labels corresponding to each historical action data, random matching is performed in the source domain action dataset to determine the sample action data corresponding to each historical action data and the source domain sample labels corresponding to each historical action data.

[0031] The sample action data and historical action data corresponding to each historical action data are input into the action recognition model. The encoder in the action recognition model extracts features from the sample action data and historical action data corresponding to each historical action data to obtain the source domain intermediate vector and the target intermediate vector corresponding to each historical action data.

[0032] Based on the source domain intermediate vector, the target intermediate vector, and the source domain sample label corresponding to each historical action data, the parameters of the user's corresponding adapter are calibrated to determine the parameters of the user's corresponding target adapter.

[0033] In one embodiment, the step of calibrating the parameters of the user's corresponding adapter based on the source domain intermediate vector, the target intermediate vector, and the source domain sample labels corresponding to each historical action data, and determining the parameters of the user's corresponding target adapter includes:

[0034] Input the target intermediate vector and the source domain intermediate vector corresponding to each historical action data into the user's corresponding adapter to obtain the source domain feature vector and the target feature vector corresponding to each historical action data.

[0035] The source domain feature vectors corresponding to each historical action data are input into the output layer of the action recognition model to obtain the source domain predicted labels corresponding to each historical action data.

[0036] The target loss function is determined by calculating the source domain predicted label, source domain feature vector, target feature vector, and source domain sample label corresponding to each historical action data.

[0037] The parameters of the user's corresponding adapter are calibrated based on the target loss function to determine the target adapter parameters for the user.

[0038] In one embodiment, the step of calculating the target loss function based on the source domain predicted label corresponding to each historical action data, the source domain feature vector corresponding to each historical action data, the target feature vector corresponding to each historical action data, and the source domain sample label corresponding to each historical action data includes:

[0039] The joint distribution difference of the target feature vector and the source domain feature vector corresponding to each historical action data is calculated to determine the domain adaptation loss function.

[0040] The cross-entropy loss function is determined by calculating the cross-entropy loss function based on the source domain prediction labels and source domain sample labels corresponding to each historical action data.

[0041] Regularization is calculated based on the current model parameters of the user's corresponding adapter to determine the regularization loss function;

[0042] The target loss function is determined by calculating the loss function based on the cross-entropy loss function, the domain adaptation loss function, the regularization loss function, and the objective function weights.

[0043] In one embodiment, when the amount of target action data corresponding to a user exceeds a preset data amount threshold, the step of calibrating the parameters of the adapter corresponding to the user based on the user's target domain action dataset and source domain action dataset, and determining the parameters of the target adapter corresponding to the user, includes:

[0044] Upon receiving a model update request from a terminal device, the user whose model is being updated and the amount of target action data corresponding to that user are determined based on the model update request.

[0045] When the amount of target action data corresponding to a user exceeds a preset data threshold, data is searched in the parameter calibration queue.

[0046] If the user's identity information is not found in the parameter calibration queue, the adapter corresponding to the user is calibrated according to the user's target domain action dataset and source domain action dataset to determine the target adapter parameters corresponding to the user.

[0047] Furthermore, to achieve the above objectives, this application also proposes a cloud server, which includes:

[0048] The second calibration module is used to calibrate the parameters of the user's corresponding adapter based on the user's target domain action dataset and source domain action dataset when the amount of target action data corresponding to the user exceeds a preset data amount threshold, thereby determining the parameters of the user's corresponding target adapter.

[0049] The second sending module is used to send the target adapter parameters corresponding to the user to the terminal device. The terminal device is used to forward the target adapter parameters corresponding to the user to the wearable device, so that the wearable device can update the parameters of the action recognition model according to the target adapter parameters corresponding to the user, and recognize the user's current action data through the updated action recognition model.

[0050] In addition, to achieve the above objectives, this application also proposes a cloud server, which includes: a memory, a processor, and a computer program stored on the memory and executable on the processor, the computer program being configured to implement the steps of the action recognition method based on the edge-cloud collaborative system described above.

[0051] In addition, to achieve the above objectives, this application also proposes an action recognition method based on an edge-cloud collaborative system. The edge-cloud collaborative system includes a cloud server, multiple terminal devices and wearable devices corresponding to each terminal device. Each terminal device is communicatively connected to its corresponding wearable device and to the cloud server.

[0052] Action recognition methods based on edge-cloud collaborative systems are applied to wearable devices, and the methods include:

[0053] Upon receiving the adapter parameters corresponding to the user sent by the mobile terminal, the action recognition model is updated according to the adapter parameters corresponding to the user to obtain the updated action recognition model;

[0054] The updated action recognition model is used to identify the user's current action data, and the target action label and the label confidence of the target action label are obtained.

[0055] In one embodiment, after the step of performing action recognition on the user's current action data using the updated action recognition model to obtain the target action label and the label confidence of the target action label, the method further includes:

[0056] Compare the label confidence level with the confidence threshold;

[0057] When the label confidence level is greater than the confidence threshold, the current action data and the target action label of the current action data are sent to the mobile terminal, so that the mobile terminal sends the current action data and the target action label of the current action data to the cloud server, so that the cloud server can construct the target domain action dataset based on the current action data and the target action label of the current action data.

[0058] Furthermore, to achieve the above objectives, this application also proposes a wearable device, which includes:

[0059] The update module is used to update the action recognition model based on the adapter parameters corresponding to the user when it receives the user's corresponding adapter parameters sent by the mobile terminal, so as to obtain the updated action recognition model.

[0060] The recognition module is used to perform action recognition on the user's current action data using the updated action recognition model, and to obtain the target action label and the label confidence of the target action label for the current action data.

[0061] In addition, to achieve the above objectives, this application also proposes a wearable device, which includes: a memory, a processor, and a computer program stored on the memory and executable on the processor, the computer program being configured to implement the steps of the action recognition method based on the edge-cloud collaborative system described above.

[0062] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the steps of the action recognition method based on the edge-cloud collaborative system described above.

[0063] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the action recognition method based on the edge-cloud collaborative system described above.

[0064] In addition, to achieve the above objectives, this application also proposes an edge-cloud collaborative system, which includes a cloud server, multiple terminal devices and wearable devices corresponding to each terminal device, each terminal device is communicatively connected to its corresponding wearable device, and each terminal device is communicatively connected to the cloud server.

[0065] The cloud server is used to calibrate the parameters of the user's corresponding adapter based on the user's target domain action dataset and source domain action dataset, determine the user's corresponding target adapter parameters, and send the user's corresponding target adapter parameters to the terminal device. The terminal device is used to forward the target adapter parameters to the wearable device.

[0066] The terminal device is used to perform parameter calibration on the adapter corresponding to the user based on multiple fine-tuning action data sent by the wearable device and the actual action label of each fine-tuning action data, determine the adapter fine-tuning parameters corresponding to the user, and send the adapter fine-tuning parameters to the wearable device.

[0067] Wearable devices are used to update the parameters of the motion recognition model based on the adapter parameters corresponding to the user sent by the terminal device, and then use the updated motion adaptation model to recognize the user's current motion data.

[0068] This application provides an action recognition method based on an edge-cloud collaborative system. The method is applied to a terminal device. When model fine-tuning is required, it acquires multiple fine-tuning action data points and the actual action labels of each user's action data sent by a wearable device. Based on these fine-tuning action data points and their actual action labels, the corresponding adapter for the user is parameter-calibrated to determine the corresponding adapter fine-tuning parameters. These fine-tuning parameters are then sent to the wearable device, enabling it to update the action recognition model. The updated model then identifies the user's current action data. This approach calibrates the user's adapter parameters using only a small amount of labeled action data, achieving accurate modeling and rapid adaptation of personalized user action features with low resource consumption. This improves the accuracy and adaptability of the action recognition model for individual users, avoids the computational overhead and generality issues associated with full model updates, and maintains the model's compatibility and stability in multi-user scenarios. By sending the fine-tuning parameters to the wearable device, the device can recognize the user's current action data in real-time and efficiently based on the updated model, enhancing the system's practicality and user experience. Attached Figure Description

[0069] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0070] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0071] Figure 1 This is a flowchart illustrating an embodiment of the action recognition method based on an edge-cloud collaborative system in this application.

[0072] Figure 2 This is a schematic diagram of the overall architecture of the edge-cloud collaborative system provided in Embodiment 1 of this application;

[0073] Figure 3 A schematic diagram of the framework of the action recognition model provided in Embodiment 1 of this application;

[0074] Figure 4 A schematic diagram of the short-term fine-tuning calibration process provided in Embodiment 1 of this application;

[0075] Figure 5 This is a flowchart illustrating Embodiment 2 of the action recognition method based on the edge-cloud collaborative system of this application.

[0076] Figure 6 This is a flowchart illustrating the monitoring program provided in Embodiment 2 of this application;

[0077] Figure 7 This is a flowchart illustrating Embodiment 3 of the action recognition method based on the edge-cloud collaborative system of this application.

[0078] Figure 8 A schematic diagram of the long-term fine-tuning calibration process provided in Embodiment 3 of this application;

[0079] Figure 9 This is a flowchart illustrating Embodiment 4 of the action recognition method based on the edge-cloud collaborative system of this application.

[0080] Figure 10 This is a schematic diagram of the system architecture of the edge-cloud collaborative system of this application.

[0081] The purpose, features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0082] It should be understood that the specific embodiments described herein are merely used to explain the technical solutions of the present application and are not intended to limit the present application.

[0083] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0084] The main solution of this application embodiment is as follows: when there is a need for model fine-tuning, acquire multiple fine-tuning action data of the user sent by the wearable device and the real action label of each fine-tuning action data; perform parameter calibration on the adapter corresponding to the user based on each fine-tuning action data and the real action label of each fine-tuning action data to determine the adapter fine-tuning parameters corresponding to the user; send the adapter fine-tuning parameters corresponding to the user to the wearable device so that the wearable device updates the parameters of the action recognition model according to the adapter fine-tuning parameters corresponding to the user, and recognizes the user's current action data through the updated action recognition model.

[0085] In existing technologies, wearable devices often encounter significant individual differences when recognizing user movements (such as gestures, gait, head movements, and upper limb movements) using sensors like electromyography (EMG), accelerometers, and gyroscopes. Because different users have varying physical characteristics (such as gender, age, BMI, and muscle strength) and movement habits, it's difficult to achieve satisfactory prediction results for all users using a universal motion recognition model. Therefore, model calibration is typically required for specific users to adapt the motion recognition model to their characteristics.

[0086] Currently, most motion recognition model calibration schemes involve collecting a large amount of labeled data on the user's corresponding actions and updating the recognition model's parameters in the cloud to complete the calibration and adaptation of the user model. This approach has several problems. First, because the overall model parameters are changed, the performance of switching users to use the same model will significantly decrease. Second, as users use the model for a long time, changes in their physical condition (weight changes, changes in movement habits, age changes, etc.) will cause their movement habits to gradually shift, potentially requiring users to recalibrate periodically, thus deteriorating the user experience.

[0087] This application calibrates the parameters of the user's corresponding adapter using only a small amount of labeled action data. With low resource consumption, it achieves accurate modeling and rapid adaptation of the user's personalized action features, thereby improving the accuracy and adaptability of the action recognition model for individual users. This avoids the computational overhead and loss of universality associated with full model updates, and maintains the model's compatibility and stability in multi-user scenarios. By distributing the fine-tuning parameters to the wearable device, the device can recognize the user's current action data in real time and efficiently based on the updated model, enhancing the system's practicality and user experience.

[0088] Based on this, embodiments of this application provide an action recognition method based on an edge-cloud collaborative system, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the action recognition method based on the edge-cloud collaborative system of this application.

[0089] In this embodiment, the edge-cloud collaborative system includes a cloud server, multiple terminal devices and wearable devices corresponding to each terminal device. Each terminal device is communicatively connected to its corresponding wearable device and to the cloud server.

[0090] The action recognition method based on the edge-cloud collaborative system is applied to terminal devices. The action recognition method based on the edge-cloud collaborative system includes steps S10 to S30:

[0091] Step S10: When there is a need for model fine-tuning, obtain multiple fine-tuning action data of the user sent by the wearable device and the real action labels of each fine-tuning action data.

[0092] It should be noted that the execution entity in this embodiment is the terminal device in the edge-cloud collaborative system, which is a computing device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, and mobile phone. The edge-cloud collaborative system includes a cloud server, multiple terminal devices, and wearable devices corresponding to each terminal device. Each wearable device is bound to one or more terminal devices, and communication connections exist between each wearable device and its bound terminal device. In this embodiment, we will illustrate the example of a wearable device being bound to only one terminal device, activating the various functions of the wearable device through the bound terminal device. In the edge-cloud collaborative system, multiple wearable devices can be bound to the same terminal device; this embodiment does not restrict the binding method between wearable devices and terminal devices.

[0093] It is understandable that an edge-cloud collaborative system is a system architecture that intelligently allocates and collaboratively processes computing tasks among the edge device, edge nodes, and cloud computing center. By rationally allocating computing resources, it optimizes system performance, reduces latency, and improves efficiency and reliability.

[0094] The edge-cloud collaborative system in this embodiment is as follows: Figure 2 As shown, the edge is a wearable device. Wearable devices are used for deploying user-adapted motion models, storing user motion data, and recognizing user actions through motion recognition models. Wearable devices include, but are not limited to, wristbands, rings, watches, head-mounted devices, and other devices with data acquisition functions.

[0095] The terminal device is an edge computing node in the edge-cloud collaborative system, used for short-term and rapid calibration of the user's corresponding action recognition model and user action data storage. The memory card / hard disk in the terminal device stores the current user's basic information, some historical data of the current user, some parameters of the base model encoder, and some parameters of the general model adapter.

[0096] In the edge-cloud collaborative system, the cloud server serves as the cloud computing center. It is used for long-term calibration of user models and batch calibration and updates. The cloud server interacts with a cloud-based database, which stores basic information about all users, all historical action data for all users, and all historical model parameters for all users. The combination of the cloud server and the cloud-based database provides stronger computing power and storage resources, enabling the recording of user information and the storage of user data.

[0097] In the edge-cloud collaborative system, the cloud server and terminal devices can communicate bidirectionally via public networks, dedicated lines, or other communication methods. When the terminal device is connected to the network, it can automatically upload historical user data stored at the edge to the cloud server and download the latest model obtained from the long-term calibration process of the user from the cloud to the terminal device. The terminal device and its corresponding wearable device can communicate bidirectionally via Bluetooth Low Energy (BLE), Wi-Fi, or other communication methods. When the connection is successful, the data collected in the wearable device is uploaded to the terminal device, and the terminal device simultaneously transmits the new user model, after short-term or long-term fine-tuning, to the wearable device for updating. Based on this embodiment, the edge-cloud collaborative system makes reasonable use of the resources of each part of the architecture and adopts a collaborative algorithm to achieve model calibration, making the adaptation process between the user and the model more seamless and natural.

[0098] It should be understood that each wearable device contains a microprocessor and a small storage unit, RAM (Random Access Memory). The microprocessor typically only performs inference of the action model and data preprocessing, and cannot perform fine-tuning and training tasks. The RAM stores a small amount of filtered historical action data (pseudo-labels) for the current user, short-term rapid calibration data (labeled), some parameters of the base model encoder, and some parameters of the user-specific adapter. When connected to the corresponding terminal device, the data stored in RAM is uploaded to the terminal device, and the stored records in RAM are cleared. Typically, the RAM storage space is <10MB, capable of storing approximately 1000 historical data entries. In addition, the RAM also stores user-adapted action recognition model parameters (encoder and user-specific adapter) for action recognition tasks performed by the microprocessor. Each wearable device needs to be connected and paired with a corresponding terminal device during use. When the corresponding terminal device communicates with the cloud server, it can transmit user data, fine-tuning requests, and other information to the cloud server and database for unified storage and scheduling.

[0099] Understandably, the mobile terminal serves as the primary carrier for executing functions after recognizing user actions and movements. It connects and pairs with the wearable device via Bluetooth / Wi-Fi, and connects to the cloud server to access cloud services when connected to the network. Simultaneously, the terminal device has a powerful CPU capable of performing simple model fine-tuning tasks. Therefore, short-term, rapid fine-tuning tasks can be executed directly on its CPU, quickly obtaining the calibrated adapter parameters and sending them back to the wearable device (without accessing cloud services). Furthermore, the terminal device has substantial storage capacity (GB-level) to store more historical user data, even historical data from different users. When connected to the network, this data can be uploaded to a cloud database for long-term calibration services for each user on the cloud server.

[0100] In its implementation, the cloud server receives information in parallel from different edge terminal devices, including basic user information and historical user data. The cloud server periodically updates the user model through a listening program. When a user sends a model update request via an edge terminal device, the latest user model is transmitted to the terminal device, which then deploys it to the wearable device. Furthermore, the cloud maintains a database to store the user's historical data (pseudo-tags), tagged data stored during rapid calibration, and all historically updated user models.

[0101] It should be noted that before model deployment, the cloud server acquires multiple sample action data from different users, along with the corresponding ground truth labels for each sample action data. The cloud server then preprocesses and embeds this data to obtain the action feature representations for each sample action data. Preprocessing includes, but is not limited to, denoising, outlier handling, missing value repair, and data standardization. The multiple sample action data from different users, along with the corresponding ground truth labels, can be obtained through data collection from various types of sensors on wearable devices. Different preprocessing and embedding strategies are typically required for different types of data collected by different sensors.

[0102] Understandably, the cloud server pre-trains the initial model using pre-processed sample action data and the corresponding real labels for each sample action data, thereby obtaining the basic recognition model. In this embodiment, the framework of the initial model is as follows: Figure 3 As shown, the initial model includes an embedding layer, an input layer, an encoder, an adapter, and an output layer. Different embedding strategies are typically required for different data types collected by different sensors. The data embedding layer transforms different action data types into a unified data format, which is then linearly mapped by the input layer to obtain the input vector V1. The input vector is then encoded by the encoder, resulting in an intermediate vector V2. V2 is then input into the adapter network structure. During the pre-training phase, the parameters in the encoder and adapter are updated as a whole. After training, a basic recognition model is obtained. This basic recognition model is a universal base model, where the parameters of the adapters for all users are consistent. The basic recognition model is used to identify user action types, which include, but are not limited to, waving, clenching fists, clicking, and engaging, among other action types.

[0103] In the specific implementation, after obtaining the basic recognition model, all parameters in the encoder are fixed and do not participate in the gradient descent update process for model fine-tuning. The encoder structure can be a Long Short-Term Memory (LSTM) network, a Transformer, or other backbone networks. However, in this embodiment, a... Figure 3The given structure is used as an example. The encoder consists of N sets of convolutional neural network (CNN) modules. Each CNN module sequentially performs convolution, pooling, batch normalization, and the activation function RULE. The adapter in this embodiment is implemented using a fully connected network with a bottleneck structure. A bottleneck structure refers to inserting one or more layers into the network whose dimensions are significantly lower than the dimensions of the input and output layers, thus forming a "bottleneck." Besides the above-described adapter structure, other structures can also be used; this embodiment does not impose any restrictions on this.

[0104] It's important to note that when users are using wearable devices for motion recognition, during the short-term and long-term calibration and fine-tuning phases, the parameters in the encoder are fixed, while the adapter parameters for each specific user are adjusted. The cloud database stores each user's adapter parameters, which are then downloaded and deployed when the wearable device needs to update its model. Adapter parameters refer to the model parameters of the adapter. Initially, each user's adapter parameters are those of the adapter in the basic recognition model. After short-term and long-term fine-tuning using the user's motion data, each user will have their own unique adapter.

[0105] Understandably, when users use wearable devices for motion recognition, individual differences mean that the same recognition model parameters cannot be universally applied across different users. Therefore, user-specific calibration is necessary. For a new user, a user who has not used the device for an extended period, or other situations where model fine-tuning is required, the wearable device needs to collect several motion data points from that user for rapid fine-tuning. This process, known as short-term calibration, accurately learns the user's motion characteristics, allowing the user to quickly access an accurate motion recognition model.

[0106] In practice, as users use the platform over a long period, they accumulate more and more historical data, which includes their usage habits. As time goes on, when a user's physical condition or usage habits undergo slight changes, the system can learn from these latent changes in user characteristics through a fine-tuning process using historical data in the cloud. This process, known as long-term calibration, allows the model parameters to be subtly adjusted without the user's awareness, making the action recognition model increasingly aligned with the user's data characteristics and usage habits.

[0107] It should be noted that when the mobile terminal detects a need for model fine-tuning, indicating a short-term, rapid adjustment is required, it will send an action recognition task to the wearable device. The wearable device will then collect a small amount of action data with real labels based on the action recognition task. Specifically, this can be achieved as follows: When the wearable device receives the action recognition task, it sends an action command. The user performs the corresponding action according to the command, and the wearable device collects data on the user's actions, thereby obtaining the raw sensor data for each action.

[0108] Understandably, after collecting the raw sensor data corresponding to each action command, the wearable device will send each action command and its corresponding raw sensor data to the mobile terminal. In this embodiment, each action command includes a motion authenticity label, which refers to the specific action type. Fine-tuning motion data refers to the raw sensor data corresponding to each action command.

[0109] In one feasible implementation, the model fine-tuning requirement includes at least one of the following: the number of times the user uses the device is a preset number of times; the number of times the user uses the device is not a preset number of times and the time interval between user uses the device is greater than a preset time interval; or a model fine-tuning request is received from the wearable device.

[0110] It should be noted that in this embodiment, when a user uses the wearable device, they will log in on the wearable device to obtain basic user information, including but not limited to name, age, and identification identifier. The wearable device will send the obtained basic information to the mobile terminal. The mobile terminal will identify the current user based on the basic information and send a request to the cloud server to check if the user's basic information exists in the cloud database. If the cloud server reports that the user's basic information does not exist previously, it indicates that the user is a new user, and the user's usage count is a preset usage count. At this point, it is determined that there is a need for model fine-tuning. The user usage count refers to the number of times the user has used the device in the past. In this embodiment, the preset usage count can be set to 0, or it can be set to other values ​​as needed. This embodiment does not impose any restrictions on this.

[0111] Understandably, if the cloud database reports that the user's basic information already exists, it indicates that the user is not a new user, and the user's usage count is not the preset usage count. In this case, the mobile terminal needs to send a request to the cloud server to check if the user's last usage time exists in the cloud database. Based on the user's last usage time and the current time, the user's usage time interval is determined. If the user's usage time interval is greater than the preset time interval, it indicates that the user has not used the service for a long time, and a model fine-tuning requirement is identified. In this embodiment, the preset time interval can be set to one month, or it can be set to other values ​​as needed; this embodiment does not impose any restrictions on this.

[0112] In practice, if the number of times a user uses the model is not the preset number of times and the time interval between user uses the model is less than or equal to the preset time interval, it means that there is no need for model fine-tuning, and no short-term rapid fine-tuning will be performed in this case.

[0113] It should be noted that when a user uses a wearable device, the wearable device can proactively initiate a model update request to the mobile terminal, at which point the mobile terminal determines that a model fine-tuning requirement exists; or, the user can initiate a model update request to the mobile terminal through the wearable device, at which point the mobile terminal determines that a model fine-tuning requirement exists. In this embodiment, in addition to the above methods, a model fine-tuning requirement can also be determined when other situations are detected.

[0114] Step S20: Based on each fine-tuning action data and the actual action label of each fine-tuning action data, perform parameter calibration on the adapter corresponding to the user to determine the fine-tuning parameters of the adapter corresponding to the user.

[0115] It should be noted that each user corresponds to an adapter. If the number of times the current user has used the device is the preset number of times, it means that the adapter corresponding to the user has not been fine-tuned or calibrated for a long or short period of time. In this case, the parameters of the adapter corresponding to the user are consistent with the parameters of the adapter in the basic recognition model.

[0116] Understandably, if the current user's usage count is not the preset limit, the mobile terminal will obtain the time of the most recent short-term fine-tuning of the adapter corresponding to that user, and also obtain the time of the most recent long-term fine-tuning of the adapter corresponding to that user from the cloud server. If the short-term fine-tuning time is the most recent, the mobile terminal will use the adapter after the previous short-term fine-tuning as the user's adapter; if the long-term fine-tuning time is the most recent, the mobile terminal will use the adapter after the previous long-term fine-tuning as the user's adapter.

[0117] In the specific implementation, the mobile terminal calibrates the parameters of the adapter corresponding to the user based on each fine-tuning action data and the corresponding real action label. During the parameter calibration process, all parameters of the encoder in the action recognition model are fixed, and the parameters in the adapter are updated using cross-entropy loss gradient. The adapter parameters obtained after short-term fine-tuning on the mobile terminal are the adapter fine-tuning parameters corresponding to the user.

[0118] It should be noted that each user corresponds to an action recognition model. When the number of times the current user has used the device is the preset number of times, the action recognition model is the basic recognition model. When the number of times the current user has used the device is not the preset number of times, the action recognition model consists of the user's corresponding adapter, the embedding layer, the input layer, the encoder, and the output layer in the basic recognition model.

[0119] In one feasible implementation, step S20 may further include steps A11 to A14:

[0120] Step A11: Input each fine-tuning action data into the action recognition model to obtain the model prediction probability of each fine-tuning action data.

[0121] It should be noted that the mobile terminal inputs each fine-tuning action data into the user's corresponding action recognition model. The action recognition model then outputs the action label corresponding to each fine-tuning action data and the prediction probability corresponding to that action label. In this embodiment, the model prediction probability of each fine-tuning action data refers to the prediction probability of the action label corresponding to each fine-tuning action.

[0122] Step A12: Calculate the cross-entropy loss based on the model prediction probability of each fine-tuned action data and the true action label of each fine-tuned action data, and determine the first loss function.

[0123] It should be noted that, based on the true action labels of each fine-tuning action data, the true label probability corresponding to all action types under each fine-tuning action data can be clearly determined. In this embodiment, the true label probability of the action type corresponding to the true action label is 1, and the true label probability of other action types is set to 0. Other settings are also possible, and this embodiment does not limit them.

[0124] The first loss function is obtained by calculating the cross-entropy loss based on the model prediction probability of each fine-tuned action data and the true action label of each fine-tuned action data. In this embodiment, the calculation formula of the first loss function is as follows: Where C represents the number of action types, P i y represents the model prediction probability that the model predicts the i-th action type. i This represents the true label probability of the i-th action type.

[0125] Step A13: Perform regularization calculations based on the current model parameters of the user's corresponding adapter to determine the second loss function.

[0126] It should be noted that the current model parameters refer to the parameters of the adapter corresponding to the user. Regularization is performed using the current model parameters of the adapter corresponding to the current user and a preset regularization coefficient to obtain the second loss function. In this embodiment, the calculation formula for the second loss function is as follows: Where λ is a preset regularization coefficient used to control the degree of influence of the regularization term; w t This refers to the current model parameters.

[0127] Step A14: Perform parameter calibration on the adapter corresponding to the user based on the first loss function and the second loss function to determine the fine-tuning parameters of the adapter corresponding to the user.

[0128] It should be noted that the total loss is calculated based on the first loss function and the second loss function: Total Loss L S = L1 + L2. Based on the total loss, calculate the gradient of each parameter in the user's adapter relative to the total loss. Based on the calculated gradients, use an optimization algorithm (e.g., SGD, Adam) to update the parameters in the user's adapter. The goal is to find a set of parameters that minimizes the total loss, meaning the model's prediction is as close to the true value as possible. Repeat steps A11 to A14 until a preset stopping condition is met, at which point the user's adapter parameters are adjusted to their optimal state. The resulting adapter parameters are the user's adapter fine-tuning parameters. In this embodiment, the preset stopping condition includes, but is not limited to: the total loss no longer significantly decreases; reaching the maximum number of iterations.

[0129] It is understandable that the process of short-term fine-tuning and calibration of mobile terminals is as follows: Figure 4 As shown, multiple fine-tuned action data are input into the action recognition model to obtain the action label of each fine-tuned action data and its corresponding model prediction probability. In the short-term fine-tuning process, all parameters in the encoder are fixed, and the parameters in the adapter are updated by gradient through cross-entropy loss. At the same time, regularization is performed by setting a certain L1 norm to reduce the risk of overfitting due to few samples.

[0130] Step S30: Send the user's corresponding adapter fine-tuning parameters to the wearable device so that the wearable device can update the parameters of the action recognition model according to the user's corresponding adapter fine-tuning parameters, and recognize the user's current action data through the updated action recognition model.

[0131] It should be noted that after the mobile terminal obtains the adapter fine-tuning parameters corresponding to the user, it directly sends them back to the wearable device on the device side, without accessing cloud services. At this time, the wearable device updates the parameters of the user's corresponding adapter in the user's corresponding action recognition model according to the user's corresponding adapter fine-tuning parameters, thereby obtaining the updated action recognition model for the user.

[0132] Understandably, wearable devices receive current motion data from users through sensors, such as electromyography, inertial measurement units, photoplethysmography, and electroencephalography. They preprocess the current motion data, call the updated motion recognition model to determine the type of the preprocessed current motion data, and thus obtain the motion type corresponding to the current motion data.

[0133] This embodiment provides an action recognition method based on an edge-cloud collaborative system, applied to terminal devices. When model fine-tuning is required, it acquires multiple fine-tuning action data of the user sent by a wearable device, along with the actual action labels for each fine-tuning action data. Based on the fine-tuning action data and their actual action labels, it calibrates the parameters of the user's corresponding adapter to determine the corresponding adapter fine-tuning parameters. These fine-tuning parameters are then sent to the wearable device, enabling it to update the action recognition model. The updated model then identifies the user's current action data. This method calibrates the user's corresponding adapter parameters using only a small amount of labeled action data, achieving accurate modeling and rapid adaptation of personalized user action features with low resource consumption. This improves the accuracy and adaptability of the action recognition model for individual users, avoids the computational overhead and generality issues associated with full model updates, and maintains the model's compatibility and stability in multi-user scenarios. By sending the fine-tuning parameters to the wearable device, the device can recognize the user's current action data in real-time and efficiently based on the updated model, enhancing the system's practicality and user experience.

[0134] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to that in Embodiment 1 above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 5 , refer to Figure 5 , Figure 5 This is a flowchart illustrating the second embodiment of the action recognition method based on the edge-cloud collaborative system of this application.

[0135] In this embodiment, the edge-cloud collaborative system includes a cloud server, multiple terminal devices and wearable devices corresponding to each terminal device. Each terminal device is communicatively connected to its corresponding wearable device and to the cloud server.

[0136] The action recognition method based on the edge-cloud collaborative system is applied to cloud servers. The action recognition method based on the edge-cloud collaborative system includes steps S01 to S02:

[0137] Step S01: When the amount of target action data corresponding to the user is greater than the preset data amount threshold, the parameters of the adapter corresponding to the user are calibrated according to the user's target domain action dataset and source domain action dataset to determine the parameters of the target adapter corresponding to the user.

[0138] It should be noted that the execution entity in this embodiment is a cloud server in the edge-cloud collaborative system, which is a computing device deployed in a remote data center and has powerful data processing capabilities, high-speed network communication functions, and a distributed program running environment.

[0139] Understandably, during user interaction with a wearable device, the device performs action recognition on the user's real-time behavior data based on a continuously updated action recognition model, determining the corresponding action label and its confidence level. The wearable device compares the confidence level of this label with a confidence threshold, saving behavior data with a confidence level greater than the threshold and its corresponding action label. When the wearable device connects to a mobile terminal, the behavior data with a confidence level greater than the threshold and its corresponding action label are uploaded to the mobile terminal. When the mobile terminal communicates with the cloud server, the mobile terminal forwards the behavior data with a confidence level greater than the threshold and its corresponding action label to the wearable device. In this embodiment, the confidence threshold can be set to 90%, or it can be adjusted as needed; this embodiment does not impose any restrictions on this. For example, if a set of behavior data is judged as a fist-clenching action with a 90% probability after being processed by the action recognition model, then the action label for this action data is "fist-clenching."

[0140] In its implementation, when the cloud server receives user behavior data uploaded by a mobile terminal with a confidence level greater than a confidence threshold, along with the corresponding action tags, it stores this data in the cloud database. The action tags corresponding to this behavior data serve as pseudo-tags for that behavior data. The cloud server then constructs a target domain action dataset for the user based on multiple behavior data points uploaded by the mobile terminal for the same user and the pseudo-tags for each behavior data point.

[0141] It should be noted that the user's target domain action dataset is used for long-term calibration of the corresponding adapter, and the action data contained therein has not been involved in long-term fine-tuning; the target action data volume refers to the amount of action data contained in the user's corresponding target domain action dataset. The source domain action dataset contains a large number of sample action data from different users, along with the corresponding ground truth labels for each sample action data.

[0142] Understandably, the cloud server has a monitoring program that periodically executes long-term fine-tuning tasks. This program periodically queries the cloud database for the target action data volume corresponding to all users. When the target action data volume for a user exceeds a preset data volume threshold, the cloud server combines the user's target domain action dataset and source domain action dataset to perform a long-term fine-tuning task, calibrating the parameters of the user's corresponding adapter. The adapter parameters obtained after the cloud server performs this long-term fine-tuning are the user's corresponding target adapter parameters.

[0143] In the specific implementation, long-term fine-tuning adopts a loss function that combines JMMD (Joint Maximum Mean Discrepancy) and cross-entropy. While ensuring that the model does not produce excessive bias, it guides the model parameters to shift towards the user's historical data features, thereby obtaining an adapter that adapts to the user.

[0144] It should be noted that each user corresponds to an adapter. If the number of times the current user has used the device is the preset number of times, it means that the adapter corresponding to the user has not been fine-tuned or calibrated for a long or short period of time. In this case, the parameters of the adapter corresponding to the user are consistent with the parameters of the adapter in the basic recognition model.

[0145] Understandably, if a user's current usage count is not within the preset limit, the cloud server will obtain the time of the last short-term fine-tuning of the adapter corresponding to that user, and the time of the last long-term fine-tuning of the adapter corresponding to that user. If the short-term fine-tuning time is closest to the current time, the cloud server will use the adapter after the last short-term fine-tuning as the user's adapter; if the long-term fine-tuning time is closest to the current time, the cloud server will use the adapter after the last long-term fine-tuning as the user's adapter.

[0146] In one feasible implementation, step S01 may include steps B11 to B13:

[0147] Step B11: Upon receiving a model update request from the terminal device, determine the user whose model is being updated and the amount of target action data corresponding to that user based on the model update request.

[0148] It should be noted that, in addition to the cloud server automatically executing long-term fine-tuning tasks through a listening program, users can also actively initiate long-term fine-tuning requests to manually start the task. Users can initiate model update requests through wearable devices, which then send the request to a mobile terminal, which forwards it to the cloud server. Alternatively, users can initiate model update requests through their mobile terminals, which then forward the request to the cloud server. In this embodiment, the model update request is used to manually initiate the long-term fine-tuning task.

[0149] Understandably, when the cloud server receives a model update request from a mobile terminal, it analyzes the request to determine the user initiating the request and the corresponding target action data volume. In this embodiment, the user initiating the model update request refers to the user who initiated the request.

[0150] Step B12: When the amount of target action data corresponding to the user is greater than the preset data amount threshold, perform data search in the parameter calibration queue.

[0151] It should be noted that the parameter calibration queue includes a queue to be fine-tuned and a queue to be fine-tuned, both maintained by the cloud server. The cloud server fine-tunes adapters for multiple users based on its computing power; therefore, the queue to be fine-tuned includes user identity information (e.g., username, user identifier) ​​for the users currently being fine-tuned. The queue to be fine-tuned includes user identity information for users about to undergo fine-tuning.

[0152] Understandably, when the amount of target action data corresponding to a user exceeds a preset data volume threshold, the system checks the parameter calibration queue for the user's identity information. When the amount of target action data corresponding to a user is less than or equal to the preset data volume threshold N, the system returns a result indicating "no fine-tuning required".

[0153] Step B13: If the user's identity information is not found in the parameter calibration queue, perform parameter calibration on the user's corresponding adapter based on the user's target domain action dataset and source domain action dataset to determine the user's target adapter parameters.

[0154] It should be noted that if the user's identity information is not present in the parameter calibration queue, the cloud server will add the user's identity information to the head of the fine-tuning queue. The cloud server will execute long-term fine-tuning tasks according to the order of the fine-tuning queue. The cloud server will perform parameter calibration on the user's corresponding adapter according to the user's target domain action dataset and source domain action dataset in the queue to determine the target adapter parameters for the user.

[0155] It is understandable that the process of a cloud server performing a long-term fine-tuning task is as follows: Figure 6As shown, the cloud server has a listening program. Listening program 1 periodically executes automatic fine-tuning tasks. After entering the automatic fine-tuning process, the program performs the following steps: Step 1: Periodically query the database for the amount of target action data N that has not been fine-tuned for all users. If a user's target action data amount is greater than a certain threshold (e.g., a user has accumulated more than 1000 data entries that have not been fine-tuned), then proceed to Step 2: Insert the username into the queue to be fine-tuned. The cloud server maintains a unified fine-tuning queue, which includes a queue to be fine-tuned and a queue to be fine-tuned. The action recognition models of X users are fine-tuned in parallel according to the server's computing power. If X=1, that is, the data of 1 user is in the fine-tuning state, and the other users are in the queue to be fine-tuned. When the user in the fine-tuning process completes the fine-tuning calibration task, the database will update the model parameters of that user, and at the same time, the next user will be selected from the queue for the fine-tuning calibration task.

[0156] In addition to the automatic fine-tuning mode, users can also actively submit a fine-tuning request to manually initiate a long-term fine-tuning task. When the listening program 2 in the cloud server receives a fine-tuning request initiated by the user, it will enter the manual fine-tuning process. The program will execute the following steps: Step 3: Check the amount of target action data (i.e., new data) N that the user has not participated in fine-tuning from the database, and determine whether the new data has accumulated to a certain amount. If the amount of target action data is less than a certain threshold, it means that the amount of new data is insufficient to execute the long-term calibration task, and proceed to Step 4: Return the result of "no fine-tuning required". If the amount of target action data is greater than a certain threshold, proceed to Step 5: If the username is currently being fine-tuned or is already in the fine-tuning queue, the fine-tuned result will be returned directly after the fine-tuning is completed; if the username has not been fine-tuned and is not in the fine-tuning queue, the username will be inserted at the head of the fine-tuning queue.

[0157] Both of the above fine-tuning modes insert user identity information into the fine-tuning queue. A monitoring program maintains the fine-tuning queue to ensure that there is no redundancy or conflict between users currently fine-tuning and those awaiting fine-tuning. During the fine-tuning calibration task, after a user completes their fine-tuning calibration task, the fine-tuning program on the cloud server executes the following steps: Step 6: Obtain the username at the head of the queue awaiting fine-tuning; Step 7: Obtain the user's data to be fine-tuned (i.e., the target domain action dataset) based on the username; Step 8: Execute a long-term fine-tuning calibration task based on the data to be fine-tuned; Step 9: After the task is completed, update the model parameters of the adapter corresponding to the user, update the user's relevant information in the database, and mark the user's target domain action dataset in the database as having participated in the fine-tuning calibration task.

[0158] Step S02: Send the target adapter parameters corresponding to the user to the terminal device. The terminal device forwards the target adapter parameters corresponding to the user to the wearable device so that the wearable device updates the parameters of the action recognition model according to the target adapter parameters corresponding to the user, and recognizes the user's current action data through the updated action recognition model.

[0159] It should be noted that after obtaining the target adapter parameters corresponding to the user, the cloud server will proactively send the target adapter parameters to the terminal device. Alternatively, it can send the target adapter parameters to the terminal device when it receives a parameter update request from the mobile terminal.

[0160] Understandably, after receiving the target adapter parameters corresponding to the user from the cloud server, the mobile terminal sends them back to the wearable device. At this point, the wearable device updates the parameters of the user's corresponding adapter within the user's action recognition model based on the target adapter parameters, thus obtaining the updated action recognition model for the user.

[0161] Understandably, wearable devices receive current motion data from users through sensors, such as electromyography, inertial measurement units, photoplethysmography, and electroencephalography. They preprocess the current motion data, call the updated motion recognition model to determine the type of the preprocessed current motion data, and thus obtain the motion type corresponding to the current motion data.

[0162] This embodiment provides an action recognition method based on an edge-cloud collaborative system. When the amount of target action data corresponding to a user exceeds a preset data threshold, the method calibrates the parameters of the user's corresponding adapter based on the user's target domain action dataset and source domain action dataset to determine the user's target adapter parameters. These target adapter parameters are then sent to a terminal device, which forwards them to a wearable device. The wearable device updates its action recognition model based on these parameters, and the updated model then identifies the user's current action data. Through this method, long-term fine-tuning and calibration are performed on a cloud server to generate personalized adapter parameters, which are then distributed to the terminal and wearable devices. This allows the action recognition model to adaptively update based on the user's actual behavioral characteristics. This allows for model updates without the user's awareness, effectively improving the model's recognition accuracy for specific users, enhancing its generalization and personalization capabilities, while reducing the computational burden on the terminal device. This achieves efficient model optimization and deployment in an edge-cloud collaborative environment.

[0163] Based on the first and / or second embodiments of this application, in the third embodiment of this application, the content that is the same as or similar to that in embodiments one and two above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 7 Step S01, the action recognition method based on the edge-cloud collaborative system includes steps S11 to S14:

[0164] Step S11: Determine multiple historical action data and action pseudo-labels for each historical action data based on the user's target domain action dataset.

[0165] It should be noted that when the cloud server receives user behavior data with a confidence level greater than the confidence threshold and corresponding action tags uploaded by the mobile terminal, it stores it in the cloud database. The action tags corresponding to this behavior data serve as pseudo-tags for that behavior data. The cloud server constructs a target domain action dataset for that user based on multiple behavior data points uploaded by the mobile terminal for the same user and the pseudo-tags for each behavior data point. In this embodiment, historical action data refers to action data existing in the target action dataset.

[0166] Step S12: Based on the source domain sample labels corresponding to each sample action data in the source domain action dataset and the action pseudo labels corresponding to each historical action data, perform random matching in the source domain action dataset to determine the sample action data corresponding to each historical action data and the source domain sample labels corresponding to each historical action data.

[0167] It's important to note that the source domain sample label refers to the actual label of the sample action data. When the cloud server executes a long-term fine-tuning task, for each historical action data point: it combines the pseudo-label of the historical action data with the source domain sample label corresponding to each sample action data point. From the multiple sample action data points existing in the source domain action dataset, it randomly selects a sample action data point whose source domain sample label matches the pseudo-label of the historical action data point. This sample action data is then used as the corresponding sample action data point for that historical action data point. In other words, each historical action data point dt has a corresponding sample action data point ds, and the source domain sample label of this sample action data point is the source domain sample label corresponding to that historical action data point.

[0168] Step S13: Input the sample action data and historical action data corresponding to each historical action data into the action recognition model. The encoder in the action recognition model extracts features from the sample action data and historical action data corresponding to each historical action data to obtain the source domain intermediate vector and the target intermediate vector corresponding to each historical action data.

[0169] It should be noted that when the cloud server performs a long-term fine-tuning task, it simultaneously inputs historical action data and its corresponding sample action data into the action recognition model. The encoder in the action recognition model, using shared parameters, extracts features from both the sample action data and the historical action data, obtaining the intermediate vectors of the historical action data and the sample action data. In this embodiment, the target intermediate vector corresponding to the historical action data refers to the intermediate vector of the historical action data output by the encoder of the action recognition model; the source domain intermediate vector corresponding to the historical action data refers to the intermediate vector of the sample action data corresponding to the historical action data output by the encoder of the action recognition model.

[0170] It is understandable that the user-specific action recognition model consists of a user-specific adapter, an embedding layer, an input layer, an encoder, and an output layer in the basic recognition model.

[0171] Step S14: Based on the source domain intermediate vector, the target intermediate vector, and the source domain sample label corresponding to each historical action data, perform parameter calibration on the user's corresponding adapter to determine the user's corresponding target adapter parameters.

[0172] It should be noted that for each historical action data: after obtaining the corresponding source domain intermediate vector and target intermediate vector, the cloud server will input the source domain intermediate vector and target intermediate vector into the adapter corresponding to the user with shared parameters, thereby obtaining a set of intermediate feature vectors corresponding to the source domain intermediate vector and a set of intermediate feature vectors corresponding to the target intermediate vector.

[0173] Understandably, during long-term fine-tuning, cloud servers combine the intermediate feature vectors corresponding to the source domain intermediate vectors of each historical action data and the intermediate feature vectors corresponding to the target intermediate vectors of each historical action data. They then use a loss function combining JMMD and cross-entropy to calibrate the parameters. This ensures that the model does not produce excessive deviations while guiding the model parameters to shift towards the features of the user's historical data, thereby obtaining target adapter parameters that are adapted to the user.

[0174] In one feasible implementation, step S14 may include steps C11 to C14:

[0175] Step C11: Input the target intermediate vector and the source domain intermediate vector corresponding to each historical action data into the user's corresponding adapter to obtain the source domain feature vector and the target feature vector corresponding to each historical action data.

[0176] It should be noted that, for each historical action data point: after obtaining the corresponding source domain intermediate vector and target intermediate vector, the cloud server will input the source domain intermediate vector and target intermediate vector into the adapter corresponding to the user with shared parameters, thereby obtaining a set of intermediate feature vectors corresponding to the source domain intermediate vector and a set of intermediate feature vectors corresponding to the target intermediate vector. In this embodiment, the source domain feature vector corresponding to each historical action data point refers to the intermediate feature vector corresponding to the source domain intermediate vector of each historical action data point; the target feature vector corresponding to each historical action data point refers to the intermediate feature vector corresponding to the target intermediate vector of each historical action data point.

[0177] Step C12: Input the source domain feature vector corresponding to each historical action data into the output layer of the action recognition model to obtain the source domain predicted label corresponding to each historical action data.

[0178] It should be noted that the source domain feature vectors corresponding to each historical action data are input into the output layer of the action recognition model. The output layer maps the source domain feature vectors to specific action categories, thereby obtaining the source domain predicted labels corresponding to each historical action data.

[0179] Step C13: Calculate the loss function based on the source domain predicted label, source domain feature vector, target feature vector, and source domain sample label corresponding to each historical action data, and determine the target loss function.

[0180] It should be noted that, for each historical action data point: cross-entropy loss is calculated based on the source domain predicted label and the source domain sample label; joint distribution difference is calculated based on the target feature vector and the source domain feature vector to achieve unsupervised domain adaptation; and regularization is calculated based on the parameters of the user's corresponding adapter. Finally, the results of the cross-entropy loss calculation, the joint distribution difference calculation, and the regularization calculation for multiple historical action data points are summarized to obtain the target loss function corresponding to long-term fine-tuning calibration.

[0181] Step C14: Perform parameter calibration on the user's corresponding adapter according to the target loss function to determine the target adapter parameters for the user.

[0182] It should be noted that, based on the calculated target loss function, backpropagation is performed on the adapter corresponding to the user to calculate the gradient of each parameter with respect to the total loss. Optimization algorithms (such as SGD, Adam, etc.) are then used to update the parameters to minimize the total loss, making the model's predictions closer to the true values. Steps C11 to C14 are repeated until a preset stopping condition is met. At this point, the adapter parameters are the fine-tuned results for that user, yielding the target adapter parameters for that user. In this embodiment, the preset stopping conditions include, but are not limited to: the total loss no longer significantly decreasing; and reaching the maximum number of iterations.

[0183] In one feasible implementation, step C13 may include steps D11 to D14:

[0184] Step D11: Perform joint distribution difference calculation on the target feature vector corresponding to each historical action data and the source domain feature vector corresponding to each historical action data to determine the domain adaptation loss function.

[0185] It should be noted that, for each historical action data point: to measure the difference between the target feature vector and the source domain feature vector in the joint feature-label distribution, this embodiment uses the joint maximum mean difference method for measurement, thereby obtaining the domain adaptation loss function. The domain adaptation loss function aligns the feature distribution extracted by the model in the target domain with that in the source domain, reducing the distribution difference between the source and target domains and minimizing domain differences.

[0186] It is understandable that the domain adaptation loss function is calculated as follows: k(·) represents the kernel function, which is a Gaussian kernel function in this embodiment. S represents the source domain action dataset, T represents the target domain action dataset, and n s n t These represent the number of action data points in the source domain action dataset and the target domain action dataset, respectively. Let i be the feature vector of the i-th action data in the source domain at layer k. Let be the feature vector of the i-th action data in the target domain at layer k. In this embodiment, the JMMD method is adopted, which calculates the outer product of the output features of multiple specific network layers to obtain the joint distribution of domain features, thereby achieving unsupervised domain adaptation.

[0187] Step D12: Calculate the cross-entropy loss based on the source domain predicted labels and source domain sample labels corresponding to each historical action data, and determine the cross-entropy loss function.

[0188] Understandably, for each historical action data point, to avoid excessively large changes in the parameters of the user's corresponding adapter, leading to excessive deviation from the source domain, a cross-entropy loss calculation is added to the output layer of the source domain. The cloud server calculates the difference between the predicted label in the source domain and the label of the sample in the source domain, measuring the difference between the model's predicted probability distribution and the true label distribution, thus obtaining the cross-entropy loss function. The cross-entropy loss function optimizes the model's classification task performance in the source domain. By minimizing the cross-entropy loss, the model can accurately predict labels on the source domain data.

[0189] Step D13: Perform regularization calculations based on the current model parameters of the user's corresponding adapter to determine the regularization loss function.

[0190] It should be noted that the current model parameters refer to the parameters of the adapter corresponding to the user. Regularization is calculated using the current model parameters of the adapter corresponding to the current user and the preset regularization coefficients to obtain the regularization loss function. The regularization loss function constrains the magnitude of the model parameters, making the model simpler, with stronger generalization ability, and preventing overfitting.

[0191] Step D14: Calculate the loss function based on the cross-entropy loss function, domain adaptation loss function, regularization loss function, and objective function weights to determine the objective loss function.

[0192] It should be noted that the objective function weights include preset weight coefficients for each loss term. In this embodiment, the objective loss function consists of three parts, and the specific calculation formula is as follows: L L =L JMMD (S,T)+αL CE (S)+βL z L CE (S) is the cross-entropy loss function. α is the regularization loss function, and β are the weights of the objective function, both of which take values ​​in the range (0, 1).

[0193] It is understandable that the specific process of cloud servers performing long-term fine-tuning is as follows: Figure 8As shown, during fine-tuning, the cloud server simultaneously inputs the source domain data ds and the target domain data dt into an encoder with the same shared parameters. After passing through a shared-parameter encoder, the source and target domain data each produce their own intermediate vectors, which are then input into the user-specific adapter with shared parameters, resulting in a set of intermediate feature vectors zs and zt. The loss function LJMMD is calculated using the maximum mean difference of the joint distribution to obtain the joint distribution of domain features, thus achieving unsupervised domain adaptation. Finally, to avoid excessive changes in the adapter parameters leading to significant deviation from the source domain action dataset, cross-entropy loss is added to the output layer of the source domain. The final loss function consists of three parts: LJMMD for domain alignment with new user data, LCE(S) to prevent excessive bias in the model, ensuring strong recognition capabilities in the source domain, and LZ, which, like the short-term calibration process, uses L1 regularization to prevent overfitting.

[0194] In practice, the fine-tuning process for long-term calibration on cloud servers differs from that of short-term calibration on mobile terminals. Short-term calibration uses only a very small amount of user data for fine-tuning, which is very fast and does not require a large number of iterations. However, the long-term calibration process uses a large amount of historical user data, and the influence of randomness needs to be avoided during the JMMD domain matching stage, requiring a large number of iteration rounds and training time. Therefore, the long-term calibration process is uniformly executed automatically on the cloud server, that is, the calibration process of the user action model is automatically executed without the user's awareness.

[0195] This embodiment provides an action recognition method based on an edge-cloud collaborative system. The method involves determining multiple historical action data and pseudo-labels for each historical action data based on the user's target domain action dataset; randomly matching the source domain sample labels and pseudo-labels of each historical action data in the source domain action dataset to determine the sample action data and source domain sample labels corresponding to each historical action data; inputting the sample action data and historical action data into the action recognition model, and extracting features from the encoder within the model to obtain the source domain intermediate vector and target intermediate vector; and calibrating the user's corresponding adapter parameters based on the source domain intermediate vector, target intermediate vector, and source domain sample labels to determine the user's corresponding target adapter parameters. By using the above method, under the premise of freezing the encoder, the adapter parameters are dynamically optimized only for the user's target domain data, realizing unsupervised adaptation of cross-domain action features: it can align the action feature distribution between the target domain and the source domain, maintain the basic classification performance through the source domain label, and finally achieve high-precision action recognition in the target domain, while avoiding overfitting, significantly improving the model's generalization and robustness in complex real-world scenarios.

[0196] Based on the first and / or second and / or third embodiments of this application, in the fourth embodiment of this application, the content that is the same as or similar to the first, second, and third embodiments described above can be referred to the above description and will not be repeated hereafter. Based on this, please refer to... Figure 9 In this embodiment, the edge-cloud collaborative system includes a cloud server, multiple terminal devices and wearable devices corresponding to each terminal device. Each terminal device is communicatively connected to its corresponding wearable device and to the cloud server.

[0197] The action recognition method based on the edge-cloud collaborative system is applied to wearable devices. The action recognition method based on the edge-cloud collaborative system includes steps S40 to S50:

[0198] Step S40: Upon receiving the adapter parameters corresponding to the user sent by the mobile terminal, update the action recognition model according to the adapter parameters corresponding to the user to obtain the updated action recognition model.

[0199] It should be noted that the execution subject of this embodiment is a wearable device in the edge-cloud collaborative system, which is a smart wearable device with data processing, data acquisition, network communication and program execution functions, including but not limited to wristbands, rings, watches, head-mounted devices, etc.

[0200] It is understandable that the adapter parameters corresponding to a user include the target adapter parameters and the adapter fine-tuning parameters. The target adapter parameters are the parameters of the adapter corresponding to the user obtained by the cloud server after long-term fine-tuning and calibration based on the user's historical data; the adapter fine-tuning parameters are the parameters of the adapter corresponding to the user obtained by the mobile terminal after short-term fine-tuning and calibration based on the user's action data with real tags.

[0201] In practical implementation, when a user uses a wearable device for the first time, the action recognition model is a basic recognition model sent to the mobile terminal. This basic recognition model is trained on a cloud server and then sent to the mobile terminal corresponding to the wearable device. When the user is not using the wearable device for the first time, the action recognition model consists of a user-specific adapter, an embedding layer, an input layer, an encoder, and an output layer from the basic recognition model.

[0202] It is understandable that when a wearable device receives the adapter parameters corresponding to the user sent by the mobile terminal, it updates the adapter parameters in the action recognition model to obtain the updated action recognition model.

[0203] Step S50: Perform action recognition on the user's current action data using the updated action recognition model to obtain the target action label and the label confidence of the target action label for the current action data.

[0204] It should be noted that wearable devices receive the user's current motion data through sensors, such as electromyography, inertial measurement unit, photoplethysmography, and electroencephalography. The device preprocesses the current motion data according to a window, calls the updated motion recognition model to determine the type of the preprocessed current motion data, and thus obtains the motion type corresponding to the current motion data. The updated motion recognition model outputs a target motion label that reflects the motion type corresponding to the current motion data, as well as the label confidence of the target motion label.

[0205] In one feasible implementation, after step S50, steps E11 to E12 may also be included:

[0206] Step E11: Compare the label confidence level with the confidence threshold.

[0207] Step E12: When the label confidence is greater than the confidence threshold, send the current action data and the target action label of the current action data to the mobile terminal, so that the mobile terminal sends the current action data and the target action label of the current action data to the cloud server, so that the cloud server constructs the target domain action dataset based on the current action data and the target action label of the current action data.

[0208] It should be noted that the wearable device compares the tag confidence score of the current action data with a confidence threshold. If the tag confidence score is greater than the confidence threshold, the current action data and its corresponding target action tag are saved. When the wearable device and its corresponding terminal device are in communication connection, the user's behavior data with a tag confidence score greater than the confidence threshold and its corresponding action tag are uploaded to the mobile terminal. In this embodiment, the confidence threshold is set to 90%, but it can be adjusted according to needs; this embodiment does not impose any restrictions on this.

[0209] Understandably, when the cloud server receives user behavior data with a confidence level greater than a confidence threshold and its corresponding action tags uploaded by a mobile terminal, it stores this data in the cloud database. The action tags corresponding to this behavior data serve as pseudo-labels for that behavior data. The cloud server constructs a target domain action dataset for the user based on multiple behavior data points uploaded by the same user from the mobile terminal and the pseudo-labels for each behavior data point. This target domain action dataset is used for long-term calibration of the user's corresponding adapter; the action data within it has not participated in long-term fine-tuning.

[0210] This embodiment provides an action recognition method based on an edge-cloud collaborative system. Upon receiving adapter parameters corresponding to the user from the mobile terminal, the method updates the action recognition model based on these parameters, resulting in an updated model. The updated model is then used to recognize the user's current action data, yielding the target action label and its confidence level. By dynamically updating the user-specific adapter parameters, this method achieves high precision and reliability of personalized action recognition on the device side, significantly improving the accuracy of action recognition in specific scenarios while ensuring efficient operation even in resource-constrained environments on wearable devices.

[0211] For example, to aid in understanding the implementation process of the action recognition method based on an edge-cloud collaborative system obtained by combining this embodiment with the above-described embodiments one, two, and three, specifically: This embodiment provides an edge-cloud collaborative architecture that allows users to obtain a recognition model that better matches their action characteristics and habits through a small number of calibration operations, and fully utilizes the computing and storage resources of the edge, making the resource scheduling of fine-tuning tasks more reasonable. It rationally allocates computing power and storage resources between the edge and cloud, enabling short-term rapid calibration on the terminal device and long-term batch calibration in the cloud. In the long-term calibration process, through the synergistic effect of action filtering strategies on the edge, historical data scheduling on the edge, and historical data integration and automatic fine-tuning calls in the cloud, it achieves seamless calibration based on user habits. This meets the user's needs for short-term rapid calibration and long-term user habit adaptation calibration. Short-term calibration employs an adapter fine-tuning strategy, fixing the encoder parameters and only updating the parameters in the adapter for a specific user using gradients. Long-term calibration uses a loss function combining JMMD and cross-entropy, ensuring that the model does not produce excessive bias while guiding the model parameters to shift towards the user's historical data features, thereby obtaining a motion recognition model adapted to the user.

[0212] This embodiment fully leverages the characteristics of the edge-cloud system architecture to reduce the computational burden on the edge. Wearable devices on the edge typically have limited computing and storage resources, and battery power consumption also affects usage. Therefore, the fine-tuning process needs to make reasonable use of edge computing power and cloud computing and storage resources, so that the model can be well fine-tuned and trained with user data, while minimizing the computational and storage burden on the edge device.

[0213] Furthermore, full parameter calibration of the model typically leads to significant model shifts. While this may improve user experience in the short term, it reduces the model's versatility, makes it difficult for other users to use, and introduces greater anomalies (e.g., users performing short-term calibration incorrectly or recording outliers). This embodiment performs short-term calibration at the edge and long-term calibration in the cloud, achieving a balance between short-term and long-term calibration within the same model and ensuring the stability of the calibration results.

[0214] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the action recognition method based on the edge-cloud collaborative system of this application. Any simple modifications based on this technical concept are within the protection scope of this application.

[0215] This application also provides an edge-cloud collaborative system, which includes a cloud server, multiple terminal devices and wearable devices corresponding to each terminal device, each terminal device is communicatively connected to its corresponding wearable device, and each terminal device is communicatively connected to the cloud server.

[0216] The cloud server is used to calibrate the parameters of the user's corresponding adapter based on the user's target domain action dataset and source domain action dataset, determine the user's corresponding target adapter parameters, and send the user's corresponding target adapter parameters to the terminal device. The terminal device is used to forward the target adapter parameters to the wearable device.

[0217] The terminal device is used to perform parameter calibration on the adapter corresponding to the user based on multiple fine-tuning action data sent by the wearable device and the actual action tags of each fine-tuning action data, determine the adapter fine-tuning parameters corresponding to the user, and send the adapter fine-tuning parameters to the wearable device.

[0218] Wearable devices are used to update the parameters of the motion recognition model based on the adapter parameters corresponding to the user sent by the terminal device, and then use the updated motion adaptation model to recognize the user's current motion data.

[0219] The edge-cloud collaborative system provided in this application employs the action recognition method based on the edge-cloud collaborative system described in the above embodiments. This method addresses the challenge of achieving seamless and continuous adaptation of the action recognition model to users' short- and long-term behavioral characteristics under limited edge-side resources, while avoiding the loss of model universality and stability degradation caused by full parameter updates. Compared to existing technologies, the beneficial effects of the edge-cloud collaborative system provided in this application are the same as those of the action recognition method based on the edge-cloud collaborative system provided in the above embodiments. Furthermore, other technical features of the edge-cloud collaborative system are the same as those disclosed in the methods of the above embodiments, and will not be elaborated upon here.

[0220] The above are only some embodiments of this application and do not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the content of this application specification and drawings, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. An action recognition method based on an edge-cloud collaborative system, characterized in that, The edge-cloud collaborative system includes a cloud server, multiple terminal devices, and wearable devices corresponding to each terminal device. Each terminal device is communicatively connected to its corresponding wearable device and to the cloud server. The action recognition method based on the edge-cloud collaborative system is applied to a terminal device, and the method includes: When there is a need for model fine-tuning, acquire multiple fine-tuning action data of the user sent by the wearable device and the real action labels of each fine-tuning action data; Based on each fine-tuning action data and the actual action label of each fine-tuning action data, the parameters of the adapter corresponding to the user are calibrated to determine the fine-tuning parameters of the adapter corresponding to the user. The adapter fine-tuning parameters corresponding to the user are sent to the wearable device so that the wearable device updates the parameters of the motion recognition model according to the adapter fine-tuning parameters corresponding to the user, and recognizes the user's current motion data through the updated motion recognition model.

2. The method as described in claim 1, characterized in that, The step of calibrating the adapter corresponding to the user based on each fine-tuning action data and the actual action label of each fine-tuning action data, and determining the fine-tuning parameters of the adapter corresponding to the user, includes: Each fine-tuning action data is input into the action recognition model to obtain the model prediction probability of each fine-tuning action data. The first loss function is determined by calculating the cross-entropy loss based on the model prediction probability of each fine-tuned action data and the true action label of each fine-tuned action data. The second loss function is determined by performing regularization calculations based on the current model parameters of the user's corresponding adapter. The adapter corresponding to the user is calibrated according to the first loss function and the second loss function to determine the fine-tuning parameters of the adapter corresponding to the user.

3. The method as described in claim 1, characterized in that, The model fine-tuning requirements include at least one of the following: The number of times a user can use the service is the preset number of times. The number of times a user uses the service is not the preset number of times, and the time interval between user uses the service is greater than the preset time interval; A model fine-tuning request was received from the wearable device.

4. An action recognition method based on an edge-cloud collaborative system, characterized in that, The edge-cloud collaborative system includes a cloud server, multiple terminal devices, and wearable devices corresponding to each terminal device. Each terminal device is communicatively connected to its corresponding wearable device and to the cloud server. The action recognition method based on the edge-cloud collaborative system is applied to a cloud server, and the method includes: When the amount of target action data corresponding to a user is greater than a preset data amount threshold, the adapter corresponding to the user is calibrated according to the target domain action dataset and the source domain action dataset of the user to determine the target adapter parameters corresponding to the user. The target adapter parameters corresponding to the user are sent to the terminal device. The terminal device forwards the target adapter parameters corresponding to the user to the wearable device, so that the wearable device updates the parameters of the action recognition model according to the target adapter parameters corresponding to the user, and recognizes the user's current action data through the updated action recognition model.

5. The method as described in claim 4, characterized in that, The step of calibrating the parameters of the adapter corresponding to the user based on the user's target domain action dataset and source domain action dataset, and determining the parameters of the target adapter corresponding to the user, includes: Based on the user's target domain action dataset, determine multiple historical action data and action pseudo-labels for each historical action data; Based on the source domain sample label corresponding to each sample action data in the source domain action dataset and the action pseudo label corresponding to each historical action data, random matching is performed in the source domain action dataset to determine the sample action data corresponding to each historical action data and the source domain sample label corresponding to each historical action data. The sample action data and historical action data corresponding to each historical action data are respectively input into the action recognition model. The encoder in the action recognition model extracts features from the sample action data and historical action data corresponding to each historical action data to obtain the source domain intermediate vector and the target intermediate vector corresponding to each historical action data. Based on the source domain intermediate vector, the target intermediate vector, and the source domain sample label corresponding to each historical action data, the parameters of the adapter corresponding to the user are calibrated to determine the target adapter parameters corresponding to the user.

6. The method as described in claim 5, characterized in that, The step of calibrating the parameters of the user's corresponding adapter based on the source domain intermediate vector, the target intermediate vector, and the source domain sample labels corresponding to each historical action data, and determining the parameters of the user's corresponding target adapter, includes: Input the target intermediate vector and the source domain intermediate vector corresponding to each historical action data into the user's corresponding adapter to obtain the source domain feature vector and the target feature vector corresponding to each historical action data. The source domain feature vectors corresponding to each historical action data are input into the output layer of the action recognition model to obtain the source domain predicted labels corresponding to each historical action data. The target loss function is determined by calculating the source domain predicted label, source domain feature vector, target feature vector, and source domain sample label corresponding to each historical action data. The parameters of the adapter corresponding to the user are calibrated according to the target loss function to determine the target adapter parameters corresponding to the user.

7. The method as described in claim 6, characterized in that, The step of calculating the loss function based on the source domain predicted label, source domain feature vector, target feature vector, and source domain sample label of each historical action data, and determining the target loss function, includes: The joint distribution difference of the target feature vector and the source domain feature vector corresponding to each historical action data is calculated to determine the domain adaptation loss function. The cross-entropy loss function is determined by calculating the cross-entropy loss function based on the source domain prediction labels and source domain sample labels corresponding to each historical action data. Regularization calculations are performed based on the current model parameters of the adapter corresponding to the user to determine the regularization loss function; The target loss function is determined by calculating the loss function based on the cross-entropy loss function, the domain adaptation loss function, the regularization loss function, and the objective function weights.

8. The method according to any one of claims 4 to 7, characterized in that, The step of determining the parameters of the target adapter corresponding to the user when the target action data volume for the user exceeds a preset data volume threshold includes: Upon receiving a model update request from a terminal device, the user whose model is being updated and the amount of target action data corresponding to that user are determined based on the model update request. When the amount of target action data corresponding to the user is greater than a preset data amount threshold, data is searched in the parameter calibration queue. If the target domain action dataset corresponding to the user is not found in the parameter calibration queue, the adapter corresponding to the user is parameter calibrated based on the user's target domain action dataset and source domain action dataset to determine the target adapter parameters corresponding to the user.

9. An action recognition method based on an edge-cloud collaborative system, characterized in that, The edge-cloud collaborative system includes a cloud server, multiple mobile terminals and wearable devices corresponding to each mobile terminal. Each mobile terminal is communicatively connected to its corresponding wearable device and to the cloud server. The action recognition method based on the edge-cloud collaborative system is applied to wearable devices, and the method includes: Upon receiving the adapter parameters corresponding to the user sent by the mobile terminal, the action recognition model is updated according to the adapter parameters corresponding to the user to obtain the updated action recognition model. The updated action recognition model is used to identify the user's current action data, thereby obtaining the target action label and the label confidence of the target action label.

10. The method as described in claim 9, characterized in that, After the step of performing action recognition on the user's current action data using the updated action recognition model to obtain the target action label and the label confidence of the target action label, the method further includes: The label confidence level and the confidence threshold are compared. When the confidence level of the label is greater than the confidence threshold, the current action data and the target action label of the current action data are sent to the mobile terminal, so that the mobile terminal sends the current action data and the target action label of the current action data to the cloud server, so that the cloud server constructs a target domain action dataset based on the current action data and the target action label of the current action data.

11. An edge-cloud collaborative system, characterized in that, The edge-cloud collaborative system includes a cloud server, multiple terminal devices, and wearable devices corresponding to each terminal device. Each terminal device is communicatively connected to its corresponding wearable device and to the cloud server. The cloud server is used to perform parameter calibration on the adapter corresponding to the user based on the user's target domain action dataset and source domain action dataset, determine the target adapter parameters corresponding to the user, and send the target adapter parameters corresponding to the user to the terminal device. The terminal device is used to forward the target adapter parameters to the wearable device. The terminal device is used to perform parameter calibration on the adapter corresponding to the user based on multiple fine-tuning action data sent by the wearable device and the actual action label of each fine-tuning action data, determine the adapter fine-tuning parameters corresponding to the user, and send the adapter fine-tuning parameters to the wearable device. The wearable device is used to update the parameters of the action recognition model according to the adapter parameters corresponding to the user sent by the terminal device, and to recognize the user's current action data through the updated action adaptation model.

Images