Exercise counting method, server and system

Abstract

The invention discloses an exercise counting method, a server and a system. The method comprises the following steps: acquiring exercise data and voice data of a user in an exercise process collected by intelligent wearable equipment; determining a voice counting result according to the voice data; wherein the voice counting result comprises a voice counting value; matching a corresponding target action recognition model according to the motion data; inputting the motion data into the target motion recognition model to obtain a motion counting result of the target motion; wherein the action counting result comprises an action counting value; and under the condition that the voice count value and the action count value are not equal, performing model correction processing on the target action recognition model, and determining the corrected action recognition model as a customized action recognition model of the user.

Description

Technical Field

[0001] This invention relates to the field of motion detection technology, and more specifically, to a motion counting method, server, and system. Background Technology

[0002] With the rise of emerging sports such as smart fitness, smart wearable devices are often embedded with exercise counting modules to record the number of times a user exercises. For example, they can record the number of times a user jumps rope or bounces a ball.

[0003] In current technologies, the motion counting algorithms used in smart wearable devices are universal and fixed, failing to adapt to the personalized data of the user, thus resulting in certain deficiencies in the accuracy of motion counting. Furthermore, existing motion recognition algorithms are generally based on video stream motion recognition or on three-axis, six-axis, or nine-axis sensors. These algorithms exhibit poor accuracy in motion recognition and counting when the user is performing slow or fast movements. Summary of the Invention

[0004] One objective of this invention is to provide a new technical solution for action counting.

[0005] According to a first aspect of the present invention, a motion counting method is provided, comprising:

[0006] Acquire motion and voice data collected by smart wearable devices during user exercise;

[0007] Based on the voice data, a voice counting result is determined; wherein, the voice counting result includes a voice count value;

[0008] Based on the motion data, a corresponding target action recognition model is matched;

[0009] The motion data is input into the target action recognition model to obtain the action count result of the target action; wherein, the action count result includes the action count value;

[0010] If the voice count value and the action count value are not equal, the target action recognition model is modified, and the modified action recognition model is determined as the user's customized action recognition model.

[0011] Optionally, when the voice count value and the action count value are not equal, performing model correction processing on the target action recognition model and determining the corrected action recognition model as the user's customized action recognition model includes:

[0012] If the voice count value and the action count value are not equal, the target action recognition model is modified, and the motion data is analyzed through the modified target action recognition model to obtain the modified action count value.

[0013] If the corrected action count value and the voice count value are equal, the corrected target action recognition model is determined as the customized action recognition model.

[0014] Optionally, determining the voice counting result based on the voice data includes:

[0015] Identify the semantic content in the speech data;

[0016] The speech count value is determined based on the semantic content.

[0017] Optionally, matching the corresponding target action recognition model based on the motion data includes:

[0018] Extract motion features from the motion data;

[0019] Based on the motion characteristics, determine the target action category corresponding to the motion characteristics;

[0020] Based on the target action category, determine the corresponding target action recognition model.

[0021] Optionally, the speech counting result further includes speech counting time points, and the action counting result further includes action counting time points. The step of performing model correction processing on the target action recognition model when the speech count value and the action count value are not equal includes:

[0022] If the voice count value and the action count value are not equal, the voice count time point and the action count time point are compared to determine the error timestamp; wherein, the error timestamp is the time point at which the voice count and the action count are inconsistent;

[0023] Determine whether the target action recognition model has made an error in recognizing the action of the motion data corresponding to the error timestamp;

[0024] If an error is found in the action recognition, the action recognition model is corrected.

[0025] Optionally, after determining the customized action recognition model, the method further includes:

[0026] The customized motion recognition model is sent to the smart wearable device so that the smart wearable device can input the measured motion data of the user when performing the target motion in the next collection into the customized motion recognition model to obtain and output the measured count value.

[0027] Optionally, after determining the customized action recognition model, the method further includes:

[0028] Receive the measured motion data of the user when performing the target action movement, which will be collected by the smart wearable device next time;

[0029] The measured motion data is input into the customized motion recognition model to obtain the measured count value;

[0030] The measured count value is sent to the smart wearable device so that the smart wearable device can output the measured count value.

[0031] Optionally, after the smart wearable device outputs the measured count value, the method further includes:

[0032] Receive a target count value sent by the smart wearable device; wherein the target count value is determined based on the user's input to the smart wearable device;

[0033] If the target count value and the measured count value are not equal, the customized motion recognition model is modified so that the modified measured count value obtained by the modified customized motion recognition model from the measured motion data is equal to the target count value.

[0034] According to a second aspect of this disclosure, a server is provided, including a memory and a processor, the memory being configured to store executable instructions; the processor being configured to operate under the control of the instructions to perform the method as described in the first aspect.

[0035] According to a third aspect of this disclosure, a motion counting system is provided, comprising a smart wearable device and the server described in the second aspect, wherein the smart wearable device is used to collect motion data and voice data of a user during exercise and send them to the server.

[0036] One beneficial effect of this invention is that by verifying the motion recognition accuracy of the target motion recognition model based on the user's voice and motion data during movement, if the voice count and motion count values ​​are not equal, it indicates that the motion recognition accuracy of the model does not meet the requirements and the model is not suitable for the user's personalized motion recognition. In this case, the model is corrected to customize a personalized motion recognition model for the user's subsequent motion monitoring. This method allows for the customization of a personalized motion recognition model based on voice and motion data, improving the model's fit with the user, enhancing the user experience, and significantly improving the accuracy of motion counting through this customized motion recognition model for user motion monitoring. Attached Figure Description

[0037] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments of the invention and, together with their description, serve to explain the principles of the invention.

[0038] Figure 1 This is a schematic diagram of the hardware structure of a motion counting system according to an embodiment of the present invention;

[0039] Figure 2 This is a schematic flowchart of a motion counting method according to an embodiment of the present invention;

[0040] Figure 3 This is a schematic block diagram of a server according to an embodiment of the present invention;

[0041] Figure 4 This is a schematic diagram of the structure of a motion counting system according to an embodiment of the present invention. Detailed Implementation

[0042] Various exemplary embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of the invention.

[0043] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the invention or its application or use.

[0044] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0045] In all the examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values.

[0046] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0047] <Hardware Configuration>

[0048] Figure 1 This is a block diagram of the hardware configuration of a motion counting system 100 according to an embodiment of the present invention.

[0049] like Figure 1 As shown, the motion counting system 100 includes a smart wearable device 1000 and a server 2000.

[0050] The smart wearable device 1000 can be, for example, a smart bracelet, a smart armband, a smart watch, smart glasses, a smart earphone, etc., and is not limited here.

[0051] The smart wearable device 1000 can collect the user's exercise data and voice data during exercise and send them to the server 2000.

[0052] In this embodiment, refer to Figure 1 As shown, the smart wearable device 1000 may include a processor 1100, a memory 1200, an interface device 1300, a communication device 1400, a display device 1500, an input device 1600, a speaker 1700, a sensor 1800, etc.

[0053] Processor 1100 may be a mobile processor. Memory 1200 includes, for example, ROM (Read-Only Memory), RAM (Random Access Memory), and non-volatile memory such as a hard disk. Interface device 1300 includes, for example, a USB interface and a headphone jack. Communication device 1400 is capable of wired or wireless communication. Communication device 1400 may include short-range communication devices, such as any device that performs short-range wireless communication based on short-range wireless communication protocols such as Hilink, WiFi (IEEE 802.11), Mesh, Bluetooth, ZigBee, Thread, Z-Wave, NFC, UWB, and LiFi. Communication device 1400 may also include long-range communication devices, such as any device that performs WLAN, GPRS, or 2G / 3G / 4G / 5G long-range communication. Display device 1500 is, for example, an LCD screen or a touch screen, used to display measured count values ​​for operation monitoring. Input device 1600 may include, for example, a touch screen or a keyboard. Users can input / output voice information via speaker 1700. Sensor 1800 can be, for example, a microphone, accelerometer, gyroscope, heart rate sensor, etc.

[0054] Microphones are used to collect users' voice data. Smart wearable devices such as smartwatches and health trackers typically have one or more built-in microphones to collect voice signals.

[0055] Accelerometers, gyroscopes, heart rate sensors, and other sensors are used to collect users' motion data.

[0056] An accelerometer, for example, can be a triaxial accelerometer sensor, used to measure the acceleration of a device in space along the X, Y, and Z axes. An accelerometer can detect changes in acceleration caused by gravity or motion. A gyroscope, for example, can be a triaxial gyroscope sensor, used to measure the rotational speed, or angular velocity, of a device around its X, Y, and Z axes.

[0057] In this embodiment, the memory 1200 of the smart wearable device is used to store instructions for controlling the processor 1100 to operate to at least execute the motion counting method implemented by the smart wearable device according to any embodiment of the present invention. Those skilled in the art can design the instructions according to the disclosed scheme of the present invention. How the instructions control the processor to operate is well known in the art and will not be described in detail here.

[0058] Despite Figure 1 The invention illustrates multiple devices of a smart wearable device 1000; however, the invention may refer to only some of these devices. For example, the smart wearable device 1000 may refer only to the memory 1200 and the processor 1100.

[0059] Server 2000 is used to receive motion data and voice data collected by smart wearable device 1000 during user exercise, and to train a customized action recognition model for the user based on the motion data and voice data.

[0060] Server 2000 may include processor 2100, memory 2200, interface device 2300, communication device 2400, display device 2500, input device 2600, speaker 2700, microphone 2800, etc.

[0061] Processor 2100 may be a mobile processor. Memory 2200 includes, for example, ROM (Read-Only Memory), RAM (Random Access Memory), and non-volatile memory such as a hard disk. Interface device 2300 includes, for example, a USB interface and a headphone jack. Communication device 2400 is capable of wired or wireless communication. Communication device 2400 may include short-range communication devices, such as any device that performs short-range wireless communication based on short-range wireless communication protocols such as Hilink, WiFi (IEEE 802.11), Mesh, Bluetooth, ZigBee, Thread, Z-Wave, NFC, UWB, and LiFi. Communication device 2400 may also include long-range communication devices, such as any device that performs WLAN, GPRS, or 2G / 3G / 4G / 5G long-range communication. Display device 2500 is, for example, an LCD screen or a touch screen. Input device 2600 may include, for example, a touch screen or a keyboard. Users can input / output voice information through speaker 2700 and microphone 2800.

[0062] In this embodiment, the memory 2200 of the server 2000 is used to store instructions for controlling the processor 2100 to operate in order to at least execute the motion counting method performed by the server 2000 according to any embodiment of the present invention. Those skilled in the art can design the instructions according to the disclosed scheme of the present invention. How the instructions control the processor to operate is well known in the art and will not be described in detail here.

[0063] Despite Figure 1 The invention illustrates multiple devices of server 2000, but may refer to only some of these devices; for example, server 2000 may refer only to memory 2200 and processor 2100.

[0064] In this embodiment, the smart wearable device is used to collect the user's voice data and motion data during exercise and provide them to the server 2000. The server 2000 then constructs a customized motion recognition model for the user based on the voice data and motion data.

[0065] It should be understood that, despite Figure 1 Only one smart wearable device 1000 and one server 2000 are shown, but this does not mean that the number of each is limited. The motion counting system 100 may contain multiple smart wearable devices 1000 and / or servers 2000.

[0066] <Method Implementation>

[0067] Figure 2 This is a flowchart illustrating a motion counting method according to an embodiment of the present invention, which can be implemented by a server 2000.

[0068] according to Figure 2 As shown, the motion counting method of this embodiment may include the following steps S2100 to S2500:

[0069] Step S2100: Obtain the user's exercise data and voice data collected by the smart wearable device during exercise.

[0070] In this embodiment, when wearing a smart wearable device, the user can operate the device to activate a target exercise mode, such as a rope skipping mode or a ball-bouncing mode. Once the target exercise mode is activated, the user begins performing the target action corresponding to that mode and loudly announces the number of times the target action has been performed.

[0071] For example, when a user activates the jump rope exercise mode, the user starts jumping rope and loudly counts. The counting can be done by adding a number with each jump, such as 1, 2, 3, or by making rhythmic sounds, such as 1 2 1, etc. There are no restrictions here.

[0072] When the target exercise mode is activated, the smart wearable device collects the user's voice and exercise data and uploads them to the server 2000.

[0073] In some examples, smart wearable devices are equipped with sensors such as microphones, accelerometers, and gyroscopes. The microphones are used to collect the user's voice data, while the accelerometers and gyroscopes collect the user's motion data.

[0074] The motion data can be three-axis motion data, six-axis motion data, or nine-axis motion data; there is no limitation here.

[0075] Step S2200: Determine the voice counting result based on the voice data.

[0076] In this embodiment, artificial intelligence technology can be used to recognize speech data and determine the speech counting result. Commonly used artificial intelligence technologies for speech recognition include automatic speech recognition (ASR), feature extraction, deep learning models, acoustic models, etc., which are not limited here.

[0077] The voice counting result includes the voice count value, that is, the total number of times the user counts by voice while moving.

[0078] When the user counts by incrementing the count with each movement, the voice count is the count at the end of the movement. When the user counts by making rhythmic sounds, the voice count is the total number of rhythmic sounds.

[0079] In some embodiments, determining the voice counting result based on the voice data in step S2200 includes steps S2200.1 and S2200.2.

[0080] Step S2200.1, identify the semantic content in the speech data.

[0081] In this embodiment, speech features are extracted from the speech data, and a deep learning model (such as a recurrent neural network RNN or a long short-term memory network LSTM) is used to analyze the speech features to identify the semantic content in the speech. The semantic content can be words or numbers.

[0082] Step S2200.2, determine the speech count value according to the semantic content.

[0083] In this embodiment, the count shouted by the user is determined from the recognized words (One, two, three...) or numbers (one, two, three...), and the count integration is performed according to the count shouted by the user to obtain the final speech count value.

[0084] In some examples, the speech count result further includes the speech count time point.

[0085] The speech count time point refers to the time stamp corresponding to each count. The determination of the speech count time point is to record the time stamp corresponding to each count when determining the speech count value according to the semantic content in step S2200.2 and when determining the count shouted by the user from the recognized words or numbers.

[0086] Step S2300, match the corresponding target action recognition model according to the motion data.

[0087] In this embodiment, in order to improve the accuracy of action recognition and the accuracy of action counting, different motion actions correspond to different action recognition models. For example, the action recognition model corresponding to skipping rope is different from the action recognition model corresponding to bouncing a ball. Therefore, before recognizing the motion count in the motion data, the corresponding target action recognition model is matched for the motion data.

[0088] In some embodiments, matching the corresponding target action recognition model according to the motion data in step S2300 includes: steps S2300.1 to S2300.3.

[0089] Step S2300.1, extract the motion features in the motion data.

[0090] In this embodiment, the motion features can be features that show differences in different actions, which are helpful for action recognition.

[0091] The motion features can be, for example, acceleration features, gyroscope features, heart rate features, etc., which are not limited here.

[0092] Step S2300.2: Determine the target action category corresponding to the motion feature based on the motion feature.

[0093] In this embodiment, the motion feature can be compared with the motion features corresponding to a preset action category to determine the target action category to which the motion feature belongs.

[0094] For example, the extracted motion features can be input into a classifier, which will compare the motion features with the motion features corresponding to preset action categories to determine which preset action category the motion feature best matches. The action category that best matches is the target action category.

[0095] Step S2300.3: Determine the corresponding target action recognition model based on the target action category.

[0096] In this embodiment, the server has a model library containing multiple action recognition models for different action categories. Furthermore, the action recognition models in the model library are action recognition models that have not been customized by the user.

[0097] The server can determine the target action recognition model corresponding to the target action category based on the motion data. For example, if the target action category is rope skipping, then the target action recognition model is the rope skipping recognition model.

[0098] Step S2400: Input the motion data into the target action recognition model to obtain the action count result of the target action.

[0099] In this embodiment, when motion data is input into the target action recognition model, the target action recognition model can identify the target action in the motion data and count the identified target actions to obtain the action count result for the target action. The action count result includes the action count value.

[0100] For example, the rope skipping recognition model can identify rope skipping movements in motion data and count the number of rope skipping movements as 56, thus obtaining the movement count result for rope skipping, that is, the movement count value is 56 times.

[0101] In some examples, the action count results also include the action count time point, that is, the timestamp corresponding to each action count.

[0102] For example, the rope skipping recognition model can identify rope skipping actions in motion data, count the number of rope skipping actions as 56, and record the timestamp corresponding to each rope skipping action, thus obtaining the action count result for rope skipping, that is, the action count value is 56 times and the timestamp corresponding to each rope skipping action.

[0103] Step S2500: If the voice count value and the action count value are not equal, perform model correction processing on the target action recognition model, and determine the corrected action recognition model as the user's customized action recognition model.

[0104] In this embodiment, if the voice count and action count are not equal, it indicates that the target action recognition model has certain defects in recognizing motion data. In this case, the target action recognition model is modified, for example, by adjusting model parameters, adjusting model structure, and improving feature extraction methods. The modified action recognition model is then designated as the user's customized action recognition model. Subsequently, when monitoring the user's motion, this customized action recognition model can be used for action recognition to improve the accuracy of motion counting.

[0105] According to an embodiment of this application, the accuracy of the target action recognition model is verified based on the user's voice and motion data during exercise. If the voice count and motion count are not equal, it indicates that the accuracy of the action recognition model does not meet the requirements, and the model is not suitable for the user's action recognition. In this case, the model is corrected to customize a personalized action recognition model for the user's subsequent motion monitoring. This method allows for the customization of a personalized action recognition model based on voice and motion data, improving the model's fit with the user, enhancing the user experience, and significantly improving the accuracy of motion counting through this customized action recognition model for motion monitoring.

[0106] In some embodiments, the method further includes: if the voice count value and the action count value are equal, determining the target action recognition model as the user's customized action recognition model.

[0107] In this embodiment, when the voice count value and the action count value are equal, it indicates that the target action recognition model is accurate in recognizing the action of the motion data. The model can be used for subsequent motion counting of the user. At this time, directly using the model as the user's customized action recognition model can save server power consumption and improve the efficiency of the customized action recognition model.

[0108] In some embodiments, step S2500 involves performing model correction processing on the target action recognition model when the voice count value and the action count value are not equal, and determining the corrected action recognition model as the user's customized action recognition model, including steps S3100 and S3200.

[0109] Step S3100: If the voice count value and the action count value are not equal, perform model correction processing on the target action recognition model, and analyze the motion data through the corrected target action recognition model to obtain the corrected action count value.

[0110] In this embodiment, model correction processing can be performed by changing the model's parameters, adjusting the model's structure, etc., and is not limited here.

[0111] For example, the target action recognition model has an interface for developers to modify the model parameters. Developers can then modify the model parameters through this interface. The modified target action recognition model is then used to re-identify the target action from the motion data to obtain the corrected action count value for the target action.

[0112] Step S3200: If the corrected action count value and the voice count value are equal, the corrected target action recognition model is determined as the customized action recognition model.

[0113] In this embodiment, when the voice count value is equal to the corrected action count value, it indicates that the target action recognition model is accurate in recognizing the action of the motion data. The corrected model can be used for subsequent motion counting of the user. At this time, the corrected model is used as the user's customized action recognition model.

[0114] According to the embodiments of this application, by performing model correction processing on the target action recognition model, the accuracy of the corrected target action recognition model in recognizing motion data meets the requirements. Then, the corrected target action recognition model can be used as a customized action recognition model for subsequent motion monitoring of users, which can improve the accuracy of motion counting.

[0115] In some embodiments, the voice counting result also includes voice counting time points, which represent the timestamps corresponding to each voice count. The action counting result also includes action counting time points, which represent the timestamps corresponding to each action count.

[0116] It should be noted that the methods for determining the voice counting time point and the action counting time point have been explained above and will not be elaborated here.

[0117] Based on this, in step S2500, when the voice count value and the action count value are not equal, the target action recognition model is corrected, including steps SA1 to SA3.

[0118] Step SA1: If the voice count value and the action count value are not equal, compare the voice count time point and the action count time point to determine the error timestamp.

[0119] In this embodiment, the error timestamp is the point in time when the voice count and action count are inconsistent.

[0120] For example, the voice count is 4 times, and the time points for these 4 voice counts are 0.5s, 1s, 1.5s, and 2s respectively. The action count is 3 times, and the time points for these 3 action counts are 0.5s, 1.5s, and 2s respectively. The error timestamp is 1s.

[0121] Step SA2: Determine whether the target action recognition model has made an error in recognizing the action of the motion data corresponding to the error timestamp.

[0122] In this embodiment, cross-validation, error analysis, and other analytical algorithms can be used to determine whether the model has made a mistake in recognizing the motion data of the error timestamp.

[0123] Step SA3: If it is determined that the action recognition is incorrect, perform model correction processing on the action recognition model.

[0124] According to the embodiments of this application, when the voice count value and the action count value are not equal, the voice count time point and the action count time point are compared to determine an error timestamp. This determines whether the target action recognition model has made an error in recognizing the action data corresponding to the error timestamp. If an error in action recognition is determined, the action recognition model is corrected. This avoids unnecessary model correction due to inaccurate voice counting or accidental factors, improving the accuracy of the customized action recognition model in motion counting.

[0125] In some embodiments, after determining the customized action recognition model, the method further includes:

[0126] The customized motion recognition model is sent to the smart wearable device so that the smart wearable device can input the measured motion data of the user when performing the target motion in the next collection into the customized motion recognition model to obtain and output the measured count value.

[0127] In this embodiment, after determining the customized motion recognition model for the user, the server can send the customized motion recognition model to the smart wearable device, allowing the smart wearable device to perform motion counting independently. Upon receiving the customized motion recognition model, the smart wearable device saves the model and, the next time it collects measured motion data of the user performing the target motion, can input the measured motion data into the model and output the measured count value. In other words, after saving the customized motion recognition model, the smart wearable device can automatically determine the measured count value based on the currently collected measured motion data of the user performing the target motion and display the measured count value to the user.

[0128] According to the embodiments of this application, by sending a customized motion recognition model to a smart wearable device, and the smart wearable device saving the model, the smart wearable device can automatically count motions, which improves the real-time performance of motion counting, protects user privacy, and enhances user experience.

[0129] In other embodiments, after determining the customized action recognition model, the method further includes steps S4100 to S4300.

[0130] Step S4100: Receive the measured motion data of the user when performing the target action movement, which is collected by the smart wearable device next time.

[0131] Step S4200: Input the measured motion data into the customized motion recognition model to obtain the measured count value.

[0132] Step S4300: Send the measured count value to the smart wearable device so that the smart wearable device can output the measured count value.

[0133] In this embodiment, the server stores the user's customized action recognition model, which is not sent to the smart wearable device.

[0134] In one example, the customized motion recognition model is a customized rope skipping recognition model, with rope skipping as the target motion. When the smart wearable device next collects measured motion data from a user performing rope skipping, it sends this data to the server. The server inputs this measured motion data into the customized rope skipping recognition model corresponding to that user, outputs a measured count value, and sends this count value to the smart wearable device for display.

[0135] By using interaction between servers and smart wearable devices to count motion, it is possible to achieve synchronous motion counting and data sharing across multiple devices, and it is also convenient to monitor and provide feedback on motion counting.

[0136] After the smart wearable device outputs the measured count value, there may be a problem that the output measured count value is inaccurate, that is, the measured count value is inconsistent with the user's actual count.

[0137] Based on this, in some embodiments, after the smart wearable device outputs the measured count value, the method further includes steps S5100 and S5200.

[0138] Step S5100: Receive the target count value sent by the smart wearable device.

[0139] In this embodiment, after the smart wearable device outputs the measured count value, the user can confirm that the measured count value is correct or incorrect.

[0140] In scenarios where the user confirms the measured count value is incorrect, the user can input the correct count value, i.e., the target count value, into the smart wearable device. Therefore, the target count value is determined based on the user's input to the smart wearable device. The smart wearable device then sends this target count value to the server.

[0141] In the example corresponding to motion counting based on interaction between a smart wearable device and a server, the server only sends the target count value to the server.

[0142] In the example where the corresponding smart wearable device performs motion counting on its own, the smart wearable device sends the target count value to the server, and also sends the measured count value and the customized motion recognition model stored by the smart wearable device to the server.

[0143] Step S5200: If the target count value and the measured count value are not equal, perform model correction processing on the customized action recognition model so that the corrected measured count value obtained by the modified customized action recognition model from the measured motion data is equal to the target count value.

[0144] In this embodiment, after receiving the target count value, the server compares the target count value with the measured count value. If the two are not equal, the customized action recognition model is corrected to obtain a corrected customized action recognition model. The server then uses this corrected customized action recognition model to re-recognize the measured motion data, obtaining a corrected measured count value. If the corrected measured count value equals the target count value, the customized action recognition model is updated to the corrected customized action recognition model.

[0145] It should be noted that the method of performing model correction processing on the customized action recognition model based on the target count value is basically the same as the method of performing model correction processing on the target action recognition model based on the speech count value. For details, please refer to the description in the above method embodiments, which will not be repeated here.

[0146] <Device Embodiment>

[0147] Figure 3 This is a schematic diagram of a server 300 according to an embodiment of the present invention.

[0148] In this embodiment, as Figure 3 As shown, server 300 includes memory 310 and processor 320, wherein memory 310 is used to store executable instructions; and processor 320 is used to operate under the control of the instructions to execute the method as described in any of the above method embodiments.

[0149] Figure 4 This is a schematic diagram of a motion counting system according to an embodiment of the present invention.

[0150] In this embodiment, as Figure 4 As shown, the motion counting system 400 includes a smart wearable device 410 and a server 420. The smart wearable device 410 is used to collect motion data of the user during historical motion processes and send it to the server 420.

[0151] In some embodiments, the smart wearable device 410 may be as follows: Figure 1 The smart wearable device 1000 shown.

[0152] In some embodiments, server 420 may be as follows: Figure 1 Or such as Figure 3 The server shown.

[0153] This invention can be a system, method, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for causing a processor to implement various aspects of the invention.

[0154] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0155] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0156] The computer program instructions used to perform the operations of this invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing state information from the computer-readable program instructions. This electronic circuitry can execute the computer-readable program instructions to implement various aspects of the invention.

[0157] Various aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0158] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0159] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0160] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions. It will be known to those skilled in the art that implementation in hardware, implementation in software, and implementation using a combination of software and hardware are equivalent.

[0161] The various embodiments of the present invention have been described above. These descriptions are exemplary and not exhaustive, and are not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein. The scope of the invention is defined by the appended claims.

Claims

1. A motion counting method, characterized in that, The method includes: Acquire motion and voice data collected by smart wearable devices during user exercise; Based on the voice data, a voice counting result is determined; wherein, the voice counting result includes a voice count value; Based on the motion data, a corresponding target action recognition model is matched; The motion data is input into the target action recognition model to obtain the action count result of the target action; wherein, the action count result includes the action count value; If the voice count value and the action count value are not equal, the target action recognition model is modified, and the modified action recognition model is determined as the user's customized action recognition model.

2. The method according to claim 1, characterized in that, When the voice count value and the action count value are not equal, the target action recognition model is corrected, and the corrected action recognition model is determined as the user's customized action recognition model, including: If the voice count value and the action count value are not equal, the target action recognition model is modified, and the motion data is analyzed through the modified target action recognition model to obtain the modified action count value. If the corrected action count value and the voice count value are equal, the corrected target action recognition model is determined as the customized action recognition model.

3. The method according to claim 1, characterized in that, Determining the voice counting result based on the voice data includes: Identify the semantic content in the speech data; The speech count value is determined based on the semantic content.

4. The method according to claim 1, characterized in that, The step of matching the corresponding target action recognition model based on the motion data includes: Extract motion features from the motion data; Based on the motion characteristics, determine the target action category corresponding to the motion characteristics; Based on the target action category, determine the corresponding target action recognition model.

5. The method according to claim 1, characterized in that, The speech counting result also includes speech counting time points, and the action counting result also includes action counting time points. The step of performing model correction processing on the target action recognition model when the speech count value and the action count value are not equal includes: If the voice count value and the action count value are not equal, the voice count time point and the action count time point are compared to determine the error timestamp; wherein, the error timestamp is the time point at which the voice count and the action count are inconsistent; Determine whether the target action recognition model has made an error in recognizing the action of the motion data corresponding to the error timestamp; If an error is found in the action recognition, the action recognition model is corrected.

6. The method according to claim 1, characterized in that, After determining the customized action recognition model, the method further includes: The customized motion recognition model is sent to the smart wearable device so that the smart wearable device can input the measured motion data of the user when performing the target motion in the next collection into the customized motion recognition model to obtain and output the measured count value.

7. The method according to claim 1, characterized in that, After determining the customized action recognition model, the method further includes: Receive the measured motion data of the user when performing the target action movement, which will be collected by the smart wearable device next time; The measured motion data is input into the customized motion recognition model to obtain the measured count value; The measured count value is sent to the smart wearable device so that the smart wearable device can output the measured count value.

8. The method according to claim 6 or 7, characterized in that, After the smart wearable device outputs the measured count value, the method further includes: Receive a target count value sent by the smart wearable device; wherein the target count value is determined based on the user's input to the smart wearable device; If the target count value and the measured count value are not equal, the customized motion recognition model is modified so that the modified measured count value obtained by the modified customized motion recognition model from the measured motion data is equal to the target count value.

9. A server comprising a memory and a processor, the memory for storing executable instructions; the processor for operating under the control of the instructions to perform the method as claimed in any one of claims 1 to 8.

10. A motion counting system, comprising a smart wearable device and the server of claim 9, wherein the smart wearable device is used to collect motion data of a user during historical motion processes and send it to the server.

Images