Throwing action recognition method, device, equipment, system and storage medium

Abstract

The invention discloses a throwing action recognition method, device, equipment and system and a storage medium, and belongs to the technical field of computers. The method comprises the following steps: collecting motion data of a hand of a user; candidate throwing data is determined from the motion data, the candidate throwing data including motion data generated within the first period of time. And screening out extreme value data from the candidate throwing data. And then, according to the generation moment of the extreme value data and a preset moment in the first time period, determining whether the user throws in the first time period or not. According to the scheme, the position of the extreme value data in the first time period is judged by screening the extreme value data and judging whether the generation moment of the extreme value data is matched with the preset moment in the first time period or not, and then whether the throwing action occurs or not is judged; compared with a scheme of singly comparing the acceleration to judge whether the throwing action occurs or not, the accuracy of recognizing the throwing action is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, in particular to a casting action recognition method, device, equipment, system and storage medium. BACKGROUND

[0002] In fishing activities, a user needs to record actions in a fishing process, so as to analyze the fishing process of the day after the fishing activities are completed, or to facilitate the sharing of the fishing process between users. The action records in the fishing process include casting, hooking fish, collecting line, and hooking fish, etc., wherein the record of the casting action is particularly important, which can best reflect the fishing strategy of the user.

[0003] In the current technology, a motion sensor is usually used to record the casting action in fishing. The motion sensor is used to collect acceleration in the casting process, and it is determined that the casting action occurs when the acceleration is greater than a threshold. The casting action is determined only by the size of the acceleration, which is prone to misjudgment, resulting in low accuracy of casting action determination. SUMMARY

[0004] The present application provides a casting action recognition method, device, equipment, system and storage medium, which improves the accuracy of casting action recognition. The technical solution is as follows:

[0005] In a first aspect, a casting action recognition method is provided, the method comprising: collecting motion data of a user's hand; determining candidate casting data from the motion data; determining whether the user casts in a first time period according to a time when extreme value data of the candidate casting data occurs and a preset time in the first time period; wherein the candidate casting data includes motion data occurring in the first time period.

[0006] In a second aspect, a casting action recognition device is provided, the device comprising: a memory and a processor; the memory is used to store a computer program; the processor is used to execute the computer program to realize the method of the first aspect.

[0007] In a third aspect, a wearable device is provided, the wearable device comprising a motion data collection device and a casting action recognition device as described in the second aspect; the wearable device is worn on the user's hand.

[0008] In a fourth aspect, a method for recognizing a throwing action is provided. The method is applied to a system for recognizing a throwing action, which includes a first smart device and a second smart device in communication connection with the first smart device. The method includes: controlling the first smart device to collect motion data of a user's hand; determining candidate throwing data from the motion data; controlling the second smart device to determine whether the user is throwing in a first time period according to a time when extreme value data of the candidate throwing data is generated and a preset time in the first time period; and wherein the candidate throwing data includes motion data generated in the first time period.

[0009] In a fifth aspect, a system for recognizing a throwing action is provided. The system includes a first smart device, a second smart device in communication connection with the first smart device, a memory and a processor. The memory is configured to store a computer program. The processor is configured to execute the computer program to implement the method of the fourth aspect.

[0010] In a sixth aspect, a computer readable storage medium is provided. The computer readable storage medium stores a computer program. The computer program is executed by a processor to implement the method of the first aspect.

[0011] In a seventh aspect, a computer program product including instructions which, when executed on a computer, cause the computer to carry out the method of the first aspect.

[0012] The embodiment of the present application provides a method, device, equipment, system and storage medium for cast motion recognition. According to the scheme provided by the present application, motion data of a user's hand is collected; candidate cast data is determined from the motion data, and the candidate cast data includes motion data generated in a first period. In the scheme, the entire hand motion process is divided into a plurality of first periods in advance, and the length of the first period is determined according to the duration of the cast motion in the fishing process, so as to ensure that the first period can cover the cast process and does not affect the cast motion recognition efficiency. The motion data included in adjacent two first periods can overlap, and the motion data generated in the first period is taken as the candidate cast data. Extreme value data is selected from the candidate cast data. Then, whether the user casts in the first period is determined according to the generation time of the extreme value data and a preset time in the first period. The preset time in the first period can be near the middle position of the first period. If the generation time of the extreme value data does not match the preset time in the first period, it indicates that the extreme value data is located at the edge position of the current first period, and it indicates that the motion data is still in the rising or falling trend in the adjacent next first period, and therefore, it cannot be taken as the cast point, and the user does not have the cast motion in the current first period. If the generation time of the extreme value data matches the preset time in the first period, it indicates that the extreme value data is located near the middle position in the current first period, and it indicates that the extreme value data is the point with the cast intention in the current first period, and therefore, it can be taken as the cast point, and the user has the cast motion in the current first period. In the scheme, the extreme value data is selected, and whether the extreme value data is located in the first period in which the extreme value data is located is determined by judging whether the generation time of the extreme value data matches the preset time in the first period, and then whether the cast motion occurs is determined. Compared with the scheme of determining whether the cast motion occurs by comparing the size of the acceleration alone, the accuracy of the cast motion recognition is improved. Moreover, the cast motion recognition is performed by dividing the process into a plurality of first periods, and the motion data of the entire cast process is avoided, and the cast motion recognition efficiency is improved. BRIEF DESCRIPTION OF DRAWINGS

[0013] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0014] Figure 1 is a flowchart of a cast motion recognition method provided by the embodiment of the present application;

[0015] Figure 2 is a flowchart of another cast motion recognition method provided by the embodiment of the present application;

[0016] Figure 3 is a flow chart of still another method for recognizing a throwing action provided by an embodiment of the present application;

[0017] Figure 4 is a flow chart of still another method for recognizing a throwing action provided by an embodiment of the present application;

[0018] Figure 5 is a flow chart of still another method for recognizing a throwing action provided by an embodiment of the present application;

[0019] Figure 6 is a flow chart of still another method for recognizing a throwing action provided by an embodiment of the present application;

[0020] Figure 7 is a flow chart of still another method for recognizing a throwing action provided by an embodiment of the present application;

[0021] Figure 8 is a structural schematic diagram of a device for recognizing a throwing action provided by an embodiment of the present application;

[0022] Figure 9 is a structural schematic diagram of a wearable device provided by an embodiment of the present application;

[0023] Figure 10 is a structural schematic diagram of a system for recognizing a throwing action provided by an embodiment of the present application. DETAILED DESCRIPTION

[0024] In order to make the objectives, technical solutions and advantages of the present application clearer, the embodiments of the present application will be further described in detail below with reference to the accompanying drawings.

[0025] It should be understood that the "multiple" mentioned in the present application refers to two or more than two. In the description of the present application, unless otherwise specified, " / " represents the meaning of or, for example, A / B can represent A or B; "and / or" in the present application only represents the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B, which means that there are three cases of A alone, A and B together, and B alone. In addition, in order to clearly describe the technical solutions of the present application, the same items or similar items with basically the same functions and effects are distinguished by using "first", "second", etc. The skilled in the art can understand that "first", "second", etc. do not limit the quantity and execution order, and "first", "second", etc. also do not necessarily mean different.

[0026] Before the embodiments of the present application are explained in detail, the application scenarios and related technologies of the embodiments of the present application will be described.

[0027] The cast action recognition method provided by the embodiments of the present application can be applied to fishing activities, and the fishing activities can be lure fishing. At present, the recording of the cast action in lure fishing is largely dependent on manual completion by the user, which causes great inconvenience to the user during the fishing process in lure fishing, and manual recording is prone to omissions.

[0028] In addition, the acceleration during the cast process can also be collected by using a motion sensor, and the cast action is determined to occur when the acceleration satisfies the condition of being greater than a threshold. This method has a single determination condition and is prone to misjudgment, so that the accuracy of the cast action determination is not high.

[0029] In the current technology, other cast action recognition methods can also be used, and the motion data of the entire fishing process is calculated. This determination method needs to continuously process the motion data during the entire fishing process and continuously determine whether the cast action occurs. This method has high requirements for the computing power of the processor and the power supply unit, and it is difficult to support the function of recording the cast action in a fishing activity for a long time. The power consumption is large, and the cast action recognition efficiency is low.

[0030] The cast action recognition method provided by the embodiments of the present application belongs to the field of automatic fishing action recognition and can be used to recognize the cast action in fishing activities.

[0031] The embodiments of the present application provide a cast action recognition method, as shown in Figure 1 Figure 1 is a flowchart of a cast action recognition method provided by the embodiments of the present application. The cast action recognition method comprises the following steps:

[0032] S101, collect motion data of a user's hand.

[0033] In the embodiments of the present application, the motion data collection device (i.e., the motion sensor) can be worn on the user's hand to collect the motion data of the user's hand. For example, the motion data collection device can be integrated on a wearable device (e.g., a smart watch, a smart bracelet, a ring). The motion data collection device (i.e., the motion sensor) can also be fixed on a fishing rod or a fishing reel to collect the motion data reflecting the user's hand. For example, the motion data collection device can be a grip ball provided on the fishing rod.

[0034] ​The motion sensor can include an accelerometer. The accelerometer is configured to collect acceleration data during the throwing process. The acceleration data can include acceleration data of three axes, i.e., X-axis acceleration data, Y-axis acceleration data, and Z-axis acceleration data. The sampling frequency of the accelerometer can be any one of 10 HZ, 20 HZ, 25 HZ, 50 HZ, 80 HZ, 100 HZ, 200 HZ, 500 HZ, and 1000 HZ, or a range between any two of the above values, for example, 30 HZ, which is not limited in the embodiments of the present application.

[0035] The motion sensor can also include a gyroscope. The gyroscope is configured to collect angular velocity data during the throwing process. The angular velocity data can include angular velocity data of three axes, i.e., X-axis angular velocity data, Y-axis angular velocity data, and Z-axis angular velocity data. The three axes represent a pitch axis, a roll axis, and a yaw axis, respectively. The sampling frequency of the gyroscope can be any one of 10 HZ, 20 HZ, 25 HZ, 50 HZ, 80 HZ, 100 HZ, 200 HZ, 500 HZ, and 1000 HZ, or a range between any two of the above values, for example, 30 HZ, which is not limited in the embodiments of the present application. The sampling frequency of the gyroscope can be consistent with the sampling frequency of the accelerometer.

[0036] In some embodiments, the motion sensor can include at least one of an accelerometer and a gyroscope, and the motion data can include at least one of acceleration or angular velocity corresponding to the user's hand. The motion sensor can be a nine-axis sensor or a six-axis sensor.

[0037] In S102, candidate throwing data is determined from the motion data. The candidate throwing data includes motion data generated in a first time period.

[0038] The length of the first time period is determined according to the duration of the throwing action in the throwing process.

[0039] In the embodiments of the present application, the point of the last time of backwardly pulling the rod to forwardly exerting before the bait is thrown out in the fishing process is the casting point in the casting process, and the casting point needs to be found in the present example. The whole hand movement process is divided into a plurality of first time periods, the movement data included by adjacent two first time periods can be overlapped, and extreme value data is filtered out from the candidate casting data in sequence. That is, the movement data is arranged in sequence according to the sampling time, a first time window with a time length of the first time period is set, and the first time window can slide according to a first step length. In another way, a first fixed window with a time length of the first time period is set, and the movement data slides according to the first step length in sequence according to the sampling time. Then, according to the movement data corresponding to the plurality of sampling time points falling in the first time window (or the first fixed window), the extreme value data is determined, and the extreme value data can be the maximum amplitude of the movement data generated in the first time window (or the first fixed window).

[0040] In some embodiments, the movement data can include the acceleration corresponding to the hand of the user, and correspondingly, the extreme value data of the candidate casting data includes the maximum acceleration in the acceleration acquired in the first time period. The movement data can include the angular velocity corresponding to the hand of the user, and correspondingly, the extreme value data of the candidate casting data includes the maximum angular velocity in the angular velocity acquired in the first time period. The movement data can include the acceleration and the angular velocity corresponding to the hand of the user, and correspondingly, the extreme value data of the candidate casting data includes the maximum value of the weighted average value determined by the acceleration and the angular velocity acquired in the first time period, and the acceleration and the angular velocity are collected at the same time stamp.

[0041] Taking the first time period as the sliding first time window as an example, three examples are listed to explain the process of selecting the extreme value data of the candidate casting data.

[0042] The first example, the throwing point can be the maximum force point in the throwing process throughout the whole throwing process. The maximum force point indicates the time when the force is the maximum. According to Newton's second law, the point with the maximum acceleration is the point with the maximum force (i.e., the maximum force point). Therefore, the maximum force point in the throwing process can be found by detecting the magnitude of the acceleration. Based on this, the maximum force point in the throwing process can be found by calculating the magnitude of the acceleration in real time. The identification process of the maximum force point in the throwing process (i.e., the maximum value point of the acceleration magnitude in the throwing process) can be implemented in the following way: a sliding time window (i.e., a first time window) is created, the acceleration data of the three axes of the accelerometer in the first time window is collected, the acceleration magnitude at each collection time in the first time window is calculated, the acceleration magnitude can be the weighted average of the acceleration data of the three axes of the accelerometer, and the maximum value point (i.e., extreme value data) of the acceleration magnitude of the first time window is selected. Further, it is determined whether the maximum value point of the acceleration magnitude of the first time window is at the preset time of the first time window, so as to determine whether the first time window includes the maximum force point (i.e., the throwing point).

[0043] The second example, the throwing point can be the maximum turning point in the throwing process throughout the whole throwing process. The maximum turning point indicates the time when the direction change is the maximum (i.e., the point with the most rotation). Based on this, the maximum turning point in the throwing process can be found by calculating the change of the angular velocity in real time. The identification process of the maximum turning point in the throwing process can be implemented in the following way: a sliding time window (i.e., a first time window) is created, the angular velocity data of the three axes of the gyroscope in the first time window is collected, the angular velocity magnitude at each collection time in the first time window is calculated, the angular velocity magnitude can be the weighted average of the angular velocity data of the three axes of the gyroscope, and the maximum value point (i.e., extreme value data) of the angular velocity magnitude of the first time window is selected. Further, it is determined whether the maximum value point of the angular velocity magnitude of the first time window is at the preset time of the first time window, so as to determine whether the first time window includes the maximum turning point (i.e., the throwing point).

[0044] In a third example, the release point can be the maximum point of the weighted average of the force point and the direction change point during the release process. In an ideal state, the maximum force point and the maximum turning point are the same point. The motion sensor includes an accelerometer and a gyroscope, which can be a six-axis sensor or a nine-axis sensor. The accelerometer collects acceleration data of three axes, calculates the acceleration amplitude, and the acceleration amplitude is time-stamped. The gyroscope collects angular velocity data of three axes, calculates the angular velocity amplitude, and the angular velocity amplitude is time-stamped. Then, the weighted average of the acceleration amplitude and the angular velocity amplitude of the same timestamp is calculated. The maximum point of the weighted average in the first time window (i.e., the extreme value data) is selected. Then, it is determined whether the maximum point of the weighted average in the first time window is at the preset time of the first time window, so as to determine whether the first time window includes the release point.

[0045] Further, in determining whether the first time window includes the release point, any two or more of the first example, the second example, and the third example can be combined. When the results of each example in the combination scheme are the release point, it is determined that the first time window includes the release point, and the recognition results can be corrected with each other, thereby improving the accuracy of the release point recognition result.

[0046] In this example, not only the acceleration amplitude but also the angular velocity amplitude is considered in determining the release point, thereby improving the accuracy of the release point recognition result.

[0047] It should be noted that the proportions of the acceleration amplitude and the angular velocity amplitude can change when the acceleration and the angular velocity are weighted and averaged, for example, the proportion of the acceleration amplitude is 3 / 4, and the proportion of the angular velocity amplitude is 1 / 4; or the proportion of the acceleration amplitude is 1 / 4, and the proportion of the angular velocity amplitude is 3 / 4; or the proportion of the acceleration amplitude is 3 / 5, and the proportion of the angular velocity amplitude is 2 / 5. The embodiments of the present application do not limit this.

[0048] The execution subject of the release action recognition method provided by the embodiments of the present application can be a release action recognition device, which includes a memory and a processor. The calculation of the acceleration amplitude and the angular velocity amplitude described above can be performed by a release point determination unit in the release action recognition device. The motion data corresponding to each of the plurality of sampling time points in the first time window is read by the release point determination unit in the release action recognition device, for determining the release point in the release process. The release point determination unit can be an operation module packaged in the processor, or the processor itself, and the embodiments of the present application do not limit this.

[0049] The length of the first time period (i.e., the first time window or the first fixed window) is determined according to the maximum duration of the casting action in the fishing process, so as to ensure that the first time period can cover the casting process and does not affect the recognition efficiency of the casting action. The first step length is less than half the length of the first time window or the first fixed window.

[0050] Generally, the maximum duration of the casting action in the fishing process is about 8 seconds. The above motion data is read by the casting point determination unit in a sliding time window, which can be 2 seconds, 3 seconds, 4 seconds, 5 seconds, 6 seconds, 7 seconds, 8 seconds, 9 seconds, 10 seconds, etc. Alternatively, the time window can be any time width from 1 second to 10 seconds. For example, a too short time window cannot cover the entire casting process, and a too long time window will affect the recognition efficiency of the casting action and also increase the computing power requirement of the casting point determination unit and the subsequent casting action recognition unit, and increase the energy consumption. Therefore, the sliding time window can be set to 4 seconds or 5 seconds, so as to balance the accuracy and efficiency of the casting action recognition, and reduce the computing power and energy consumption of the casting action recognition device while ensuring the accuracy of the casting action recognition.

[0051] The sliding step length (i.e., the first step length) can be an integer multiple of the sampling point interval. For example, the sampling frequency of the motion sensor is 10HZ, i.e., 10 samples per second, and the sliding step length can be set to 0.1s (corresponding to one sampling point), 0.2s, 0.3s, 0.5s, 1s, etc. The sliding step length is less than half the length of the time window, so as to improve the operation efficiency on the basis of ensuring the data coverage.

[0052] S103, determining whether the user casts in the first time period according to the time when the extreme value data of the candidate casting data is generated and the preset time in the first time period.

[0053] In the embodiments of the present application, the extreme value data of the candidate throwing data may or may not be a throwing point. By judging whether the time when the extreme value data of the candidate throwing data is generated matches the preset time in the first time period, that is, judging whether the position of the extreme value data in the first time period is located in the preset interval of the first time period, whether the first time period includes a throwing point is determined. If the extreme value data is located at the edge position of the current first time period, it indicates that the motion data is still in an upward or downward trend in the adjacent next first time period, therefore, it cannot be regarded as a throwing point, and it is determined that the user does not have a throwing action in the current first time period. If the extreme value data is located at a position close to the middle of the current first time period, it indicates that the extreme value data is a point with throwing intention in the current first time period, therefore, it can be regarded as a throwing point. It is determined that the user has a throwing action in the current first time period, or, according to the motion data corresponding to each sampling time in the current first time period, whether the user has a throwing action is further judged by a throwing action recognition model.

[0054] Taking the first time period as a sliding first time window, the extreme value data of the candidate throwing data is the maximum amplitude in the first time window, whether the position of the sampling time corresponding to the maximum amplitude (that is, the time when the extreme value data is generated) in the first time window is located in the preset time of the first time window is judged. If the maximum amplitude is not located in the preset time, the maximum amplitude is not a throwing point, the first time window is slid backward (for example, the sampling time point of the motion sensor is slid backward by one), whether the extreme value data in the slid first time window is located in the preset time is judged, until the extreme value data in the slid first time window is located in the preset time. If the maximum amplitude is located in the preset time, the extreme value data is regarded as a throwing point, and it is determined that the user has a throwing action, or, according to the motion data corresponding to each sampling time in the time window, whether the user has a throwing action is further judged by a throwing action recognition model.

[0055] According to the scheme provided in the present application, motion data of a user's hand is collected; candidate casting data is determined from the motion data, the candidate casting data including motion data generated in a first time period. The scheme divides the entire hand motion process into a plurality of first time periods in advance, the length of the first time period being determined according to the duration of the casting action in the fishing process, so as to ensure that the first time period can cover the casting process and does not affect the casting action recognition efficiency. The motion data included in two adjacent first time periods can overlap, and the motion data generated in the first time period is taken as the candidate casting data. Extreme value data is screened from the candidate casting data. Then, according to the generation time of the extreme value data and a preset time in the first time period, it is determined whether the user casts in the first time period. The preset time in the first time period can be near the middle position of the first time period. If the generation time of the extreme value data does not match the preset time in the first time period, it indicates that the extreme value data is located at the edge position of the current first time period, and it indicates that the motion data is still in an upward or downward trend in the adjacent next first time period, and therefore, it cannot be taken as a casting point, and the user does not have a casting action in the current first time period. If the generation time of the extreme value data matches the preset time in the first time period, it indicates that the extreme value data is located near the middle position of the current first time period, and it indicates that the extreme value data is a point with casting intention in the current first time period, and therefore, it can be taken as a casting point, and the user has a casting action in the current first time period. The scheme determines the position of the extreme value data in the first time period in which the extreme value data is located by screening the extreme value data and by judging whether the generation time of the extreme value data matches the preset time in the first time period, and then determines whether a casting action occurs. Compared with the scheme of judging whether a casting action occurs by comparing the size of the acceleration alone, the scheme improves the accuracy of casting action recognition. Moreover, the scheme divides the casting action recognition into a plurality of first time periods, avoids calculating the motion data of the entire casting process, and improves the casting action recognition efficiency.

[0056] In some embodiments, S102 described above can be implemented in the following manner. In a case where it is detected that the motion data generated in the first time period meets the preset condition, the motion data generated in the first time period is determined as the candidate casting data.

[0057] The embodiment of the present application is to divide the motion data into multiple first time periods in advance, and not all motion data generated in each first time period can be used as candidate throwing data. The motion data in some first time periods is relatively flat, for example, the angular velocity is small and / or the acceleration is small, although there are some fluctuations, but the overall trend of the motion data indicates that there is no throwing point in this first time period. Therefore, the embodiment of the present application also screens each first time period, and in the case that the motion data generated in the first time period meets the preset condition, for example, the angular velocity reaches a certain threshold and / or the acceleration reaches a certain threshold, the overall trend indicates that the first time period may have a throwing point, and the motion data generated in the first time period is determined as the candidate throwing data.

[0058] By judging whether the motion data generated in the first time period meets the preset condition, the candidate throwing data with a throwing point is screened as much as possible, and the data that does not meet the preset condition is reduced to enter the subsequent judgment, that is, whether the generation time of the extreme value data of the candidate throwing data matches the preset time in the first time period, thereby improving the data processing efficiency.

[0059] In some embodiments, the motion data includes at least one of the acceleration and the angular velocity corresponding to the user's hand; the preset condition includes: the acceleration corresponding to the user's hand is greater than or equal to a preset acceleration threshold; and / or, the angular velocity corresponding to the user's hand is greater than or equal to a preset angular velocity threshold.

[0060] Example one, the motion data includes the acceleration corresponding to the user's hand, and in the case that the acceleration generated at each sampling time in the first time period is greater than or equal to a preset acceleration threshold, the acceleration generated in the first time period is determined as the candidate throwing data.

[0061] Example two, the motion data includes the angular velocity corresponding to the user's hand, and in the case that the angular velocity generated at each sampling time in the first time period is greater than or equal to a preset angular velocity threshold, the angular velocity generated in the first time period is determined as the candidate throwing data.

[0062] Example three, the motion data includes the acceleration and the angular velocity corresponding to the user's hand, and in the case that the acceleration generated at each sampling time in the first time period is greater than or equal to a preset acceleration threshold, and the angular velocity generated at each sampling time is greater than or equal to a preset angular velocity threshold, the motion data generated in the first time period is determined as the candidate throwing data.

[0063] The preset acceleration threshold can be appropriately set by those skilled in the art according to the actual situation, for example, 1g, 0.9g, 1.2g, etc., g represents the acceleration of gravity, 1g≈9.80m / s 2 , and correspondingly, 0.9g≈0.9×9.80m / s 2The preset angular velocity threshold value can be appropriately set by those skilled in the art according to actual conditions, for example, 20 rad / s (radians per second), 21 rad / s, 19 rad / s, and the like.

[0064] Further, the preset condition can further include that the acceleration corresponding to the hand of the user is greater than or equal to a preset acceleration threshold value and less than an upper acceleration threshold value, and / or the angular velocity corresponding to the hand of the user is greater than or equal to a preset angular velocity threshold value and less than an upper angular velocity threshold value.

[0065] The upper acceleration threshold value can be set to 120 g, 150 g, and the upper angular velocity threshold value can be set to 300 rad / s, 400 rad / s.

[0066] Generally, the acceleration corresponding to the hand of the user will not exceed the upper acceleration threshold value, and the angular velocity will not exceed the upper angular velocity threshold value, but in actual application, the motion data acquisition device can have acquisition errors, data noise, and the like. Therefore, the upper acceleration threshold value and the upper angular velocity threshold value can be set in the embodiments of the present application to reduce the processing of obvious error data and improve the data processing efficiency.

[0067] In some embodiments, the preset moment is any moment in a preset interval, a distance between a starting point of the preset interval and a lower boundary of the first time period is greater than or equal to one fourth of the first time period, and a distance between an ending point of the preset interval and an upper boundary of the first time period is greater than or equal to one fourth of the first time period.

[0068] In the embodiments of the present application, the distance between the preset moment and the lower boundary and the upper boundary of the first time period is greater than or equal to a preset time period, in other words, the preset moment is any moment in a preset interval deviating from a center point of the first time window by a preset distance.

[0069] For example, the preset interval can be [1 / 4, 3 / 4] or [1 / 3, 2 / 3] of the first time window. The preset moment can be a center point of the first time window, that is, 1 / 2 of the first time window, or the preset moment can be 1 / 4 of the first time window, or the preset moment can be 3 / 4 of the first time window, and the like. The embodiments of the present application do not limit this. The preset moment can be approximately at a middle position of the first time period. The preset moment can be a middle point (that is, 1 / 2) of the first time period, 1 / 3, 2 / 3, 1 / 4, or 3 / 4, and the like.

[0070] For example, when the first time period is 4s, the preset time can be 1 / 4 of the first time period, i.e., the distance between the preset time and the lower boundary of the first time period is 1s, and the distance between the preset time and the upper boundary of the first time period is 3s. The preset time can also be 1 / 2 of the first time period, i.e., the distance between the preset time and the lower boundary of the first time period is 2s, and the distance between the preset time and the upper boundary of the first time period is 2s. The preset time can also be 3 / 4 of the first time period, i.e., the distance between the preset time and the lower boundary of the first time period is 1s, and the distance between the preset time and the upper boundary of the first time period is 3s. In this way, if the throwing point is detected, the motion data 1s before and after the throwing point are both within the first time period. The preset time period can also be set to 1s, and the present application does not limit the preset time period.

[0071] In some embodiments, the above Figure 1 In some embodiments, the above

[0072] In the present application, the extreme value data of the candidate throwing data is an extreme value data in the motion data corresponding to each sampling time included in the first time period, and the generation time of the extreme value data indicates the sampling time of the extreme value data. Whether the position of the generation time in the first time period is located at the preset time within the first time period is determined. If the generation time is located at the preset time, it is indicated that the generation time matches the preset time, the generation time is the time when the key point of throwing appears, the extreme value data is the point with throwing intention in the first time period, and the extreme value data is determined as the throwing point. It is determined that the user throws in the first time period. If the generation time is not located at the preset time, it is indicated that the generation time does not match the preset time, the generation time can be located at the edge position of the first time period, and the motion data is still in the rising or falling trend in the adjacent next first time period. The extreme value data is not the throwing point, and it is determined that the user does not throw in the first time period.

[0073] For example, the first time period is a sliding first time window, and three examples of determining whether the extreme value data is the throwing point are listed.

[0074] In the first example, the motion data includes the acceleration corresponding to the hand of the user, and the corresponding extreme value data is the maximum value point of the acceleration amplitude in the first time window. Whether the maximum value point of the acceleration amplitude in the first time window is located at a specific position (i.e., the preset time) of the first time window is determined. For example, the specific position is any position in [1 / 4, 3 / 4] of the first time window, e.g., 1 / 2. If the maximum value point of the acceleration amplitude in the first time window is exactly located at 1 / 2 of the first time window, it is determined that the maximum value point of the acceleration amplitude in the first time window is the maximum force point in the throwing process, i.e., the extreme value data is the throwing point.

[0075] In the second example, the motion data includes angular velocity corresponding to the user's hand, and the corresponding extreme value data is the maximum point of the angular velocity amplitude in the first time window. It is determined whether the maximum point of the angular velocity amplitude in the first time window is located at a specific position of the first time window. Taking the specific position as any position in [1 / 3, 2 / 3] of the first time window, for example, 2 / 3, if the maximum point of the angular velocity amplitude in the first time window is exactly located at 2 / 3 of the first time window, it is determined that the maximum point of the angular velocity amplitude is the maximum turning point in the casting process, that is, the extreme value data is the casting point.

[0076] In the third example, the motion data includes acceleration and angular velocity corresponding to the user's hand, and the corresponding extreme value data is the maximum point of the weighted average of the acceleration and angular velocity amplitude in the first time window. It is determined whether the maximum point of the weighted average in the first time window is located at a specific position of the first time window. Taking the specific position as any position in [2 / 5, 4 / 5] of the first time window, for example, 3 / 5, if the maximum point of the weighted average in the first time window is exactly located at 3 / 5 of the first time window, it is determined that the maximum point of the weighted average in the first time window is the maximum point in the casting process, that is, the extreme value data is the casting point.

[0077] In this example, the extreme value data in the first period is selected, and it is determined whether the generation time of the extreme value data is located at a preset time in the first period in which the extreme value data is located, to determine whether it is a casting point, and further to determine whether a casting action occurs. Compared with the scheme of determining whether a casting action occurs by comparing the size of the acceleration alone, the accuracy of casting action recognition is improved.

[0078] In the second example, the motion data includes angular velocity corresponding to the user's hand, and the corresponding extreme value data is the maximum point of the angular velocity amplitude in the first time window. It is determined whether the maximum point of the angular velocity amplitude in the first time window is located at a specific position of the first time window. Taking the specific position as any position in [1 / 3, 2 / 3] of the first time window, for example, 2 / 3, if the maximum point of the angular velocity amplitude in the first time window is exactly located at 2 / 3 of the first time window, it is determined that the maximum point of the angular velocity amplitude is the maximum turning point in the casting process, that is, the extreme value data is the casting point.

[0079] In the embodiments of the present application, it is determined whether the position of the generation time in the first period is located at a preset time in the first period. If the generation time is located at the preset time, it is indicated that the generation time matches the preset time, the extreme value data is a point with a casting intention in the first period, and the extreme value data is taken as a casting point. The motion data corresponding to each of the plurality of sampling times in the first period is input into a casting action recognition model, and it is determined whether the user casts in the first period by the casting action recognition model. Since the occurrence of the casting point in the actual fishing process is not necessarily a casting action, for example, some smoking actions may have a casting point, but are not a casting action. Therefore, in this example, after the casting point is determined, the motion data corresponding to each of the plurality of sampling times in the first period is identified by the casting action recognition model, so as to improve the accuracy of casting action recognition.

[0080] It should be noted that the first time period is a sliding first time window, and when the motion data is processed, one or more first time periods meeting the condition (i.e., the generation time matches the preset time) can be screened out. In this example, by screening out one or more first time periods, some data that is obviously not a throwing action (e.g., noise data without a throwing point) can be filtered out. The throwing action recognition model is used to identify the motion data included in the one or more screened first time periods to obtain a throwing action recognition result. In one way, the throwing action recognition model identifies the motion data included in each of the screened first time periods to obtain an identification result for each first time period, and comprehensively considers multiple identification results to take the result corresponding to the maximum probability value in the identification result as the final identification result (whether it is a throwing action or not). In another way, the throwing action recognition model identifies the motion data included in the multiple screened first time periods to obtain an identification result (whether it is a throwing action or not).

[0081] In the embodiments of the present application, the preset time is the midpoint of the first time period, 1 / 4 of the first time period, or 3 / 4 of the first time period, etc. In this way, the throwing effective information (motion sensor information in the throwing process) before and after the throwing point in the throwing process can be included, and the distortion of the data entering the throwing action recognition model due to the lack of effective information in the throwing process can be reduced, thereby improving the accuracy of throwing action recognition.

[0082] In this example, it is detected whether the generation time matches the preset time in the first time period, that is, whether the maximum value point of the acceleration amplitude in the first time window is at a specific position of the first time window, or whether the maximum value point of the angular velocity amplitude in the first time window is at a specific position of the first time window, or whether the maximum value point of the weighted average of the acceleration amplitude and the angular velocity amplitude in the first time window is at a specific position of the first time window. The number of judgments of the subsequent throwing action recognition model can be reduced, the same segment of motion data can be prevented from entering the throwing action recognition model multiple times in the sliding window process, and the calculation power consumption is reduced. Moreover, the throwing effective information (e.g., acceleration data and / or angular velocity data) before and after the maximum force point in the throwing process can be included, and the accuracy of throwing action recognition can be improved.

[0083] In the embodiments of the present application, the collected motion data can be regarded as a curve varying with time, and the curve has a peak value (the peak value is the throwing point), and the curve slides along time. A fixed time window is set, and when the peak value reaches the position of 1 / 4 (or 1 / 2, or 3 / 4) of the time window, the curve is no longer slid, and the data in the fixed time window is taken out for subsequent throwing action recognition. In this way, one or more throwing points can be screened out. The motion data included in the time window of the throwing points is input into the throwing action recognition model to judge the throwing action.

[0084] For example, the first time window is 4s, and the preset time is set to any position in [1 / 4, 3 / 4] of the first time window, for example, 1 / 2. After the maximum value point is determined, if the maximum value point is located at the position of 1 / 2 of the time window, the maximum value point is taken as the throwing point. In this way, the data 1s before and after the throwing point can be included. The subsequent throwing action recognition process is performed.

[0085] In the embodiments of the present application, it is assumed that the first time window slides by one sampling point each time (the first step is one sampling point), the sampling frequency is 25HZ, and there are 25 sampling points in 1s. Each sampling point is selected many times by sliding, and the preset time is set. When the maximum value point is located at the preset time, the motion data in the time window of the maximum value point can be input into the throwing action recognition model, so as to avoid that the same motion data enters the throwing action recognition model many times in the sliding window process, and to reduce the consumption of computing power.

[0086] The throwing action recognition model can be a machine learning model, for example, a neural network model (Neural Networks, NN), including but not limited to a convolutional neural network (Convolutional Neural Networks, CNN), a recurrent neural network (Recurrent Neural Network, RNN), etc. The throwing action recognition model is a pre-trained binary classification model using a large number of motion data samples, and has the function of identifying whether a certain period of time has multiple sampling time points corresponding to the motion data, and whether it is a throwing action, wherein the motion data samples include positive samples with a label of a winding action, and negative samples with a label that is not a winding action. The structure of the throwing action recognition model is not limited in the embodiments of the present application, as long as the model can identify whether the motion data represents a throwing action.

[0087] It should be noted that the execution subject of the throwing action recognition method provided in the embodiments of the present application can be a throwing action recognition device, and the throwing action recognition device includes a memory and a processor. The above determining whether it is a throwing action according to the motion data corresponding to each of the plurality of sampling time points in the first time period of the throwing point can be performed by a throwing action recognition unit in the throwing action recognition device. The throwing action recognition unit can be an operation module packaged in the processor, or can be the processor itself, and the embodiments of the present application do not limit this.

[0088] In the embodiments of the present application, the throwing point in the throwing process is determined in advance, so that the motion data that needs to enter the time window of the throwing action recognition unit is determined, the motion data of each time window is avoided to be calculated, the same segment of motion data is avoided to enter the throwing action recognition model multiple times in the sliding window process, the calculation power consumption is reduced, the calculation power requirement of the device is reduced, and the battery consumption is reduced. Since the throwing point in the throwing process only needs to obtain the acceleration amplitude and the angular velocity amplitude of the motion sensor through simple calculation, and the throwing action recognition unit needs to perform one-dimensional and two-dimensional transformation processing on the motion data, therefore, in the whole throwing action recognition process, the calculation power and energy consumption required by the throwing action recognition model are the largest. In the present example, the use frequency of the throwing action recognition model in the throwing action recognition process is as low as possible, the calculation power requirement of the device is reduced, and the power consumption is reduced.

[0089] In some embodiments, the step of determining whether the user throws in the first time period according to the motion data included in the first time period by the throwing action recognition model in Example Two above can be implemented in the following manner. Obtain a first feature value corresponding to the motion data included in the first time period; the first feature value includes statistical features and / or time domain features; determine whether the user throws in the first time period according to the first feature value by the throwing action recognition model.

[0090] In the embodiments of the present application, the motion data corresponding to each of the plurality of sampling time points in the first time period is subjected to frequency domain transformation to obtain a plurality of frequency domain transformation values; one-dimensional statistical features are determined according to the plurality of motion data and the plurality of frequency domain transformation values; two-dimensional statistical features and time domain features are determined according to the one-dimensional statistical features; and the one-dimensional statistical features, the two-dimensional statistical features and the time domain features are taken as the first feature values. The feature values in the present application can also be referred to as feature quantities.

[0091] The motion data corresponding to each sampling time in the qualified first time window is counted, wherein the qualified first time window refers to a time window in which a maximum turning point and / or a maximum force point (i.e., extreme value data) appear, and the maximum turning point and / or the maximum force point are located at a position in the first time window that meets a preset time (i.e., the time when the extreme value data in the first time period is generated matches the preset time in the first time period). The motion data in the qualified first time window is processed, and the feature values of the motion data corresponding to each sampling time are calculated respectively, including one-dimensional feature values, two-dimensional feature values, etc.

[0092] Taking the motion data including acceleration and angular velocity corresponding to the user's hand as an example, the motion data corresponding to each sampling time in the first time window includes X-axis acceleration, Y-axis acceleration, Z-axis acceleration, X-axis angular velocity, Y-axis angular velocity, and Z-axis angular velocity. The one-dimensional feature values include at least one of the following one-dimensional data: three-axis acceleration amplitude and its first-order difference, three-axis angular velocity amplitude and its first-order difference, weighted average of three-axis acceleration amplitude and three-axis angular velocity amplitude, frequency domain transform value of the weighted average (e.g., realized by Laplace transform, Fourier transform, or z transform), root mean square energy of the weighted average, attitude value (including roll angle, pitch angle, and yaw angle), frequency spectrum of the roll angle, pitch angle, and yaw angle, etc. The two-dimensional feature values include at least one of the following two-dimensional statistical features: mean, median, percentile, variance, standard deviation, interquartile range (IQR), ratio of maximum value to mean value, coefficient of variation (i.e., mean value / standard value), skewness, kurtosis, and approximate entropy of the above one-dimensional data; and at least one of the following time domain features: peak value, valley value, peak-valley difference, peak position, valley position, peak-valley interval zero-crossing rate, full width at half maximum, peak-valley interval, peak value / width, left-right area ratio of peak value, etc.

[0093] In the embodiments of the present application, the first feature values are taken as input data, and a throwing action recognition model formed through machine learning is used to automatically identify whether a throwing action occurs. After finding the throwing point, the motion data corresponding to each sampling time in the first time period in which the throwing point is located is further counted to obtain the first feature values, and then the throwing action recognition model is used to perform throwing action recognition according to the first feature values. Compared with the scheme of directly judging whether the user has a throwing action according to the throwing point, the accuracy of throwing action recognition is improved.

[0094] In the following, an exemplary application of the embodiments of the present application in an actual application scenario will be described.

[0095] An example is provided to illustrate the automatic identification of the casting action in the fishing process. As shown in Figure 2 Figure 2 is another flowchart of the casting action identification method provided by the embodiment of the present application.

[0096] S201, the motion sensor collects motion data.

[0097] The motion sensor can include an accelerometer and a gyroscope, and the motion data includes acceleration and angular velocity corresponding to the user's hand.

[0098] S202, determine the casting point according to the motion data.

[0099] The motion data is arranged in order of sampling time, a first time window with a first time period is set, and the first time window can be slid according to a first step. According to the motion data corresponding to each of the plurality of sampling time points in the first time window, determine the extreme value data. Determine whether the sampling time corresponding to the extreme value data is located at the preset time of the first time window, if the extreme value data is not located at the preset time, the extreme value data is not the casting point, slide the first time window, and determine whether the extreme value data in the slid first time window is located at the preset time, until the extreme value data in the slid first time window is located at the preset time, and the casting point is obtained. If the extreme value data is located at the preset time, the extreme value data is taken as the casting point.

[0100] S203, calculate the first characteristic value according to the motion data corresponding to each of the plurality of collection time points in the time window where the casting point is located.

[0101] S204, identify whether the user has a casting action according to the first characteristic value.

[0102] According to the motion data corresponding to each of the plurality of sampling time points in the time window where the casting point is located, calculate the first characteristic value. Input the first characteristic value into the casting action identification model, and determine whether the user has a casting action through the casting action identification model.

[0103] As described above Figure 2 ​As shown, the execution subject of the casting action recognition method can be a casting action recognition device. S201 and S202 can be executed by a casting point determination unit, and S203 and S204 can be executed by a casting action recognition unit. The user starts the casting action recognition device, and if the casting action recognition device is a wearable device (for example, a smart watch), the user actively starts the fishing mode. Then, the motion sensor collects motion data, and the motion sensor measures acceleration data and angular velocity data. The data measured by the motion sensor can be stored in a storage unit. After the casting point is found, the time window in which the casting point is located is also determined synchronously, the motion data of the plurality of collection time points corresponding to the time window are read by the casting action recognition unit, used to calculate the feature value, the feature value is input into the casting action recognition model, and it is determined whether the user has the casting action.

[0104] Compared with the scheme of judging whether the casting action occurs by comparing the size of the acceleration alone, the accuracy of the casting action recognition is improved. Moreover, the scheme determines the casting point in the casting process in advance, thereby determining the motion data of the time window that needs to enter the casting action recognition unit, avoiding the calculation of the motion data of each time window, reducing the consumption of computing power, and improving the casting action recognition efficiency.

[0105] The above Figure 2 S202 can further include S2021-S2025, executed by the casting point determination unit. As shown, Figure 3 Figure 3 is a flowchart of another casting action recognition method provided by the embodiments of the present application.

[0106] S2021, acquiring motion data corresponding to a plurality of sampling time points in a 4s time window.

[0107] Taking the first time window as 4s, the first step as one sampling point, and the preset time point as the midpoint of the first time window as an example, sliding is performed on the entire motion data to acquire the motion data corresponding to the plurality of sampling time points in the 4s time window.

[0108] S2022, determining the maximum amplitude in the 4s time window.

[0109] In the motion data corresponding to the plurality of sampling time points in the 4s time window, the maximum amplitude is selected. The maximum amplitude can be the maximum point of the acceleration amplitude, or the maximum point of the angular velocity amplitude, or the maximum point of the weighted average value of the acceleration amplitude and the angular velocity amplitude, and the embodiments of the present application are not limited thereto.

[0110] S2023, judging whether the maximum amplitude is at the midpoint of the 4s time window?

[0111] ​If the maximum amplitude is not at the midpoint of the 4s time window, S2024 is performed. If the maximum amplitude is at the midpoint of the 4s time window, S2025 is performed.

[0112] S2024, sliding the 4s time window backward by one sample point to form a new 4s time window.

[0113] Sliding the 4s time window backward by one sample point to obtain a new 4s time window, and continuing to perform S2022, in the new 4s time window, selecting a new maximum amplitude from the motion data corresponding to each of the plurality of sampling time points, until it is determined that the maximum amplitude is at the midpoint of the new 4s time window.

[0114] S2025, taking the maximum amplitude as the throwing point.

[0115] For the case where the maximum amplitude is the maximum value point of the acceleration amplitude, the maximum amplitude is at the midpoint of the 4s time window, indicating that the point is the maximum force point (i.e., the throwing point). For the case where the maximum amplitude is the maximum value point of the angular velocity amplitude, the maximum amplitude is at the midpoint of the 4s time window, indicating that the point is the maximum turning point (i.e., the throwing point). For the case where the maximum amplitude is the maximum value point of the weighted average of the acceleration amplitude and the angular velocity amplitude, the maximum amplitude is at the midpoint of the 4s time window, indicating that the point is the throwing point.

[0116] The present example improves the accuracy of throwing point recognition by selecting the maximum amplitude in the 4s time window and determining whether the maximum amplitude is at the midpoint of the 4s time window.

[0117] In some embodiments, based on the above Figure 1 , after S103 in the above Figure 1 , as shown in Figure 4 , the method further includes S104. Figure 4

[0118] S101, collecting motion data of a user's hand.

[0119] S102, determining candidate throwing data from the motion data, the candidate throwing data including motion data generated within a first time period.

[0120] S103, determining whether the user throws within the first time period according to the generation time of the extreme value data of the candidate throwing data and the preset time within the first time period.

[0121] S104, in the case where it is detected that the user throws within the first time period, determining whether the user winds within a preset second time period after the first time period according to the motion data.​

[0122] In the embodiments of the present application, in the case where it is determined that the user performs the casting in the first time period, candidate take-up data corresponding to a second time period is determined according to the motion data; the candidate take-up data is used to indicate whether the fishing line performs uniform motion (i.e., whether the take-up is stable and uniform), and the second time period is located after the first time period. Whether the user performs the take-up in the second time period is determined according to the candidate take-up data.

[0123] In the embodiments of the present application, regular take-up (which can also be referred to as winding) occurs after the casting action in the fishing process, that is, the fishing line performs uniform motion, that is, the take-up is stable and uniform. The motion data after the casting action is divided into a plurality of second time periods, and the motion data included in adjacent two second time periods can overlap. The candidate take-up data (including variance or standard deviation, and energy proportion) of each second time period is determined according to the motion data included in the second time period. The variance, standard deviation and energy proportion can reflect whether the fishing line performs uniform motion (i.e., whether the take-up is stable and uniform). That is, the motion data after the casting action is arranged in order of sampling time, a second time window with a time length of a second time period is set, and the second time window can slide according to a second step. Alternatively, a second fixed window with a time length of a second time period is set, and the motion data slides according to a second step in order of sampling time. Then, the candidate take-up data (including variance or standard deviation, and energy proportion) is determined according to the motion data corresponding to the plurality of sampling time points falling in the second time window (or the second fixed window).

[0124] Taking the second time period as a sliding second time window as an example, three examples are listed to explain the process of calculating the variance, standard deviation and energy proportion of the motion data included in the second time period.

[0125] In the first example, the motion sensor can include an accelerometer, and the motion data includes acceleration data of three axes of the accelerometer. A sliding time window (i.e., a second time window) is created. For the acceleration data of the three axes corresponding to each sampling time point in the second time window, the acceleration amplitude is calculated to obtain the acceleration amplitude of each sampling time point. Then, the standard deviation (or variance) of the acceleration amplitude in the entire second time window is calculated, and the frequency domain transformation is performed on the acceleration amplitude of each sampling time point to calculate the acceleration amplitude frequency energy proportion. The candidate take-up data includes the standard deviation (or variance) of the acceleration amplitude and the acceleration amplitude frequency energy proportion.

[0126] In the second example, the motion sensor can include a gyroscope, and the corresponding motion data includes angular velocity data of three axes measured by the gyroscope. A sliding time window (i.e., a second time window) is created, and for the angular velocity data of the three axes corresponding to each sampling time within the second time window, the angular velocity amplitude is calculated to obtain the angular velocity amplitude of each sampling time, and then the standard deviation (or variance) of the angular velocity amplitude in the entire second time window is calculated, and the angular velocity amplitude frequency domain energy proportion is calculated by performing frequency domain transformation on the angular velocity amplitude of each sampling time. The candidate line reeling data includes the standard deviation (or variance) of the angular velocity amplitude and the angular velocity amplitude frequency domain energy proportion.

[0127] In the third example, the motion sensor can include an accelerometer and a gyroscope, and the candidate line reeling data includes the standard deviation (or variance) of the acceleration amplitude and the acceleration amplitude frequency domain energy proportion, and the standard deviation (or variance) of the angular velocity amplitude and the angular velocity amplitude frequency domain energy proportion, in combination with the first example and the second example.

[0128] For example, for the acceleration amplitude frequency domain energy proportion and the angular velocity amplitude frequency domain energy proportion, it is assumed that the sampling frequency is 25 HZ, and the converted frequency domain is 0 HZ-12.5 HZ. Among them, it takes about 1 s-2 s to reel a circle, and the corresponding frequency band is 0.5 HZ-1 HZ. Based on this, the ratio of the frequency spectrum area composed of 0.5 HZ-1.5 HZ to the area of the entire frequency spectrum of 0-12.5 HZ is taken as the frequency domain energy proportion.

[0129] The execution subject of the casting action recognition method provided by the embodiments of the present application can be a casting action recognition device, and the casting action recognition device includes a memory and a processor. The above-mentioned calculation of variance, standard deviation, frequency domain conversion and calculation of frequency domain energy proportion can be performed by a regular line reeling determination unit in the casting action recognition device, and the motion data corresponding to each of the plurality of sampling times in the second time window is read by the regular line reeling determination unit in the casting action recognition device, for calculating the variance, the standard deviation and the frequency domain energy proportion, to determine whether the regular line reeling occurs. Among them, the regular line reeling determination unit can be an operation module packaged in the processor, or the processor itself, and the embodiments of the present application do not limit this.

[0130] Among them, the length of the second period (i.e., the second time window or the second fixed window) is determined according to the duration of one circle of line reeling in the fishing process. The first step in the above-mentioned casting action recognition process is less than half of the length of the second time window or the second fixed window. Compared with the casting action, the line reeling action is a long-time continuous action, and therefore, the second step is not particularly limited.

[0131] Generally, the regularity reeling refers to a continuous reeling action in fishing for more than a preset time; the preset time can be a time corresponding to a fixed number of turns (for example, three turns, two turns, four turns, etc.), or a fixed time (for example, 3s, 2s, 4s, etc.), and the application embodiments do not limit this.

[0132] The sliding step (i.e., the second step) can be an integer multiple of the sampling point interval. For example, the sampling frequency of the motion sensor is 10HZ, that is, 10 samples per second, and the sliding step can be set to 0.1s, 0.2s, 0.3s, 0.5s, 1s, 2s, 3s, 5s, etc.

[0133] In the application embodiments, the candidate reeling data can be stable and uniform regularity reeling (i.e., the fishing line moves uniformly), or can not be regularity reeling. It is determined whether the candidate reeling data meets the preset condition, so as to determine whether the user reeled in the second period. If the standard deviation in the candidate reeling data is less than or equal to the preset standard deviation threshold (or the variance is greater than the preset variance threshold), and / or the frequency energy ratio in the candidate reeling data is less than or equal to the preset energy ratio, it indicates that the action is unstable or uneven, and therefore it can be determined that there is no regularity reeling. It is further determined that the user does not have a reeling action in the current second period. If the standard deviation in the candidate reeling data is greater than the preset standard deviation threshold (or the variance is greater than the preset variance threshold), and the frequency energy ratio in the candidate reeling data is greater than the preset energy ratio, it indicates that the action is stable and uniform, and the fishing line moves uniformly, and therefore it can be determined that there is regularity reeling. It is further determined that the user has a reeling action in the current second period, or according to the motion data corresponding to each sampling time in the current second period, the reeling action recognition model is further used to determine whether the user has a reeling action.

[0134] It should be noted that the reeling action recognition step is performed after the above-mentioned identification of the user having the casting action, and therefore the processed motion data is the motion data after the identification of the casting action. On the one hand, the effectiveness of the casting action can be confirmed again, and the accuracy of the casting action recognition is improved, and on the other hand, multiple reeling actions can be further determined, and the timestamp of the last reeling action is recorded, and then the single rod duration is calculated according to the casting action to the last reeling action.

[0135] In the embodiments of the present application, after the user is identified to have the casting action, the motion data after the user has the casting action is continuously detected, and candidate take-up data corresponding to a second time period is determined; the second time period includes motion data corresponding to each of a plurality of collection time points, and the candidate take-up data can be data distribution dispersion degree (for example, standard deviation or variance) and frequency domain energy proportion of the second time period. The candidate take-up data can reflect whether the fishing line moves uniformly (that is, whether the take-up is stable and uniform). Therefore, whether the user takes up the fishing line in the second time period is determined according to the candidate take-up data. Compared with the scheme of manually recording the take-up, the accuracy of the casting action recognition is improved. Further, the take-up determination result can further assist in confirming the effectiveness of the above casting action, and the accuracy of the casting action recognition is improved.

[0136] In some embodiments, the step of determining whether the user takes up the fishing line in the second time period after the first time period according to the motion data in S104 can be implemented in the following manner. Whether the fishing line moves uniformly in the second time period is determined according to the motion data; and in the case that the fishing line moves uniformly, it is determined that the user takes up the fishing line in the second time period.

[0137] According to the motion data included in the second time period, the standard deviation (or variance) is determined; the motion data included in the second time period is subjected to frequency domain conversion to obtain a frequency domain distribution graph; the energy proportion of a preset frequency band in the frequency domain distribution graph is determined; the preset frequency band is determined according to the duration of one round of take-up in the fishing process; and the standard deviation (or variance) and the energy proportion are taken as the candidate take-up data corresponding to the second time period. The candidate take-up data is used to indicate whether the fishing line moves uniformly (whether the take-up is stable and uniform).

[0138] In the embodiments of the present application, the second time period includes motion data corresponding to each of a plurality of collection time points, and the data distribution dispersion degree (for example, standard deviation or variance) is determined according to the plurality of motion data in the second time period. The motion data corresponding to each of the plurality of sampling time points in the second time period is subjected to frequency domain conversion to obtain a frequency domain distribution graph; the frequency domain distribution graph reflects the energy distribution of a certain frequency band, for example, the sampling frequency is 25HZ, the converted frequency domain is 0-12.5HZ, and the frequency domain distribution graph reflects the energy distribution of 0-12.5HZ. The energy proportion of a preset frequency band in the frequency domain distribution graph is calculated. The preset frequency band is determined according to the duration of one round of take-up in the fishing process, for example, the duration of one round of take-up is 1s-2s, and the preset frequency band is set to 0.5HZ-1HZ. The ratio between the data distributed in 0.5HZ-1HZ and the data distributed in 0-12.5HZ is calculated as the energy proportion. The data distribution dispersion degree (for example, standard deviation or variance) and the energy proportion are taken as the candidate take-up data corresponding to the second time period.

[0139] In the actual winding process, a waveform is generated every time a coil is wound. After conversion into a frequency spectrum (corresponding to a frequency domain distribution diagram), a peak is generated. If several coils are wound, and each coil is wound for 1 s, the peak is generated at a position of 1 Hz. If each coil is wound for 2 s, the peak is generated at a position of 0.5 Hz. If the winding speed is gradually changed from 2 s to 1 s, the peak is generated at a position from 0.5 Hz to 1 Hz, and there is basically no value or irregular noise at other positions.

[0140] Taking the second time period as a sliding second time window, three examples are listed below to illustrate the candidate winding data.

[0141] In a first example, the motion sensor can include an accelerometer, and the motion data can include acceleration corresponding to the user's hand. The standard deviation of the acceleration amplitude (or the variance of the acceleration amplitude) in the second time window can be calculated, and the acceleration frequency domain distribution diagram can be obtained by performing frequency domain transformation on the acceleration amplitude. The energy proportion of the preset frequency band in the frequency domain distribution diagram can be calculated to obtain the acceleration energy proportion. The standard deviation of the acceleration amplitude (or the variance of the acceleration amplitude) and the acceleration energy proportion can be taken as the candidate winding data corresponding to the second time window.

[0142] In a second example, the motion sensor can include a gyroscope, and the motion data can include angular velocity corresponding to the user's hand. The standard deviation of the angular velocity amplitude (or the variance of the angular velocity amplitude) in the second time window can be calculated, and the angular velocity frequency domain distribution diagram can be obtained by performing frequency domain transformation on the angular velocity amplitude. The energy proportion of the preset frequency band in the frequency domain distribution diagram can be calculated to obtain the angular velocity energy proportion. The standard deviation of the angular velocity amplitude (or the variance of the angular velocity amplitude) and the angular velocity energy proportion can be taken as the candidate winding data corresponding to the second time window.

[0143] In a third example, the motion sensor can include an accelerometer and a gyroscope, and the motion data can include acceleration and angular velocity corresponding to the user's hand. The standard deviation of the acceleration amplitude (or the variance of the acceleration amplitude), the acceleration energy proportion, the standard deviation of the angular velocity amplitude (or the variance of the angular velocity amplitude), and the angular velocity energy proportion can be taken as the candidate winding data corresponding to the second time window.

[0144] In some embodiments, the candidate winding data includes the standard deviation and the energy proportion. The above S104 can also be implemented by the following two examples. In example one, when it is detected that the standard deviation is greater than a preset standard deviation threshold value and the energy proportion is greater than a preset proportion threshold value, it is determined that the user is winding in the second time period.

[0145] Based on the above description of the candidate winding data, taking the second time period as a sliding second time window, three examples are listed below to illustrate the candidate winding data.

[0146] In the first example, the candidate reeling data includes the standard deviation of the acceleration amplitude (or the variance of the acceleration amplitude) and the acceleration energy proportion. The greater the standard deviation or variance, the more the user has the action. In the case that the standard deviation of the acceleration amplitude is greater than the acceleration standard deviation threshold (or the variance of the acceleration amplitude is greater than the acceleration variance threshold) and the acceleration energy proportion is greater than the acceleration proportion threshold, it is determined that the regular reeling (i.e., the fishing line moves uniformly, that is, the reeling is stable and uniform) occurs, and it is determined that the user reels in the second time period. If the standard deviation of the acceleration amplitude is less than or equal to the acceleration standard deviation threshold and / or the acceleration energy proportion is less than or equal to the acceleration proportion threshold, it is determined that the regular reeling does not occur, and it is determined that the user does not reel in the second time period. The second time window is slid backward (for example, the sampling time point of the motion sensor is slid backward by one), and the determination of the candidate reeling data, the judgment of whether the candidate reeling data meets the stable and uniform steps are repeated according to the slid second time window until it is determined that the user reels in the second time window.

[0147] It should be noted that the acceleration standard deviation threshold and the acceleration proportion threshold can be set by those skilled in the art according to the actual situation, as long as they can reflect the uniform and stable reeling. The embodiments of the present application do not limit the acceleration standard deviation threshold and the acceleration proportion threshold. For example, the acceleration standard deviation threshold can be set to 3, 3.5, 2.5, etc., and the acceleration proportion threshold can be set to 50%, 60%, 45%, etc. For example, the acceleration standard deviation threshold can be set to 3, and the acceleration proportion threshold can be set to 50%. In this way, if the standard deviation of the acceleration amplitude is greater than 3 and the acceleration frequency energy proportion is greater than 50%, it is determined that the regular reeling occurs. If the above conditions are not met, the second time window is slid backward by one sampling time point of the motion sensor, and the above determination is repeated.

[0148] By determining whether the standard deviation of the acceleration amplitude is greater than the acceleration standard deviation threshold (or the variance of the acceleration amplitude is greater than the acceleration variance threshold) and whether the acceleration frequency energy proportion is greater than the acceleration proportion threshold, it is determined whether the regular reeling occurs, which improves the accuracy of the reeling identification.

[0149] In the second example, the candidate reeling data includes the standard deviation of the angular velocity amplitude (or the variance of the angular velocity amplitude) and the angular velocity energy proportion. When it is detected that the standard deviation of the angular velocity amplitude is greater than an angular velocity standard deviation threshold (or the variance of the angular velocity amplitude is greater than an angular velocity variance threshold) and the angular velocity energy proportion is greater than an angular velocity proportion threshold, it is indicated that regular reeling occurs (i.e., the fishing line moves uniformly, that is, the reeling is stable and uniform), and it is determined that the user reels in the second time period. If the standard deviation of the angular velocity amplitude is less than or equal to the angular velocity standard deviation threshold and / or the angular velocity energy proportion is less than or equal to the angular velocity proportion threshold, it is indicated that regular reeling does not occur, and it is determined that the user does not reel in the second time period. The second time window is slid backward, and the above steps of determining the candidate reeling data and judging whether the candidate reeling data meets the stable and uniform condition are continued to be repeated based on the slid second time window until it is determined that the user reels in the second time window.

[0150] It should be noted that the angular velocity standard deviation threshold and the angular velocity proportion threshold can be set by those skilled in the art according to actual conditions, as long as they can reflect that the reeling is uniform and stable, and the embodiments of the present application do not limit this. For example, the angular velocity standard deviation threshold can be set to 3, 3.5, 2.5, etc., and the angular velocity proportion threshold can be set to 50%, 60%, 45%, etc. For example, the angular velocity standard deviation threshold can be set to 3, and the angular velocity proportion threshold can be set to 50%. In this way, if the standard deviation of the angular velocity amplitude is greater than 3 and the angular velocity frequency energy proportion exceeds 50%, it is indicated that regular reeling occurs. If the above conditions are not met, the second time window is slid backward by one sampling time point of the motion sensor, and the above judgment is continued to be repeated.

[0151] By judging the size between the standard deviation of the angular velocity amplitude and the angular velocity standard deviation threshold (or judging the size between the variance of the angular velocity amplitude and the angular velocity variance threshold), and simultaneously determining whether the angular velocity frequency energy proportion is greater than the angular velocity proportion threshold, it is determined whether regular reeling occurs, and the accuracy of reeling identification is improved.

[0152] In the third example, the candidate reeling data includes the standard deviation of the acceleration amplitude (or the variance of the acceleration amplitude) and the acceleration energy proportion, and the standard deviation of the angular velocity amplitude (or the variance of the angular velocity amplitude) and the angular velocity energy proportion. The third example is a combination of the first example and the second example. When it is detected that the standard deviation of the acceleration amplitude is greater than an acceleration standard deviation threshold and the acceleration energy proportion is greater than an acceleration proportion threshold, or it is detected that the standard deviation of the angular velocity amplitude is greater than an angular velocity standard deviation threshold and the angular velocity energy proportion is greater than an angular velocity proportion threshold, it is indicated that regular reeling occurs, and it is determined that the user reels in the second time period.

[0153] In a case where the standard deviation of the acceleration amplitude is detected to be less than or equal to an acceleration standard deviation threshold value, or the acceleration energy proportion is less than or equal to an acceleration proportion threshold value, or the standard deviation of the angular velocity amplitude is detected to be greater than an angular velocity standard deviation threshold value, or the angular velocity energy proportion is greater than an angular velocity proportion threshold value, it is indicated that the regular reeling does not occur, and it is determined that the user does not reel in the second time period. The second time window is slid backward (for example, the sampling time point of a motion sensor is slid backward), and the above steps of determining the candidate reeling data and judging whether the candidate reeling data meets the stable and uniform are continued to be repeated according to the slid second time window until the regular reeling occurs, so as to determine that the user reels in the second time window.

[0154] By judging whether the standard deviation of the acceleration amplitude, the standard deviation of the angular velocity amplitude, and the acceleration frequency energy proportion and the angular velocity frequency energy proportion meet the conditions, it is judged whether the regular reeling occurs, and the accuracy of the reeling identification is improved. The first example and the second example are combined, and the diversity of the implementation manner of the reeling identification is increased.

[0155] In a case where the standard deviation is detected to be greater than a preset standard deviation threshold value and the energy proportion is greater than a preset proportion threshold value, the reeling action recognition model is used to determine whether the user reels in the second time period according to the motion data included in the second time period.

[0156] In the embodiment of the application, it is judged whether the standard deviation is greater than a preset standard deviation threshold value and whether the energy proportion is greater than a preset proportion threshold value. If the standard deviation is greater than the preset standard deviation threshold value and the energy proportion is greater than the preset proportion threshold value, it is indicated that the candidate reeling data corresponding to the second time period can represent the stable and uniform regular reeling (that is, the fishing line moves uniformly). The motion data corresponding to each of the plurality of sampling time points in the second time period is input into the reeling action recognition model, and the reeling action recognition model is used to determine whether the user has the reeling action in the second time period. Since the regular reeling in the actual fishing process does not necessarily mean the reeling action, in the example, after the regular reeling is determined, the motion data corresponding to each of the plurality of sampling time points in the second time period is identified by using the reeling action recognition model, so as to improve the accuracy of the reeling action recognition.

[0157] It should be noted that the second time period is a sliding second time window, and when the motion data is processed, one or more second time periods meeting the conditions (i.e., the standard deviation is greater than the preset standard deviation threshold, and the energy ratio is greater than the preset ratio threshold) can be screened out. In this example, by screening out one or more second time periods, some data that is obviously not a winding action (for example, noise data that does not appear regularly) can be filtered out. The winding action recognition model is used to identify the motion data included in the one or more screened second time periods to obtain a winding action recognition result. In one way, the winding action recognition model identifies the motion data included in each of the screened second time periods to obtain an identification result for each second time period, and comprehensively considers multiple identification results to take the result corresponding to the maximum probability value in the identification result as the final identification result (whether it is a winding action or not). In another way, the winding action recognition model identifies the motion data included in the multiple screened second time periods to obtain an identification result (whether it is a winding action or not).

[0158] The winding action recognition model can be a machine learning model, such as a neural network model (Neural Networks, NN), including but not limited to a convolutional neural network (Convolutional Neural Networks, CNN), a recurrent neural network (Recurrent Neural Network, RNN), etc. The winding action recognition model is a pre-trained binary classification model using a large number of motion data samples, which has the function of identifying whether a certain time period of motion data corresponding to each sampling time is a winding action, wherein the motion data samples include positive example samples labeled as a throwing action, and negative example samples labeled as not a throwing action. The structure of the winding action recognition model is not limited in the embodiments of the present application, as long as the model can identify whether the motion data represents a winding action.

[0159] It should be noted that the execution subject of the throwing action recognition method provided by the embodiments of the present application can be a throwing action recognition device, and the throwing action recognition device includes a memory and a processor. The above-mentioned judging whether the second time period in which the winding appears regularly is a winding action according to the motion data corresponding to each sampling time in the second time period can be performed by a winding action recognition unit in the throwing action recognition device. The winding action recognition unit can be an operation module packaged in the processor, or the processor itself, and the embodiments of the present application do not limit this.

[0160] In the embodiments of the present application, the time window of the motion data that needs to enter the line reeling action recognition unit is determined by pre-determining the second period in which regular line reeling occurs, thereby avoiding the calculation of the motion data of each time window, avoiding the same motion data entering the line reeling action recognition model multiple times in the sliding window process, reducing the calculation power consumption, reducing the calculation power requirement of the device, and reducing the battery consumption. Since regular line reeling occurs during fishing, only the standard deviation and energy ratio of the acceleration amplitude of the motion sensor and the standard deviation and energy ratio of the angular velocity amplitude need to be calculated, and the one-dimensional and two-dimensional transformation of the motion data is required for the line reeling action recognition unit. Therefore, in the entire casting action recognition process, the calculation power and energy consumption required by the line reeling action recognition model are the largest. In the present example, the use frequency of the line reeling action recognition model in the casting action recognition process is reduced as much as possible, the calculation power requirement of the device is reduced, and the power consumption is reduced.

[0161] In some embodiments, the step of determining whether the user reeled in the line in the second period according to the motion data included in the second period by the line reeling action recognition model in Example Two above can be implemented in the following manner. Obtain the second feature value corresponding to the motion data included in the second period; the second feature value includes at least one of the statistical feature, the time domain feature and the frequency domain feature; determine whether the user reeled in the line in the second period according to the second feature value by the line reeling action recognition model.

[0162] In the embodiments of the present application, the motion data corresponding to each of the plurality of sampling time points in the second period is subjected to frequency domain transformation to obtain a plurality of frequency domain transformation values; a one-dimensional statistical feature is determined according to the plurality of motion data and the plurality of frequency domain transformation values; a two-dimensional statistical feature, a time domain feature and a frequency domain feature are determined according to the one-dimensional statistical feature; and the one-dimensional statistical feature, the two-dimensional statistical feature, the time domain feature and the frequency domain feature are taken as the second feature value.

[0163] Taking the second time window that is sliding as an example, the motion data corresponding to each of the plurality of sampling time points in the second time window that meets the condition is counted, wherein the second time window that meets the condition refers to the second time window in which regular line reeling occurs. The motion data in the second time window that meets the condition is processed, and the feature value of the motion data corresponding to each sampling time point is calculated, including one-dimensional feature value, two-dimensional feature value, etc.

[0164] Taking the motion data including acceleration and angular velocity corresponding to the user's hand as an example, the motion data corresponding to each sampling time in the second time window includes: X-axis acceleration, Y-axis acceleration, Z-axis acceleration, X-axis angular velocity, Y-axis angular velocity, and Z-axis angular velocity. The one-dimensional feature value includes at least one of the following one-dimensional data: three-axis acceleration amplitude and its first-order difference, three-axis angular velocity amplitude and its first-order difference, weighted average of three-axis acceleration amplitude and three-axis angular velocity amplitude, frequency domain transform value (for example, frequency domain transform is realized by Laplace transform, Fourier transform or z transform) of the weighted average, root mean square energy of the weighted average, attitude value (including roll angle, pitch angle and yaw angle), frequency spectrum of the roll angle, pitch angle and yaw angle, and other one-dimensional statistical features. The two-dimensional feature value includes at least one of the following two-dimensional statistical features: mean, median, percentile, variance, standard deviation, interquartile range (IQR), maximum value and mean value ratio, coefficient of variation (i.e., average value / standard value), skewness, kurtosis and approximate entropy of the above one-dimensional data; the two-dimensional feature value further includes at least one of the following time domain features of the above one-dimensional data: peak value, valley value, peak-valley difference, peak value position, valley value position, peak-valley interval zero-crossing rate, full width at half maximum, peak-valley interval, peak value / width, and left-right area ratio of the peak value; the two-dimensional feature value further includes at least one of the following: ratio of energy of the first preset frequency band to energy of the full frequency domain, ratio of energy of the first preset frequency band to energy of the second preset frequency band, ratio of peak value of the first preset frequency band to peak value of the second preset frequency band, and ratio of positions of the peak values of the first preset frequency band and the second preset frequency band.

[0165] For example, taking the sampling frequency as 25HZ, the converted frequency domain (i.e., the full frequency domain) as 0HZ-12.5HZ, and the first preset frequency band as 0.5HZ-1.5HZ and the peak value of the second preset frequency band as 1.5HZ or more, the frequency domain features include: position of the peak value of 0.5HZ-1.5HZ, position of the peak value of 1.5HZ or more, ratio of the peak values of 0.5HZ-1.5HZ and 1.5HZ or more and ratio of the positions of the peak values; energy proportion of 0.5HZ-1.5HZ and 1.5HZ or more; total energy proportion of 0.5HZ-1.5HZ and 0HZ-12.5HZ.

[0166] In the embodiments of the present application, the second feature value is taken as input data, and a take-up action recognition model formed through machine learning is used to automatically identify whether a take-up action occurs. After regular take-up is determined, the motion data corresponding to each sampling time in the second period in which the regular take-up occurs is further statistically processed to obtain the second feature value, and then the take-up action recognition model is used to recognize the take-up action according to the second feature value. Compared with the scheme of directly determining whether the user has a take-up action according to the regular take-up, the accuracy of take-up action recognition is improved.

[0167] In some embodiments, as described above Figure 4 After S104, the casting action recognition method further includes S105 or S106.

[0168] S105, in a case where it is detected that the user winds in the line in the second time period, it is determined that the casting of the user in the first time period is valid.

[0169] S106, in a case where it is not detected that the user winds in the line in the second time period, it is determined that the casting of the user in the first time period is invalid.

[0170] In the embodiments of the present application, in a case where it is determined that the user casts and it is not determined that the user winds in the line, the casting action of the user is determined as an invalid casting action; in a case where it is determined that the user casts and it is determined that the user winds in the line, the casting action of the user is determined as a valid casting action.

[0171] In the embodiments of the present application, in a case where the casting action recognition result is a casting action and the winding action recognition result is a non-winding action, it is indicated that the user does not cast validly or the user does not cast actually, and therefore the casting action is determined as an invalid casting action. In a case where the casting action recognition result is a casting action and the winding action recognition result is a winding action, it is indicated that the user casts validly, and the casting action is determined as a valid casting action. The casting action determined as described above is verified in combination with the determination process of the winding action, and the accuracy of the casting action recognition is improved.

[0172] In some embodiments, the next candidate winding data corresponding to the next second time period is continuously determined, whether the user winds in the line in the next second time period is determined according to the next candidate winding data, until the last second time period recognized as the winding action is determined; the sampling time corresponding to the first time period recognized as the casting action (which can be the start time, the center time or the end time corresponding to the first time period recognized as the casting action, etc.) and the sampling time corresponding to the last second time period recognized as the winding action (which can be the start time, the center time or the end time corresponding to the last second time period recognized as the winding action, etc.) are used to determine the duration of the single fishing process.

[0173] In the embodiments of the present application, taking the second time period as a sliding second time window, the second time window is continuously slid according to the second step size until the time window in which the regular winding occurs for the last time and the winding action recognition result is the winding action is determined; the sampling time corresponding to the maximum amplitude when the casting action recognition result is the casting action and the sampling time corresponding to the time window in which the regular winding occurs for the last time and the winding action recognition result is the winding action are used to determine the duration of the single fishing process.

[0174] In the embodiments of the present application, after determining that the user has the casting action, the last time the line is collected is also determined, and the time difference between the two is the time length used in the single fishing process. The fishing process of the user is recorded, and the richness and diversity of the casting scene are increased.

[0175] In some embodiments, the casting action recognition method further includes the steps of: acquiring the coordinate position of the motion sensor; and binding the coordinate position and the effective casting action to determine the fishing trajectory.

[0176] In the embodiments of the present application, after determining the effective casting action, the coordinate position of the motion sensor and the effective casting action are bound to construct the fishing trajectory, record the fishing process of the user, and increase the richness and diversity of the casting scene.

[0177] In the following, an exemplary application of the embodiments of the present application in an actual application scenario will be described.

[0178] Based on the above Figure 2 , after S204 in the above Figure 2 , as shown in Figure 5 , the method further includes S205 and S206. Figure 5

[0179] S201, the motion sensor collects motion data.

[0180] S202, the casting point is determined according to the motion data.

[0181] S203, the first characteristic value is calculated according to the motion data corresponding to each of the plurality of collection time points in the time window in which the casting point is located.

[0182] S204, whether the user has the casting action is recognized according to the first characteristic value.

[0183] S205, in the case where it is recognized that the user has the casting action, whether the user has the line collecting action is recognized.

[0184] ​In the case of identifying that the user has the throwing action, a second time window with a second time length is set, and the second time window can slide according to a second step length. According to the motion data corresponding to each of the multiple sampling time points falling in the second time window, the standard deviation and the energy proportion are determined. It is judged whether the standard deviation and the energy proportion satisfy the regularity winding condition. If the standard deviation and the energy proportion do not satisfy the regularity winding condition, the second time window is slid backward, and it is judged whether the standard deviation and the energy proportion of the slid second time window satisfy the regularity winding condition, until the standard deviation and the energy proportion of the slid second time window satisfy the regularity winding condition, and the time window in which the regularity winding occurs is obtained. If the standard deviation and the energy proportion satisfy the regularity winding condition, the second characteristic value is calculated according to the motion data corresponding to each of the multiple collection time points in the time window in which the regularity winding occurs. The second characteristic value is input into the winding action recognition model, and it is judged whether the user has the winding action through the winding action recognition model.

[0185] S206, in the case of identifying that the user has the winding action, it is determined that the user has the effective throwing action.

[0186] The winding determination result is used to further assist in verifying the effectiveness of the above-mentioned throwing action, and the accuracy of the throwing action recognition is improved.

[0187] The above Figure 5 S205 can further include S2051-S2055, as shown in Figure 6 Figure 6 is a flowchart of another throwing action recognition method provided by the embodiments of the present application.

[0188] S2051, in the case of identifying that the user has the throwing action, the motion data corresponding to each of the multiple sampling time points in a 6s time window is obtained.

[0189] Taking the second time window as 6s and the second step length as one sampling point as an example, the motion data after the user has the throwing action is slid to obtain the motion data corresponding to each of the multiple sampling time points falling in the 6s time window.

[0190] S2052, whether regularity winding occurs in the 6s time window?

[0191] If the regularity winding does not occur in the 6s time window, S2053 is executed; if the regularity winding occurs in the 6s time window, S2054 is executed.

[0192] ​The regularity reeling is an example of three continuous circles or three seconds of time. The regularity reeling is reflected on the motion data as a standard deviation greater than a preset standard deviation and an energy ratio greater than a preset energy ratio. Taking the preset standard deviation as 3 and the preset energy ratio as 50% as an example, the standard deviation and the energy ratio of the motion data corresponding to each of the multiple collection time points in the 6s time window are calculated, and it is determined whether the standard deviation is greater than 3 and the energy ratio is greater than 50% to determine whether the regularity reeling occurs in the 6s time window.

[0193] For example, the motion data includes acceleration amplitude and angular velocity amplitude. In the 6s time window, any one of the acceleration amplitude and the angular velocity amplitude meets the regularity reeling condition, that is, enters the reeling identification model to judge the reeling action. The acceleration amplitude and the angular velocity amplitude are respectively subjected to frequency domain transformation to obtain acceleration frequency energy ratio and angular velocity frequency energy ratio. In the case where the standard deviation of the acceleration amplitude or the angular velocity amplitude is greater than 3 and the acceleration frequency energy ratio or the angular velocity frequency energy ratio is greater than 50%, it is determined that the regularity reeling occurs.

[0194] S2053, sliding the 6s time window by one sample point to form a new 6s time window.

[0195] The 6s time window is slid by one sample point to obtain a new 6s time window, and S2052 is continuously executed. The standard deviation and the energy ratio are calculated according to the motion data corresponding to each of the multiple sampling time points in the new 6s time window, until it is determined that the standard deviation is greater than the preset standard deviation and the energy ratio is greater than the preset energy ratio, that is, until it is determined that the regularity reeling occurs in the new 6s time window.

[0196] S2054, calculating a second feature value according to the motion data corresponding to each of the multiple collection time points in the time window where the regularity reeling occurs.

[0197] S2055, identifying whether the user has a reeling action according to the second feature value.

[0198] According to the motion data corresponding to each of the multiple sampling time points in the time window where the regularity reeling occurs, a second feature value is calculated. The second feature value is input into a reeling action identification model to determine whether the user has a reeling action through the reeling action identification model.

[0199] As described above Figure 6 The execution subject of the throwing and pitching action identification method can be a throwing and pitching action identification device. S2051-S2053 described above can be executed by a regularity reeling determination unit, and S2054 and S2055 described above can be executed by a reeling action identification unit. Figure 5 and Figure 6In the flow of the application embodiment, after the cast motion is identified, the motion sensor continues to collect motion data, which includes acceleration data and angular velocity data, and the motion data is continuously stored in the storage unit.

[0200] The regular line retraction identification unit is configured to read the motion data corresponding to each of the plurality of sampling time points in the 6s time window, and determine whether the regular line retraction occurs in the 6s time window. After the regular line retraction determination unit determines that the regular line retraction occurs, the line retraction motion identification unit determines whether the line retraction motion occurs in the fishing process. The motion data corresponding to each of the plurality of sampling time points in the qualified second time window is received. The qualified second time window refers to the time window in which the regular line retraction occurs. The motion data corresponding to each of the plurality of sampling time points in the second time window is used to calculate the second feature value. The second feature value is input into the line retraction motion identification model to determine whether the user has the line retraction motion.

[0201] In the lure fishing, the line retraction motion generally occurs after the cast. Therefore, after the cast motion is preliminarily identified, the line retraction motion identification unit is used to determine whether the user has the line retraction motion. If the user has the line retraction motion, the cast motion preliminarily identified is finally determined as the valid cast behavior, thereby further improving the accuracy of the cast motion identification.

[0202] In the following, an exemplary application of the application embodiment in an actual application scenario will be described.

[0203] For example, the cast motion identification method is described by taking the motion data including acceleration and / or angular velocity as an example, as shown in Figure 7 Figure 7 is a flowchart of another cast motion identification method provided by the application embodiment. The user starts the device to enter the fishing mode.

[0204] S301, acceleration and / or angular velocity is obtained.

[0205] S302, it is determined whether the acceleration in the first period is greater than or equal to a preset acceleration threshold value, and / or whether the angular velocity is greater than or equal to a preset angular velocity threshold value.

[0206] If yes, S303 is executed, and if no, S301 is continuously executed to determine whether the acceleration and / or angular velocity in the next first period meets the condition, that is, to continuously determine whether the acceleration is greater than or equal to the preset acceleration threshold value, and / or whether the angular velocity is greater than or equal to the preset angular velocity threshold value.

[0207] S303, the acceleration and / or angular velocity in the first period is taken as the candidate cast data.

[0208] ​S304. Determine whether the extreme values ​​of acceleration and / or angular velocity within the first time period are at the midpoint of the first time period.

[0209] If yes, then execute S305; otherwise, continue executing S301 to determine whether the acceleration and / or angular velocity in the next first time period meet the conditions.

[0210] S305. Determine if the user will throw within the first time period.

[0211] S306. Determine whether the acceleration and / or angular velocity changes stably during the second time period after the first time period.

[0212] If yes, then execute S307 and S308. If no, then execute S309 and S310.

[0213] S307. Determine that the user will receive the call during the second time period.

[0214] S308. Determine that the user's throws during the first time period are valid.

[0215] S309. It is determined that the user did not receive the call during the second time period.

[0216] S310, Determine that the user's throws during the first time period are invalid.

[0217] Figure 7 For details on the implementation process and achievable technical effects of the mid-throw motion recognition method, please refer to the above. Figures 1-6 The description will not be repeated here.

[0218] The throwing action recognition method provided in the above embodiments Figure 8 This is a schematic diagram of the structure of a throwing action recognition device provided in an embodiment of this application, as shown below. Figure 8 As shown, the throwing action recognition device 80 includes: a processor 801, a memory 802, and a computer program 803 stored in the memory 802 and executable on the processor 801. When the processor 801 executes the computer program 803, it implements the steps in the throwing action recognition method in the above embodiments.

[0219] The throwing motion recognition device 80 can be a general-purpose throwing motion recognition device or a dedicated throwing motion recognition device. In specific implementations, the throwing motion recognition device 80 can be a desktop computer, laptop computer, network server, PDA, mobile phone, tablet computer, wireless terminal device, communication device, or embedded device. This application embodiment does not limit the type of throwing motion recognition device 80. Those skilled in the art will understand that... Figure 8The example of the throwing action recognition apparatus 80 does not constitute a limitation on the throwing action recognition apparatus 80, and can include more or fewer components than shown, or combine some components, or different components, such as an input / output device, a network access device, and the like.

[0220] The processor 801 can be a central processing unit (CPU), and can also be other general-purpose processors, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or the like. The general-purpose processor can be a microprocessor or any conventional processor.

[0221] The memory 802 can be an internal storage unit of the throwing action recognition apparatus 80, such as a hard disk or a memory of the throwing action recognition apparatus 80 in some embodiments. The memory 802 can also be an external storage device of the throwing action recognition apparatus 80, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, or the like, in other embodiments. Further, the memory 802 can include both the internal storage unit and the external storage device of the throwing action recognition apparatus 80. The memory 802 is used to store an operating system, application programs, a boot loader, data, and other programs, and the like. The memory 802 can also be used to temporarily store data that has been output or will be output.

[0222] The embodiments of the present application also provide a throwing action recognition apparatus, which comprises at least one processor, a memory, and a computer program stored in the memory and executable on the at least one processor, and the processor implements the steps in any of the method embodiments described above when executing the computer program.

[0223] The throwing action recognition apparatus and the throwing action recognition method provided by the above embodiments belong to the same concept, and the specific working process of the units and modules in the above embodiments and the technical effects brought by the units and modules can be referred to the method embodiments part, which will not be described here again.

[0224] The function units and modules in the above embodiments can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software function unit. In addition, the specific names of the function units and modules are only used for mutual distinction, and do not limit the protection scope of the embodiments of the present application.

[0225] Based on the throwing action recognition device provided in the above embodiments, the embodiments of the present application provide a wearable device, as shown in Figure 9 Figure 9 The wearable device provided in the embodiments of the present application is shown in a structural schematic diagram. The wearable device 90 includes a motion data acquisition device 901, and the throwing action recognition device 80 shown in the above Figure 8 The wearable device 90 is worn on the hand of the user.

[0226] The method for automatically identifying the throwing action provided in the embodiments of the present application can be applied to a device for automatically identifying the fishing throwing action (i.e., the throwing action recognition device). The throwing action recognition device 80 and the motion data acquisition device 901 (i.e., the motion sensor) can be integrated in the wearable device 90, and the wearable device 90 can be worn on the wrist of the user.

[0227] In one way, the motion data acquisition device 901 can be integrated on the wearable device 90, and the wearable device 90 can directly process the motion data and execute the throwing action recognition method provided in the embodiments of the present application.

[0228] In another way, the motion data acquisition device 901 is not integrated on the wearable device 90, and therefore, the motion data acquisition device 901 sends the collected motion data to the wearable device 90, and the wearable device 90 processes the motion data.

[0229] The embodiments of the present application also provide a throwing action recognition method. The method is applied to a throwing action recognition system including a first smart device and a second smart device in communication connection with the first smart device. The method includes: controlling the first smart device to collect motion data of a hand of a user; determining candidate throwing data from the motion data; controlling the second smart device to determine whether the user performs throwing in a first time period according to a generation time of extreme value data of the candidate throwing data and a preset time in the first time period; and wherein the candidate throwing data includes motion data generated in the first time period.

[0230] ​In this example, the casting action recognition system controls the first smart device to collect motion data of the user's hand. If the first smart device has data processing function, the first smart device can determine the candidate casting data from the motion data; the casting action recognition system further controls the second smart device to determine whether the user casts in the first time period according to the generation time of the extreme value data of the candidate casting data and the preset time in the first time period. If the first smart device does not have data processing function, the second smart device can determine the candidate casting data from the motion data; the casting action recognition system further controls the second smart device to determine whether the user casts in the first time period according to the generation time of the extreme value data of the candidate casting data and the preset time in the first time period.

[0231] It should be noted that the specific implementation process and the achieved technical effects of the casting action recognition method performed by the casting action recognition system can be referred to the description of the casting action recognition method applied to the casting action recognition system, which will not be described here. Figures 1-6

[0232] The application embodiment further provides a casting action recognition system, as shown in Figure 10 Figure 10 is a structural schematic diagram of a casting action recognition system provided by the application embodiment. The casting action recognition system 100 includes a first smart device 1001, a second smart device 1002 in communication connection with the first smart device 1001, a memory and a processor; the memory is used to store a computer program; the processor is used to execute the computer program to realize the method applied to the casting action recognition system as described above.

[0233] In some embodiments, the first smart device 1001 is a motion data collection device arranged on a fishing rod; the second smart device 1002 is a wearable device or a server.

[0234] Example one, the casting action recognition system 100 includes a wearable device and a motion data collection device arranged on a fishing rod. The motion data can be acquired by the motion data collection device arranged on the fishing rod and sent to the wearable device, and the motion data is processed by the wearable device to determine whether the user casts.

[0235] In this example, the candidate casting data can be determined by the wearable device from the motion data.

[0236] Example two, the casting action recognition system 100 includes a server and a motion data collection device arranged on a fishing rod. The motion data can be acquired by the motion data collection device arranged on the fishing rod and sent to the server, and the motion data is processed by the server to determine whether the user casts.

[0237] In this example, the candidate casting data can be determined by the server from the motion data.​​

[0238] In some embodiments, the first smart device 1001 is a wearable device integrated with a motion data acquisition device; and the second smart device 1002 is a server.

[0239] For example, the casting action recognition system 100 includes a wearable device and a server. The motion data can be acquired by the wearable device and sent to the server. The motion data is processed by the server, and it is determined whether the user casts.

[0240] In this example, candidate casting data can be determined from the motion data by the wearable device, and then sent to the server to determine whether the user casts. Alternatively, candidate casting data can be determined from the motion data by the server, and it is determined whether the user casts. The embodiments of the present application do not limit this.

[0241] Based on the casting action recognition device provided in the above embodiments, the embodiments of the present application provide a fishing device. The fishing device includes a motion data acquisition device and the above Figure 8 The casting action recognition device 80 shown in the above embodiments; the motion data acquisition device is fixed on a fishing rod or a fishing reel, or worn on the user's hand; the motion data acquisition device and the casting action recognition device 80 are in communication.

[0242] In one way, the fishing device can be integrated with the casting action recognition device 80 and the motion data acquisition device. The fishing device is fixed on a fishing rod or a fishing reel.

[0243] In another way, the motion data acquisition device is not integrated on the fishing device. The fishing device can include a casting action recognition device 80 and a motion data acquisition device in communication with each other. The motion data acquisition device is fixed on a fishing rod or a fishing reel or worn on the user's hand. The motion data acquisition device acquires motion data and sends it to the casting action recognition device 80. The casting action recognition device 80 performs the casting action recognition method.

[0244] The casting action recognition device 80 in the above fishing device and wearable device 90 can include a casting point determination unit, a casting action recognition unit, a regularity of line retraction determination unit, a line retraction action recognition unit, and a storage unit.

[0245] The cast point determination unit can be used to determine the point at which the rod is drawn back to the forward power during the casting process. The storage unit can be composed of a semiconductor memory, a magnetic storage medium, or other storage medium. The storage unit can also use a removable external recording medium. The storage unit is used to store the data collected by the motion sensor, the data required by the cast point determination unit during the casting process, the calculation of the first and second characteristic values, the cast action recognition model, the cast action recognition result, the line collection action recognition result, and other data.

[0246] The cast action recognition device 80 can also include a positioning unit. The positioning unit is used to obtain the coordinate position of the motion sensor and send the coordinate position to the processor. The processor is used to bind the coordinate position with the effective cast action and determine the fishing trajectory.

[0247] The positioning unit is used to measure the position information (including longitude, latitude, and altitude) of the motion sensor and store it in the storage unit for reading at any time. The positioning unit can be a Global Positioning System (GPS) sensor. After identifying the effective cast behavior, the positioning unit automatically associates the current positioning with the effective cast behavior and stores it in the storage unit.

[0248] The cast action recognition device 80 can also include a display unit. The display unit is used to receive the effective cast action sent by the processor and display the number of times of the effective cast action.

[0249] The display unit is used to display the recognized cast action, which is displayed by counting. The display unit can be a screen. During the casting process or after the casting is completed, the display unit displays the time and location of the cast action, the number of casts, and the single rod duration.

[0250] The cast action recognition device 80 can also include a power supply unit. The power supply unit is electrically connected to the motion sensor, the cast point determination unit during the casting process, the cast action recognition unit, the positioning unit, the storage unit, and the display unit, and is used to supply power to the above-mentioned units. The power supply unit can be a rechargeable power source, such as a lithium battery.

[0251] The embodiments of the present application also provide a computer readable storage medium storing a computer program, which is executed by a processor to implement the steps in the above-mentioned various method embodiments.

[0252] The embodiments of the present application provide a computer program product, which, when running on a computer, causes the computer to execute the steps in the above-mentioned various method embodiments.

[0253] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, all or part of the processes in the above method embodiments can be completed by a computer program instructing related hardware, and the computer program can be stored in a computer readable storage medium. The computer program can implement the steps of each method embodiment when executed by a processor. The computer program includes computer program code, which can be in the form of source code, object code, executable files or some intermediate forms. The computer readable medium at least includes any entity or device capable of carrying the computer program code to the photographing device / terminal equipment, recording medium, computer memory, ROM (Read-Only Memory), RAM (Random Access Memory), CD-ROM (Compact Disc Read-Only Memory), magnetic tape, floppy disk and optical data storage equipment, etc. The computer readable storage medium mentioned in the present application can be a non-volatile storage medium, in other words, a non-transitory storage medium.

[0254] It should be understood that all or part of the steps of the above-mentioned embodiments can be realized by software, hardware, firmware or any combination thereof. When realized by software, it can be realized in the form of a computer program product in whole or in part. The computer program product includes one or more computer instructions. The computer instructions can be stored in the above-mentioned computer readable storage medium.

[0255] In the above embodiments, the description of each embodiment has its own focus, and the parts not described or recorded in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0256] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0257] The above-described embodiments are only used to illustrate the technical solutions of the present application, but not limit them; although the present application is described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement to part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A method for recognizing throwing motions, characterized in that, The method includes: Collect user hand movement data; Candidate throwing data are determined from the motion data; Based on the time of generation of the extreme data of the candidate throwing data and the preset time within the preset first time period, it is determined whether the user throws within the first time period; The candidate throwing data includes motion data generated during the first time period.

2. The method as described in claim 1, characterized in that, The step of determining candidate data from the motion data includes: If the motion data generated during the first time period is detected to meet the preset conditions, the motion data generated during the first time period will be identified as the candidate throwing data.

3. The method as described in claim 2, characterized in that, The motion data includes at least one of the acceleration and angular velocity corresponding to the user's hand; The preset conditions include: The acceleration corresponding to the user's hand is greater than or equal to a preset acceleration threshold; And / or, The angular velocity corresponding to the user's hand is greater than or equal to a preset angular velocity threshold.

4. The method as described in claim 1, characterized in that, The step of determining whether the user throws within the first time period based on the generation time of the extreme value data of the candidate throwing data and a preset time within the preset first time period includes: If the time of generation is detected to match a preset time within the first time period, it is determined that the user threw the object within the first time period.

5. The method as described in claim 1, characterized in that, The step of determining whether the user throws within the first time period based on the generation time of the extreme value data of the candidate throwing data and a preset time within the preset first time period includes: If the time of generation is detected to match a preset time within the first time period, the throwing action recognition model determines whether the user threw within the first time period based on the candidate throwing data.

6. The method as described in claim 1, characterized in that, The preset time is any time within a preset interval. The distance between the starting point of the preset interval and the lower boundary of the first time period is greater than or equal to one-quarter of the first time period, and the distance between the ending point of the preset interval and the upper boundary of the first time period is greater than or equal to one-quarter of the first time period.

7. The method as described in claim 5, characterized in that, The preset time is the midpoint of the first time period.

8. The method according to any one of claims 1-7, characterized in that, The method further includes: If the user is detected to have cast during the first time period, the motion data is used to determine whether the user will retrieve the line during a preset second time period after the first time period. If it is detected that the user is collecting the line during the second time period, it is determined that the user's casting during the first time period was valid.

9. The method as described in claim 8, characterized in that, The method further includes: If no user is detected retrieving the line during the second time period, the user's casting during the first time period is determined to be invalid.

10. The method as described in claim 8, characterized in that, The step of determining whether the user will retract the cable within a preset second time period after the first time period based on the motion data includes: Based on the motion data, determine whether the fishing line moves uniformly during the second time period; If uniform movement of the fishing line is detected, it is determined that the user is reeling in the line during the second time period.

11. A throwing motion recognition device, characterized in that, The device includes a memory and a processor: The memory is used to store computer programs; The processor is configured to execute the computer program to implement the method as described in any one of claims 1-10.

12. A wearable device, characterized in that, The wearable device includes a motion data acquisition device and a throwing motion recognition device as described in claim 11; the wearable device is worn on the user's hand.

13. A method for recognizing throwing motions, characterized in that, The method is applied to a throwing motion recognition system, the system including a first intelligent device and a second intelligent device communicatively connected to the first intelligent device, the method comprising: Control the first smart device to collect the user's hand movement data; Candidate throwing data are determined from the motion data; The second intelligent device is controlled to determine whether the user throws within the first time period based on the generation time of the extreme value data of the candidate throwing data and the preset time within the preset first time period. The candidate throwing data includes motion data generated during the first time period.

14. A throwing motion recognition system, characterized in that, The system includes a first intelligent device, a second intelligent device communicatively connected to the first intelligent device, a memory, and a processor; The memory is used to store computer programs; The processor is used to execute the computer program to implement the method as described in claim 13.

15. The system as described in claim 14, characterized in that, The first smart device is a motion data acquisition device installed on the fishing rod; the second smart device is a wearable device or a server.

16. The system as described in claim 14, characterized in that, The first smart device is a wearable device that integrates a motion data acquisition device; the second smart device is a server.

17. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method as described in any one of claims 1-9 and 13.

Images