Smart shoes and action type recognition method, device, equipment, and storage medium thereof

Abstract

The present application discloses a smart shoe and its action type identification method, device, equipment, and storage medium. The method includes: obtaining the first air pressure change data generated by the current first smart shoe, and judging whether the first air pressure change data enters the squatting time zone; obtaining the second air pressure change data synchronized from the second smart shoe, and judging whether the second air pressure change data enters the squatting time zone; when the first air pressure change data and the second air pressure change data enter the squatting time zone at the same time, identifying that the squatting action type is executed; outputting an event notification representing the execution of the squatting action type to a computer device using the first smart shoe as an input device. The present application measures the air pressure value of the force-applying subject by setting an air pressure measurement unit in the somatosensory sensor device of the smart shoe, and uses the air pressure change data composed of multiple air pressure values ​​to determine the squatting time zone and accurately identify the squatting action type of the force-applying subject. Compared with the method of using IMU, the judgment is more accurate.

Description

Technical Field

[0001] The present application relates to the field of motion recognition technology, and in particular to a smart shoe and a motion type recognition method and device thereof, as well as a computer-readable storage medium. Background Art

[0002] An IMU (Inertial Measurement Unit) uses its accelerometer and / or gyroscope to detect the motion of an object within a carrier's coordinate system. IMUs are often used in motion-sensing gaming peripherals. So-called motion-sensing gaming is a new type of electronic gaming that transcends the traditional controller-based input method by translating body movements into a similar control method.

[0003] A currently popular method is to design relevant inputs into smart shoes. The smart shoes recognize the relevant data generated by the foot movements performed by the lower body to determine the user's movement pattern, identify the relevant foot movements, and input them into the game device through communication means such as Bluetooth, so that the user can participate in the interaction of somatosensory games.

[0004] However, due to the relatively traditional technical methods used by existing smart shoes, existing entertainment and fitness algorithms are generally limited to analyzing step count, cadence, and length. These algorithms rely heavily on three-axis data from the gyroscope, and generally rely on dynamic data generated by large-scale leg movements. However, for situations such as squatting, where the foot itself does not move, the data provided by the gyroscope often fails to meet the requirements, making it difficult to correctly identify the relevant action type. As a result, in applications such as sports applications, some smart shoe users cannot determine whether they have squatted, and some gaming software cannot use this method to determine whether the user has performed the action. Therefore, the existing smart shoe recognition technology is not ideal, especially for squatting. Summary of the Invention

[0005] The purpose of this application is to provide a method for identifying action types, which can at least achieve accurate identification of squatting action types by introducing air pressure change data.

[0006] Another object of the present application is to provide an action type recognition device, smart shoes, computer equipment and computer-readable storage medium.

[0007] To achieve the above objectives, the present application provides an action type recognition method, comprising:

[0008] Obtaining first air pressure change data generated by the current first smart shoe, and determining whether the first air pressure change data enters a squatting time zone;

[0009] Obtaining second air pressure change data synchronized from the second smart shoe, and determining whether the second air pressure change data enters a squatting time zone;

[0010] When the first air pressure change data and the second air pressure change data simultaneously enter the squatting time zone, it is recognized that the squatting action type is performed;

[0011] An event notification indicating that a squatting action type has been performed is output to a computer device using the first smart shoe as an input device.

[0012] In a preferred embodiment, the step of determining whether the first / second air pressure change data enters the squatting time zone includes:

[0013] Setting a calculation window of length N, sliding the calculation window for calculation, wherein each calculation window contains N air pressure values ​​continuously collected from the air pressure change data, where N is a natural number greater than 2;

[0014] Determining, based on the N air pressure values, a data feature presented in each calculation window to determine whether the corresponding air pressure change data is at a rising edge indicating that the smart shoe has landed;

[0015] The calculation window corresponding to the rising edge is determined as the time zone for entering the squatting position.

[0016] In a further embodiment, the step of determining, based on the N air pressure values, the data characteristics presented in each calculation window to determine whether the corresponding air pressure change data is on a rising edge includes:

[0017] Determine the maximum and minimum air pressure values ​​within the calculation window;

[0018] Comparing the difference between the maximum air pressure value and the minimum air pressure value to see if it exceeds a preset threshold;

[0019] When it exceeds the preset threshold, it is determined whether the maximum air pressure value lags behind or leads the minimum air pressure value in time. If it lags behind, the calculation window is determined as a rising edge representing the landing of the smart shoe; if it leads, the calculation window is determined as a falling edge representing the departure of the smart shoe.

[0020] In a preferred embodiment, the data characteristics of the calculation window representing the rising edge of the smart shoe landing show a pattern that the subsequent air pressure value is greater than the previous air pressure value, and the difference between the two exceeds a preset threshold.

[0021] In some embodiments, after the first air pressure change data and the second air pressure change data simultaneously enter the squatting time zone, the motion inertia data in the first smart shoe is used to obtain the displacement information generated by the first smart shoe or the second smart shoe in the squatting time zone. When the displacement information is less than a preset value or is zero, it is recognized that the squatting action type has been performed.

[0022] In a further embodiment, the action type identification method further includes the following steps:

[0023] Receive a notification instruction from the computer device, and drive the vibration sensors of the first smart shoe and the second smart shoe to vibrate and alarm according to the notification instruction.

[0024] To achieve the above objectives, the present application also provides an action type recognition device, which includes:

[0025] a first judgment module, configured to obtain first air pressure change data generated by the current first smart shoe, and determine whether the first air pressure change data enters a squatting time zone;

[0026] a second judgment module, configured to obtain second air pressure change data synchronized from the second smart shoe, and judge whether the second air pressure change data enters a squatting time zone;

[0027] An action recognition module is configured to recognize that a squatting action type is being performed when the first air pressure change data and the second air pressure change data simultaneously enter the squatting time zone;

[0028] The event notification module is configured to output an event notification indicating that a squatting action type has been performed to a computer device using the first smart shoe as an input device.

[0029] In a preferred embodiment, the first / second judgment module includes:

[0030] a sliding submodule, configured to set a calculation window of length N and slide the calculation window for calculation, wherein each calculation window contains N air pressure values ​​continuously collected from the air pressure change data, where N is a natural number greater than 2;

[0031] a judgment submodule, configured to judge the data features presented in each calculation window based on the N air pressure values, and determine whether the data features represent that the corresponding air pressure change data is at a rising edge representing that the smart shoe has landed;

[0032] The determination submodule is used to determine the calculation window corresponding to the rising edge as the time zone for entering the squatting position.

[0033] In a further embodiment, the judgment submodule is specifically used to:

[0034] Determine the maximum and minimum air pressure values ​​within the calculation window;

[0035] Comparing the difference between the maximum air pressure value and the minimum air pressure value to see if it exceeds a preset threshold;

[0036] When it exceeds the preset threshold, it is determined whether the maximum air pressure value lags behind or leads the minimum air pressure value in time. If it lags behind, the calculation window is determined as a rising edge representing the landing of the smart shoe; if it leads, the calculation window is determined as a falling edge representing the departure of the smart shoe.

[0037] In a preferred embodiment, in the judgment submodule, the data characteristics of the calculation window representing the rising edge of the smart shoe landing show a pattern that the subsequent air pressure value is greater than the previous air pressure value, and the difference between the two exceeds a preset threshold.

[0038] In some embodiments, in the action recognition module, after the first air pressure change data and the second air pressure change data simultaneously enter the squatting time zone, the motion inertia data in the first smart shoe is used to obtain the displacement information generated by the first smart shoe or the second smart shoe in the squatting time zone. When the displacement information is less than a preset value or is zero, it is recognized that the squatting action type is performed.

[0039] In a further embodiment, the action type recognition device further includes:

[0040] The alarm response module is used to receive a notification instruction from the computer device and drive the vibration sensors of the first smart shoe and the second smart shoe to vibrate and alarm according to the notification instruction.

[0041] To achieve the above objectives, the present application further provides a smart shoe, which includes a somatosensory sensing device and is wirelessly connected to another smart shoe having the same structure and function, wherein the somatosensory sensing device of the smart shoe includes:

[0042] An inertial measurement unit, used to detect the inertial data of the smart shoe movement to determine the displacement information of the smart shoe;

[0043] An air pressure measurement unit, used to detect air pressure change data generated by pressurizing the smart shoe to determine the squatting time zone;

[0044] a control unit, configured to detect whether first air pressure change data of the current smart shoe and second air pressure change data of another connected smart shoe simultaneously enter the squatting time zone, and when both air pressure change data enter the squatting time zone, identify that a squatting action type has been performed, and output an event notification indicating that the squatting action type has been performed to the computer device through the communication module;

[0045] The communication module is used to transmit the event notification to the computer device and to obtain the displacement information and air pressure change data generated by another smart shoe.

[0046] To achieve the above objectives, the present application also provides a computer device that is wirelessly connected to the smart shoes described above, and at least one process running in the computer device responds to the event notification to trigger another computer event to change the business process it is executing.

[0047] To achieve the above objectives, the present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the action type identification method described above are implemented.

[0048] Compared to the prior art, the technical solution provided by this application utilizes the air pressure change data generated by two smart shoes to determine whether each smart shoe is in a squatting motion state. By analyzing and utilizing the data characteristics presented by the two sets of air pressure change data jointly generated by the two smart shoes, it is promptly determined whether the motion state of the smart shoe has entered the squatting time zone. If both smart shoes have entered the squatting time zone, it is determined that both smart shoes have performed a squatting action type. This judgment mechanism can more accurately describe the characteristics of the human squatting motion than the results of traditional IMU judgments using gyroscope data. This is due to the fact that air pressure change data itself depends on the relationship between force and area, rather than spatial motion displacement information. Therefore, the present invention enriches the types of user actions that smart shoes can recognize and can accurately identify the user's squatting action. This has practical significance for the development of games, sports programs, interactive services, etc. based on the human squatting motion. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without any creative work.

[0050] Figure 1 This is a principle block diagram of a somatosensory sensing device of a smart shoe used in an embodiment of the present application;

[0051] Figure 2 A schematic structural diagram of an air pressure measurement unit used in a smart shoe according to an embodiment of the present application, which roughly shows a side view of the structure provided in the form of an insole;

[0052] Figure 3 A flowchart of an action type identification method provided in an embodiment of the present application;

[0053] Figure 4A flowchart of the process of determining the squatting time zone using air pressure change data in this application;

[0054] Figure 5 This is a principle block diagram of an action type recognition device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0055] It will be understood by those skilled in the art that, unless expressly stated otherwise, the singular forms "a", "an", "said" and "the" used herein may also include the plural forms. It should be further understood that the term "comprising" used in the specification of the present application refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be intermediate elements. In addition, "connected" or "coupled" as used herein may include wireless connections or wireless couplings. The term "and / or" used herein includes all or any units and all combinations of one or more associated listed items.

[0056] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those skilled in the art to which this application belongs. It should also be understood that terms such as those defined in common dictionaries should be understood to have meanings consistent with their meanings in the context of the prior art and will not be interpreted in an idealized or overly formal sense unless specifically defined as herein.

[0057] Those skilled in the art should be aware that although the various methods and devices of this application are described based on the same concepts and thus exhibit commonality, unless otherwise specified, these methods and devices can operate independently. Similarly, the various embodiments disclosed in this application are all based on the same inventive concept. Therefore, concepts with the same expression, as well as concepts that are appropriately transformed for convenience despite different expressions, should be understood as equivalent.

[0058] The core of this application is to provide a method, apparatus, smart shoe, computer device, and computer-readable storage medium for identifying the action type of smart shoes. The method obtains first air pressure change data generated by a first smart shoe to determine whether the first air pressure change data enters a squatting time zone. Simultaneously, it obtains second air pressure change data synchronized from a second smart shoe to determine whether the second air pressure change data enters a squatting time zone. When the first and second air pressure change data simultaneously enter the squatting time zone, it is identified as a squatting action type. Finally, an event notification indicating the execution of a squatting action type is output to a computer device using the first smart shoe as an input device. This allows for accurate identification of the squatting action type performed by the user of the smart shoes.

[0059] To make the purpose, technical solutions, and advantages of the embodiments of this application more clear, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0060] The somatosensory sensor device provided by the present invention is built into the smart shoe provided by the present invention. The circuit principle block diagram of the somatosensory sensor device is as follows: Figure 1 As shown, it uses a single-chip microcomputer 213 or other smart chip processor as its control unit, powered by a battery 214, to control the inertial measurement unit 211 and the air pressure measurement unit 12 to collect various corresponding sensor data, including the motion inertial data and air pressure change data, wherein the inertial measurement unit 211 is used to collect motion inertial data, and the air pressure measurement unit 12 is used to collect air pressure change data. After using these sensor data to identify the gait and movement type of the human foot, the relevant identification results and / or relevant data are sent to a personal computer, mobile terminal, smart TV or other device wirelessly connected to it through its communication module 22 for communication, so as to participate in somatosensory games or interact with health data. The communication module 22 is preferably Bluetooth or other near-field communication technology, and of course it does not exclude communication mechanisms such as mobile communication and WiFi. Those skilled in the art can flexibly choose. In addition, the single-chip microcomputer 213 can also receive commands from the device side through its communication module to control the vibration sensor 212 to vibrate in order to implement interaction with the user.

[0061] As mentioned above, the smart shoe is equipped with an inertial measurement unit (IMU), which is mainly used to obtain the aforementioned motion inertial data, that is, the relevant data that can be collected by the IMU as known to those skilled in the art. More specifically, in one embodiment, the smart shoe measures acceleration and angular velocity data through the accelerometer and gyroscope carried by the IMU itself, and measures acceleration and geomagnetic data through the magnetometer carried by the IMU itself. These data together constitute the aforementioned motion inertial data. The control unit can subsequently use this data to identify more types of movements.

[0062] The air pressure measuring unit 12, such as Figure 2 As shown in FIG, it is a module added in the form of an insole 10 in a smart shoe. Figure 2 As shown, a shoe-pad-shaped airbag 121 is provided, and a cavity is formed within the airbag 121. One or more air pressure sensors are placed in this cavity. When the airbag 121 is pressed by a human foot, the air pressure sensor can collect air pressure change data related to the foot movement. To ensure more uniform data collection by the air pressure sensor, a flexible pad 11 can be placed above the airbag 121 to improve the user's pressure experience. Typically, the air pressure change data can be represented and read in the form of an air pressure value. Therefore, in essence, the air pressure measurement unit 12 is a barometer constructed in the form of a shoe pad.

[0063] When users use the smart shoes of the present invention, they usually use them in a pair. The two smart shoes are usually preset as a master-slave relationship when leaving the factory, so that the two smart shoes can back up each other when necessary. When working, the first smart shoe can be responsible for establishing a wireless communication connection with the computer device, and the second smart shoe only needs to synchronize the relevant data and output results generated by itself to the first smart shoe. The first smart shoe, as the host, is responsible for centrally identifying the relevant action types.

[0064] It is understood that, if necessary, in one embodiment, the master-slave relationship between the first and second smart shoes can be switched at any time or formed into a disaster recovery relationship. That is, when the communication link between the first smart shoe and the computer device is disconnected, a control instruction can be sent to control the second smart shoe to establish a communication link with the computer device, replacing the first smart shoe in communicating with the computer device. In this case, the second smart shoe and the first smart shoe have swapped roles. Therefore, those skilled in the art should understand that the designation of the serial numbers of the first and second smart shoes in the present invention is the designation of the roles played by the smart shoes, not a specific smart shoe.

[0065] The following combination Figure 3 , Figure 3 This is a flow chart of a method for identifying action types of smart shoes provided in an embodiment of the present application.

[0066] It specifically includes the following steps:

[0067] Step S1100: Obtain first air pressure change data generated by the current first smart shoe, and determine whether the first air pressure change data enters a squatting time zone:

[0068] As previously described, the first smart shoe independently processes the first air pressure change data it generates. The first air pressure change data is represented by the changing relationship between continuously collected air pressure values. In one embodiment of the present invention, the air pressure change data is represented by a calculation window containing N air pressure values. After the control unit of the first smart shoe obtains the first air pressure change data from its air pressure sensor, it calculates it using the calculation window to determine whether the movement state of the first smart shoe has entered the squatting time zone.

[0069] The squatting time zone represents a time period, and the multiple air pressure values ​​in this time period constitute the air pressure change data for this time period. In the air pressure change data for this time period, the change in the air pressure values ​​has a trend of rapid increase, and the increase exceeds a certain threshold. The understanding of the concept of squatting time zone is similar to the following.

[0070] Step S1200: Acquire the second air pressure change data synchronized from the second smart shoe, and determine whether the second air pressure change data enters the squatting time zone:

[0071] The first smart shoe is also responsible for processing the second air pressure change data synchronized by the second smart shoe. Because the first and second smart shoes establish a real-time synchronization relationship via their respective Bluetooth modules by default, the first smart shoe can continuously obtain the air pressure values ​​generated by the air pressure sensor of the second smart shoe to generate the second air pressure change data. Similarly, the first smart shoe can use a calculation window to determine whether the second air pressure change data indicates that the second smart shoe's exercise state has entered the squatting time zone.

[0072] Step S1300: When the first air pressure change data and the second air pressure change data simultaneously enter the squatting time zone, it is recognized that the squatting action type is performed:

[0073] Since the two smart shoes are usually worn by the left and right feet of the human body, and when the human body, the force-applying subject, steps on the shoes to perform a squatting action, its gravity acts on the two smart shoes through its two feet. Therefore, when the first air pressure change data and the second air pressure change data generated by the two smart shoes both indicate entering the squatting time zone, the time when the two smart shoes enter the squatting time zone is preferably synchronized, thereby determining that the force-applying subject has performed a squatting action, thereby identifying the type of squatting action performed by the user.

[0074] Of course, those skilled in the art will understand that the requirement for time synchronization allows for a certain tolerance range, and the amplitude of the air pressure values ​​received by the two smart shoes is also allowed to have a certain difference. In some embodiments, the first air pressure change data and the second air pressure change data are allowed to be uneven in time and force due to the uneven instantaneous force applied to the user's feet. However, since the present invention uses a calculation window to accommodate the use of multiple sampled values ​​for calculation when determining the squatting time zone, and a time period is formed between the multiple sampled values, as long as the time span of the air pressure values ​​in the calculation window is long enough, the change relationship presented by the air pressure change data obtained by the two smart shoes will eventually tend to be highly similar or even consistent, and tend to be synchronized or substantially synchronized. Therefore, this factor does not actually hinder the implementation of the present invention.

[0075] Step S1400: Outputting an event notification indicating that a squatting action type has been performed to a computer device using the first smart shoe as an input device:

[0076] When the first smart shoe recognizes the squatting action type, it constructs a corresponding event notification and sends it to the wirelessly connected computer device, which serves as input data required by the computer device. For example, if the computer device is running a motion-sensing game program, the event notification sent by the first smart shoe indicating the squatting action type is received and responded to by the motion-sensing game program process, causing the process to trigger a new computer event, such as a screen change on the graphical user interface or an evaluation score associated with the action type.

[0077] It can be seen that by utilizing the air pressure change data according to the method of the present invention, it is possible to effectively and accurately identify whether the force-applying subject is performing a squatting action on a pair of smart shoes.

[0078] In another embodiment, in step S1300, the control unit of the first smart shoe is responsible for calculating the first air pressure change data to determine the corresponding squatting time zone, and is also responsible for calculating the second air pressure change data to determine the corresponding squatting time zone. It can be seen that the two air pressure change data can use a completely consistent calculation method. For this reason, step S1300 is further explained. Figure 4 The process of determining whether the first / second air pressure change data enters the squatting time zone in step S1300 includes the following specific steps:

[0079] Step S1301: Set a calculation window of length N, and slide the calculation window to perform calculations, wherein each calculation window contains N pressure values ​​continuously collected from the pressure change data, where N is a natural number greater than 2:

[0080] In one embodiment, a calculation window as shown below is used to cache 10 or other nominal numbers of the most recent air pressure values.

[0081] <![CDATA[P n-9 ]]> <![CDATA[P n-8 ]]> ... <![CDATA[P n-1 ]]> <![CDATA[P n ]]>

[0082] Where p is the air pressure value and n is the data subscript.

[0083] The data is continuously read by sliding according to the calculation window, and the next step of calculation is performed for each calculation window.

[0084] Step S1302: judging the data features presented in each calculation window based on the N air pressure values, and determining whether the corresponding air pressure change data is at a rising edge indicating that the smart shoe has landed.

[0085] This step is responsible for calculating the relationship between the various air pressure values ​​in each calculation window to determine whether the data feature represented by the calculation window is on the rising edge or the falling edge. Specifically, this step judges the data features presented by each calculation window based on the N air pressure values ​​in a calculation window, mainly to determine whether it represents that the corresponding air pressure change data is on the rising edge. For the judgment condition of the rising edge, the air pressure value at which the data feature is presented later is greater than the air pressure value at which it is presented earlier, and the difference between the two exceeds the preset threshold; for the judgment condition of the falling edge, the air pressure value at which the data feature is presented earlier is greater than the air pressure value at which it is presented later, and the difference between the two exceeds the preset threshold. As for the preset threshold involved, similarly, it can be determined by a person skilled in the art through statistical averaging of the change data of the actual air pressure measurement unit when it is in use. It can be understood that the specific number of this preset threshold will be a value based on experience and testing.

[0086] Correspondingly, to determine whether the calculation window is on the rising edge or the falling edge, the judgment method can be implemented by referring to the following code:

[0087] max_index,max_val=max(press_buff)

[0088] min_index,min_val=min(press_buff)

[0089] if(max_val-min_val>threshhold)

[0090] if(max_index>min_index)

[0091] is_up

[0092] if(max_index<min_index)

[0093] is_down

[0094] This code indicates that if the difference between the maximum and minimum values ​​in the window data exceeds a certain threshold value (threshhold), and the subscript max_index of the maximum value data (max_val) is greater than the subscript min_index of the minimum value data (min_val), then a rising edge is determined. If the difference between the maximum and minimum values ​​(max_val) in the window data exceeds a certain threshold value (threshhold), and the subscript max_index of the maximum value data (max_val) is less than the subscript min_index of the minimum value data (min_val), then a falling edge is determined. In a variant embodiment, the N sampled pressure values ​​contained in the calculation window can be divided into two halves for comparison to improve comparison efficiency.

[0095] According to the idea expressed in this code, it can be understood that it is implemented in the following steps:

[0096] Determine the maximum and minimum air pressure values ​​within the calculation window;

[0097] Comparing the difference between the maximum air pressure value and the minimum air pressure value to see if it exceeds a preset threshold;

[0098] When it exceeds the preset threshold, it is determined whether the maximum air pressure value lags behind or leads the minimum air pressure value in time. If it lags behind, the calculation window is determined as a rising edge representing the landing of the smart shoe; if it leads, the calculation window is determined as a falling edge representing the departure of the smart shoe.

[0099] Step S1303: Determine the calculation window corresponding to the rising edge as the time zone for entering the squatting position:

[0100] It can be understood that if a rising edge is detected, it means that the smart shoe has entered a state of contact with the ground; if a falling edge is detected, it is usually off the ground. Among them, the time period corresponding to the calculation window when contacting the ground is regarded as the squatting time zone, so the zero-speed starting point can be determined through this calculation window. When there is no external force pressing on the air pressure sensor, the air pressure value is at its minimum value. At this time, the smart shoe is off the ground or unloaded; when external force is applied, the air pressure value rises. At this time, the smart shoe is on the ground. Therefore, by utilizing this feature and analyzing the relationship between the rising edge and the falling edge, it is possible to know how to determine the zero-speed state during strenuous exercise. As mentioned above, the air pressure change data corresponding to the calculation window corresponding to the rising edge can usually be directly determined as the basis for the zero-speed moment, and the time period of the calculation window can be determined to enter the squatting time zone.

[0101] This embodiment demonstrates that the air pressure change data is constrained using a calculation window containing N air pressure values. The value of N effectively synchronizes the temporal rhythm of the air pressure change data of the two smart shoes. When N is within an appropriate range, the accuracy of squatting can be further ensured. This range can be determined by those skilled in the art based on actual measurements based on the principles disclosed in this invention, for example, 10 to 50 consecutively sampled air pressure values, preferably 10 to 30.

[0102] In another embodiment, step S1300 can use the motion inertia data in the first smart shoe to obtain the displacement information generated by the first smart shoe or the second smart shoe in the squatting time zone after the first air pressure change data and the second air pressure change data simultaneously enter the squatting time zone. When the displacement information is less than a preset value or is zero, it is identified as a squatting action type. This embodiment can further ensure that when identifying a squatting posture, erroneous identification caused by irregular force-applying movements can be avoided. For example, when the force-applying body causes the smart shoes to slide on the ground, the pressure conditions of the two smart shoes do not change much, and sometimes they may be mistakenly identified as squatting. In this case, the influence of the displacement factor can be eliminated by using the acceleration data in the motion inertia data generated by the IMU, which is well known to those skilled in the art, to further improve the accuracy of the present invention in determining squatting movements.

[0103] In another embodiment, the following steps may be performed after step S1400: receiving a notification instruction from the computer device, and driving the vibration sensors of the first smart shoe and the second smart shoe to vibrate and alarm according to the notification instruction.

[0104] This step assumes that a data communication link has been established between the smart shoe and the computer device. Typically, the smart shoe acts as an input device for the computer device, providing it with an event notification corresponding to the squatting action. Upon receiving this event notification, the process responsible for handling it can respond by returning a notification instruction. Upon receiving this instruction, the first and second smart shoes activate their own vibration sensors to generate a vibration alarm, thereby notifying the force-applying subject of the notification.

[0105] In another embodiment, when a squatting time zone is determined for the first and second smart shoes, a zero-speed correction can be triggered for the motion inertia data obtained by the IMUs in the corresponding smart shoes. Techniques for implementing zero-speed correction for motion inertia data are well known to those skilled in the art and will not be elaborated upon here. Because using barometric pressure change data to accurately determine the squatting time zone eliminates the need for gyroscope data and provides observational information for zero-speed correction using gyroscope data in traditional techniques, this helps improve the accuracy of IMU zero-speed correction and overcomes the potential problem of traditional zero-speed correction methods that can lead to misjudgment due to the inability to identify the force-applying subject intentionally hovering the smart shoe.

[0106] As mentioned above, when the smart shoes implementing the method of the present invention are used to interact with computer devices such as smart TVs, mobile terminals, and game consoles, they can be used as input devices for user commands. In this case, the smart shoes establish a communication connection with the computer device through its communication module, and output the recognition results obtained after the action type recognition to the computer device in real time. When these computer devices start related game programs or health data APPs, these recognition results can also be regarded as related user commands or user data. Accordingly, the program process of the computer device can also feedback information or send notifications to the smart shoes in response to the recognition results, such as sending a notification instruction to control the vibration alarm of the vibration sensor of the smart shoes, and so on. On the basis that the smart shoes can provide more accurate recognition results of the user's foot action type based on the air pressure change data, it will inevitably improve the human-computer interaction experience in these application scenarios.

[0107] To sum up, this application measures the air pressure value of the force-applying subject by setting an air pressure measurement unit in the somatosensory sensing device of the smart shoe, and uses the air pressure change data composed of multiple air pressure values ​​to determine the squatting time zone and accurately identify the squatting action type of the force-applying subject. Compared with the IMU method, the judgment accuracy is high, the functions of the smart somatosensory products are enriched, and it helps to enrich and broaden the means of human-computer interaction.

[0108] The following combination Figure 5 , Figure 5This is a block diagram of a device for identifying an action type provided by an embodiment of the present application. The device includes: a first judgment module 1100, configured to obtain first air pressure change data generated by a first smart shoe and determine whether the first air pressure change data has entered a squatting time zone; a second judgment module 1200, configured to obtain second air pressure change data synchronized from a second smart shoe and determine whether the second air pressure change data has entered a squatting time zone; an action recognition module 1300, configured to identify a squatting action type as being performed when both the first air pressure change data and the second air pressure change data enter the squatting time zone; and an event notification module 1400, configured to output an event notification indicating that a squatting action type has been performed to a computer device using the first smart shoe as an input device.

[0109] In a preferred embodiment, the first / second judgment module includes: a sliding submodule, used to set a calculation window with a length of N, and slide the calculation window for calculation, wherein each calculation window contains N air pressure values ​​continuously collected from the air pressure change data, and N is a natural number greater than 2; a judgment submodule, used to judge the data characteristics presented by each calculation window based on the N air pressure values, and determine whether it represents that the corresponding air pressure change data is on the rising edge representing the landing of the smart shoe; a determination submodule, used to determine the calculation window corresponding to the rising edge as the time zone for entering the squatting position.

[0110] In a further embodiment, the judgment submodule is specifically used to: determine the maximum air pressure value and the minimum air pressure value in the calculation window; compare whether the difference between the maximum air pressure value and the minimum air pressure value exceeds a preset threshold; when it exceeds the preset threshold, determine whether the maximum air pressure value lags behind or leads the minimum air pressure value in time, if it lags behind, determine the calculation window as a rising edge representing the landing of the smart shoe; if it leads, determine the calculation window as a falling edge representing the departure of the smart shoe.

[0111] In a preferred embodiment, in the judgment submodule, the data characteristics of the calculation window representing the rising edge of the smart shoe landing show a pattern that the subsequent air pressure value is greater than the previous air pressure value, and the difference between the two exceeds a preset threshold.

[0112] In some embodiments, in the action recognition module, after the first air pressure change data and the second air pressure change data simultaneously enter the squatting time zone, the motion inertia data in the first smart shoe is used to obtain the displacement information generated by the first smart shoe or the second smart shoe in the squatting time zone. When the displacement information is less than a preset value or is zero, it is recognized that the squatting action type is performed.

[0113] In a further embodiment, the action type recognition device further includes: an alarm response module, which is used to receive a notification instruction from the computer device and drive the vibration sensors of the first smart shoe and the second smart shoe to vibrate and alarm according to the notification instruction.

[0114] The present application also provides an embodiment of a smart shoe, which includes: the smart shoe includes a somatosensory sensing device and is wirelessly connected to another smart shoe with the same structure and function, wherein the somatosensory sensing device of the smart shoe includes: an inertial measurement unit 211, used to detect inertial data of the smart shoe movement to determine displacement information of the smart shoe; an air pressure measurement unit 12, used to detect air pressure change data generated by pressurization of the smart shoe to determine a squatting time zone; a control unit, used to detect whether first air pressure change data of the current smart shoe and second air pressure change data of another smart shoe connected to it enter the squatting time zone at the same time, and when both air pressure change data enter the squatting time zone, identify that a squatting action type has been performed, and output an event notification indicating that the squatting action type has been performed to a computer device through a communication module; a communication module 22, used to transmit the event notification to the computer device and to obtain the displacement information and air pressure change data generated by the other smart shoe.

[0115] The present application also provides a computer device that is wirelessly connected to the smart shoes described above. At least one process running in the computer device responds to the event notification and triggers another computer event to change the business process it is executing.

[0116] The present application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the action type identification method described above. The storage medium may include: a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, among other media capable of storing program code.

[0117] Because the situation is complicated, it is impossible to list them one by one for explanation. Those skilled in the art should be aware that there may be many examples based on the basic method principles provided by this application in combination with actual conditions, and all of them should be within the scope of protection of this application without sufficient creative work.

[0118] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Reference can be made to the common and similar parts between the various embodiments. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method description.

[0119] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0120] This document uses specific examples to illustrate the principles and implementation methods of this application. The description of the above examples is only intended to help understand the method and core ideas of this application. It should be noted that for those skilled in the art, without departing from the principles of this application, various improvements and modifications can be made to this application, and such improvements and modifications also fall within the scope of protection of the claims of this application.

[0121] It should also be noted that, in this specification, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or device comprising the element.

Claims

1. A smart shoe, comprising a somatosensory sensor and wirelessly connected to another smart shoe having the same structure and function, characterized in that: The somatosensory sensing device of the smart shoe comprises: An inertial measurement unit, used to detect the inertial data of the smart shoe movement to determine the displacement information of the smart shoe; An air pressure measuring unit, used to detect air pressure change data generated by pressurizing the smart shoe to determine the squatting time zone, wherein the air pressure measuring unit provides a cavity inside the insole-shaped airbag, and one or more air pressure sensors are arranged in the cavity. When the airbag is pressed by a human foot, the air pressure sensor collects air pressure change data related to the foot movement; A control unit is used to detect whether the first air pressure change data of the current smart shoe and the second air pressure change data of another smart shoe connected thereto enter the squatting time zone at the same time, and when both air pressure change data enter the squatting time zone, identify that the squatting action type is executed, and output an event notification representing the execution of the squatting action type to the computer device through the communication module, and when the communication link between the current smart shoe and the computer device is disconnected, send a control instruction to control the other smart shoe to establish a communication link with the computer device, so as to communicate with the computer device instead of the current smart shoe; A communication module, used to transmit the event notification to a computer device and to obtain displacement information and air pressure change data generated by another smart shoe; The step of judging whether the first / second air pressure change data enters the squatting time zone includes: setting a calculation window with a length of N, sliding the calculation window for calculation, wherein each calculation window contains N air pressure values ​​continuously collected from the air pressure change data, and N is a natural number greater than 2; judging the data features presented in each calculation window according to the N air pressure values, and determining whether it indicates that the corresponding air pressure change data is at a rising edge indicating that the smart shoe has landed; and determining the calculation window corresponding to the rising edge as entering the squatting time zone; The step of judging the data features presented in each calculation window according to the N air pressure values ​​to determine whether it indicates that the corresponding air pressure change data is at a rising edge includes: determining the maximum air pressure value and the minimum air pressure value in the calculation window; comparing whether the difference between the maximum air pressure value and the minimum air pressure value exceeds a preset threshold; when it exceeds the preset threshold, determining whether the maximum air pressure value lags behind or leads the minimum air pressure value in time, if it lags behind, determining the calculation window as a rising edge indicating that the smart shoe touches the ground; if it leads, determining the calculation window as a falling edge indicating that the smart shoe leaves the ground; The data characteristics of the calculation window representing the rising edge of the smart shoe landing show a rule that the subsequent air pressure value is greater than the previous air pressure value, and the difference between the two exceeds a preset threshold.

2. A method for identifying action types, characterized in that: The control unit of the smart shoe according to claim 1 comprises: Acquire first air pressure change data generated by the current first smart shoe, and determine whether the first air pressure change data enters a squatting time zone; Acquire the second air pressure change data synchronized from the second smart shoe, and determine whether the second air pressure change data enters the squatting time zone; When the first air pressure change data and the second air pressure change data simultaneously enter the squatting time zone, it is recognized that the squatting action type is performed; An event notification indicating that a squatting action type has been performed is output to a computer device using the first smart shoe as an input device.

3. The method according to claim 2, characterized in that: After the first air pressure change data and the second air pressure change data simultaneously enter the squatting time zone, the motion inertia data in the first smart shoe is used to obtain the displacement information generated by the first smart shoe or the second smart shoe in the squatting time zone. When the displacement information is less than a preset value or is zero, it is recognized that the squatting action type is performed.

4. The method according to claim 2, characterized in that: The following steps are also included: A notification instruction of the computer device is received, and according to the notification instruction, a vibration sensor of each of the first smart shoe and the second smart shoe is driven to vibrate and give an alarm.

5. An action type recognition device, characterized in that: The control unit of the smart shoe according to claim 1 comprises: A first judgment module is used to obtain the first air pressure change data generated by the current first smart shoe, and judge whether the first air pressure change data enters the squatting time zone; A second judgment module is used to obtain the second air pressure change data synchronized from the second smart shoe, and judge whether the second air pressure change data enters the squatting time zone; An action recognition module is used to recognize that a squatting action type is performed when the first air pressure change data and the second air pressure change data enter the squatting time zone at the same time; The event notification module is used to output an event notification representing the type of squatting action performed to a computer device using the first smart shoe as an input device.

6. A computer device, characterized in that: The computer device is wirelessly connected to the smart shoe as claimed in any one of claims 2 to 4, and at least one process running in the computer device responds to the event notification to trigger another computer event to change the business process being executed by itself.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the action type recognition method according to any one of claims 2 to 4 are implemented.

Images