A human posture recognition system

Abstract

One or more embodiments of the present specification relate to a human gesture recognition system, comprising: at least one group of ultrasonic sensors, each group of ultrasonic sensors comprising: an ultrasonic transmitter for transmitting ultrasonic waves and an ultrasonic receiver for receiving the ultrasonic waves, wherein the ultrasonic transmitter and the ultrasonic receiver are respectively located at different parts of the user's body; and a processor configured to recognize the user's gesture based on position information of the ultrasonic transmitter and the ultrasonic receiver, information about the ultrasonic waves transmitted by the ultrasonic transmitter, and information about the ultrasonic waves received by the ultrasonic receiver.

Description

Technical Field

[0001] The present application relates to the technical field of posture recognition, and in particular to a human posture recognition system. Background Art

[0002] With the continuous advancement of science and technology, human posture recognition is widely used in many fields, including sports and fitness, gaming, healthcare, and wearable electronics. Currently, human posture recognition may require the use of multiple sensors, including infrared sensors and inertial sensors. However, the accuracy of human posture recognition using infrared sensors or inertial sensors is low, making the application of human posture recognition difficult.

[0003] Therefore, the present application hopes to provide a human posture recognition system to improve the accuracy of human posture recognition. Summary of the Invention

[0004] An embodiment of the present specification provides a human gesture recognition system, comprising: at least one group of ultrasonic sensors, each group of ultrasonic sensors comprising: an ultrasonic transmitter for transmitting ultrasonic waves and an ultrasonic receiver for receiving the ultrasonic waves, wherein the ultrasonic transmitter and the ultrasonic receiver are respectively located at different parts of the user's body; and a processor configured to recognize the user's gesture based on position information of the ultrasonic transmitter and the ultrasonic receiver, information about the ultrasonic waves transmitted by the ultrasonic transmitter, and information about the ultrasonic waves received by the ultrasonic receiver.

[0005] In some embodiments, the invention further comprises clothing, wherein the at least one set of ultrasound sensors is integrated into the clothing.

[0006] In some embodiments, the ultrasonic transmitter and the ultrasonic receiver are distributed on the sleeves or trouser legs of the garment. When the user wears the garment, the ultrasonic transmitter and the ultrasonic receiver are respectively located on both sides of the user's limbs corresponding to the elbow joints or knee joints.

[0007] In some embodiments, the identifying the user's posture based on the position information of the ultrasonic transmitter and the ultrasonic receiver, the information of the ultrasonic transmitter transmitting the ultrasonic wave, and the information of the ultrasonic receiver receiving the ultrasonic wave includes: obtaining the distance between the ultrasonic transmitter and the ultrasonic transmitter based on the information of the ultrasonic transmitter transmitting the ultrasonic wave and the information of the ultrasonic receiver receiving the ultrasonic wave; and obtaining the bending angle between the two sides of the limbs corresponding to the elbow joint or the knee joint based on the distance between the ultrasonic transmitter and the ultrasonic transmitter and the position information of the ultrasonic transmitter and the ultrasonic receiver.

[0008] In some embodiments, each group of ultrasonic sensors includes an ultrasonic transmitter and at least two ultrasonic receivers, and the at least two ultrasonic receivers include a first ultrasonic receiver and a second ultrasonic receiver. The identifying the user's posture based on the position information of the ultrasonic transmitter and the ultrasonic receiver, the information of the ultrasonic transmitter transmitting the ultrasonic wave, and the information of the ultrasonic receiver receiving the ultrasonic wave includes: determining a first distance between the ultrasonic transmitter and the first ultrasonic transmitter and a second distance between the ultrasonic transmitter and the second ultrasonic transmitter based on the information of the ultrasonic transmitter transmitting the ultrasonic wave, the information of the first ultrasonic receiver receiving the ultrasonic wave, and the information of the second ultrasonic receiver receiving the ultrasonic wave; and determining the rotation angle between the two sides of the limbs corresponding to the elbow joint or the knee joint based on the first distance and the second distance.

[0009] In some embodiments, when the elbow joint or knee joint is in a bent state, the sides of the limbs on both sides that are opposite to each other are characterized as the inner sides of the limbs corresponding to the elbow joint or knee joint, and the ultrasonic transmitter and the ultrasonic receiver are located on the inner sides of the limbs corresponding to the elbow joint or knee joint.

[0010] In some embodiments, when the user stretches out his arms or legs, the distance between the ultrasound transmitter and the ultrasound receiver is no less than 10 cm.

[0011] In some embodiments, the ultrasonic transmitter includes an output end for emitting the ultrasonic wave, and the output end faces away from the clothing; the ultrasonic receiver includes a receiving end for receiving the ultrasonic wave, and the receiving end faces away from the clothing; the angle between the plane where the output end is located and the plane where the receiving end is located is not greater than 170°.

[0012] In some embodiments, the ultrasonic transmitter is tilted relative to the user's limb contact part below it and toward the ultrasonic receiver, and the angle between the normal direction of the output end and the normal direction of the user's limb contact part below the ultrasonic transmitter is not less than 5°.

[0013] In some embodiments, the ultrasonic receiver is tilted relative to the user's limb contact part below it and toward the ultrasonic transmitter, and the angle between the normal direction of the receiving end and the normal direction of the user's limb contact part below the ultrasonic receiver is not less than 5°.

[0014] In some embodiments, the ultrasonic transmitter is located at the upper arm of the human body in the garment, and the ultrasonic receiver is located at the torso of the human body in the garment, and the ultrasonic receiver and the ultrasonic receiver cooperate to identify the posture of the upper arm relative to the torso.

[0015] In some embodiments, the posture of the upper arm relative to the torso includes the angle of the upper arm relative to the torso, and identifying the posture of the upper arm relative to the torso includes: obtaining the distance between the ultrasonic transmitter and the ultrasonic transmitter based on the information of the ultrasonic transmitter transmitting the ultrasonic wave and the information of the ultrasonic receiver receiving the ultrasonic wave; and determining the angle of the upper arm relative to the torso based on the distance between the ultrasonic transmitter and the ultrasonic transmitter and the position information of the ultrasonic transmitter and the ultrasonic receiver.

[0016] In some embodiments, the ultrasonic transmitter includes at least two ultrasonic transmitters, the ultrasonic receiver includes at least three ultrasonic receivers, and the identifying the posture of the user's upper arm relative to the torso includes: determining the position change information of the at least two ultrasonic transmitters based on the position information of the at least three ultrasonic receivers; and identifying the movement state of the upper arm relative to the torso based on the position change information of the at least two ultrasonic transmitters.

[0017] In some embodiments, the at least three ultrasound receivers are not in the same straight line.

[0018] In some embodiments, the distance between the at least two ultrasound transmitters is no less than 1 cm, and the distance between the at least three ultrasound receivers is no less than 1 cm.

[0019] In some embodiments, the parts of the garment corresponding to the torso of the human body include at least one of the front of the left shoulder, the front of the right shoulder, the left waist, the right waist, and the chest.

[0020] In some embodiments, the ultrasonic transmitter and the ultrasonic receiver are respectively located at the two trouser legs of the garment, and the identifying the user's posture based on the position information of the ultrasonic transmitter and the ultrasonic receiver, the information of the ultrasonic transmitter transmitting the ultrasonic wave, and the information of the ultrasonic receiver receiving the ultrasonic wave includes: obtaining the distance between the ultrasonic transmitter and the ultrasonic transmitter based on the information of the ultrasonic transmitter transmitting the ultrasonic wave and the information of the ultrasonic receiver receiving the ultrasonic wave; and determining the posture of the user's leg based on the distance between the ultrasonic transmitter and the ultrasonic transmitter and the position information of the ultrasonic transmitter and the ultrasonic receiver.

[0021] In some embodiments, each group of the ultrasonic sensors includes a plurality of ultrasonic transmitters, and the time points at which the plurality of ultrasonic transmitters emit ultrasonic waves are spaced apart.

[0022] In some embodiments, the time interval between two adjacent ultrasonic transmitters emitting ultrasonic waves is greater than 2.9 ms.

[0023] In some embodiments, each group of ultrasonic sensors includes a plurality of ultrasonic transmitters, and the ultrasonic waves emitted by the plurality of ultrasonic transmitters have different frequencies.

[0024] In some embodiments, each group of ultrasonic sensors includes a plurality of ultrasonic transmitters, and the ultrasonic waves emitted by the plurality of ultrasonic transmitters have different codes. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The present application will be further described in the form of exemplary embodiments, which will be described in detail with reference to the accompanying drawings. These embodiments are not limiting, and in these embodiments, like numbers represent like structures, wherein:

[0026] Figure 1 is an exemplary framework diagram of a human gesture recognition system according to some embodiments of this specification;

[0027] Figure 2 is an exemplary schematic diagram of an ultrasonic transmitter and an ultrasonic receiver distributed on both sides of a user's limbs corresponding to the elbow joint or knee joint according to some embodiments of this specification;

[0028] Figure 3 is an exemplary flow chart of recognizing user gestures according to some embodiments of this specification;

[0029] Figure 4 is an exemplary schematic diagram of recognizing user gestures according to some embodiments of this specification;

[0030] Figure 5 is an exemplary flow chart of recognizing user gestures according to other embodiments of this specification;

[0031] Figure 6A is an exemplary schematic diagram of recognizing user gestures according to other embodiments of this specification;

[0032] Figure 6B is a side view of the distribution of exemplary ultrasound transmitters and ultrasound receivers on both sides of the limbs corresponding to the elbow joint or knee joint according to some embodiments of this specification;

[0033] Figure 6C is a top view of the distribution of exemplary ultrasound transmitters and ultrasound receivers on both sides of the limbs corresponding to the elbow joints or knee joints according to some embodiments of this specification;

[0034] Figure 7 is an exemplary schematic diagram of an arrangement position of an ultrasonic sensor according to some embodiments of this specification;

[0035] Figure 8is an exemplary schematic diagram of another arrangement position of an ultrasonic sensor according to some embodiments of this specification;

[0036] Figure 9 is a schematic diagram of the distribution of ultrasonic transmitters and ultrasonic receivers on the upper arm and torso according to some embodiments of this specification;

[0037] Figure 10 is an exemplary schematic diagram of the arrangement positions of ultrasonic sensors according to some embodiments of this specification;

[0038] Figure 11 is an exemplary flow chart of a method for determining an angle of an upper arm relative to a torso according to some embodiments of the present specification;

[0039] Figure 12 is an exemplary flow chart of a method for identifying the motion state of an upper arm relative to a torso according to some embodiments of this specification;

[0040] Figure 13 is an exemplary schematic diagram of the arrangement positions of ultrasonic sensors according to other embodiments of this specification;

[0041] Figure 14 is an exemplary schematic diagram of a three-point positioning method according to some embodiments of this specification;

[0042] Figure 15 is a schematic diagram of the distribution of ultrasound transmitters and ultrasound receivers on the thigh and calf according to some embodiments of this specification;

[0043] Figure 16 is an exemplary flow chart of a method for determining the posture of a user's legs according to some embodiments of this specification;

[0044] Figure 17 This is an exemplary schematic diagram of implementing time-division multiplexing based on ultrasonic sensors according to some embodiments of this specification. DETAILED DESCRIPTION

[0045] To more clearly illustrate the technical solutions of the embodiments of this application, the following is a brief introduction to the drawings required for describing the embodiments. Obviously, the drawings described below are merely examples or embodiments of this application. Those skilled in the art can apply this application to other similar scenarios based on these drawings without inventive effort. Unless otherwise apparent from the context or otherwise noted, the same reference numerals in the figures represent the same structure or operation.

[0046] It should be understood that the terms "system," "device," "unit," and / or "module" used herein are a method for distinguishing different components, elements, parts, portions, or assemblies at different levels. However, if other terms can achieve the same purpose, the terms may be replaced by other expressions.

[0047] As used in this application and the claims, unless the context clearly indicates otherwise, the words "a," "an," "an," and / or "the" are not intended to refer to the singular but may include the plural. Generally speaking, the terms "comprises" and "include" only indicate the inclusion of the steps and elements specifically identified, and these steps and elements do not constitute an exclusive list. A method or apparatus may also include other steps or elements.

[0048] Flowcharts are used in this application to illustrate the operations performed by the systems according to the embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed in exact order. Instead, the steps may be processed in reverse order or simultaneously. Furthermore, other operations may be added to these processes, or one or more operations may be removed from these processes.

[0049] Embodiments of this specification provide a human gesture recognition system. In some embodiments, the human gesture recognition system may include at least one set of ultrasonic sensors and a processor. In some embodiments, each set of ultrasonic sensors may include an ultrasonic transmitter for transmitting ultrasonic waves and an ultrasonic receiver for receiving ultrasonic waves, wherein the ultrasonic transmitter and ultrasonic receiver are located at different parts of the user's body. For example, the ultrasonic transmitter and ultrasonic receiver may be located on the upper arm and lower arm on either side of the user's elbow joint, respectively. In another example, the ultrasonic transmitter and ultrasonic receiver may be located on the thigh and lower leg on either side of the user's knee joint, respectively. In another example, the ultrasonic transmitter and ultrasonic receiver may be located on the upper arm and torso, respectively. In another example, the ultrasonic transmitter and ultrasonic receiver may be located on each leg, respectively. The processor is configured to recognize the user's gesture based on position information of the ultrasonic transmitter and ultrasonic receiver, information about ultrasonic waves transmitted by the ultrasonic transmitter, and information about ultrasonic waves received by the ultrasonic receiver. The human gesture recognition system provided in embodiments of this specification employs ultrasonic transmitters and ultrasonic receivers located at different parts of the user's body. The processor can obtain the distances between the ultrasonic transmitter and ultrasonic receiver at different parts of the user's body based on information about ultrasonic waves transmitted by the ultrasonic transmitter and information about ultrasonic waves received by the ultrasonic receiver. Furthermore, the positions of the ultrasonic transmitter and the ultrasonic receiver on the user's body are fixed, and a spatial coordinate system can be established with any point of the human body in a standing posture as the origin to obtain the position information (for example, three-dimensional coordinates) of the ultrasonic transmitter and the ultrasonic receiver of different parts of the user's body. When the user is exercising, the processor can determine the posture of the user's limbs and the posture of the limbs relative to the torso based on the position information of the ultrasonic transmitter and the ultrasonic receiver, the information of the ultrasonic transmitter transmitting ultrasonic waves, and the information of the ultrasonic receiver receiving ultrasonic waves, thereby identifying the user's whole-body posture. The human body posture recognition system provided in this specification can improve the accuracy of user posture recognition by designing the position and / or number of the ultrasonic generator and ultrasonic receiver in the ultrasonic sensor. In addition, the human body posture recognition system provided in this specification can set different numbers and / or positions of ultrasonic sensors for different human joints or parts to improve the accuracy of the user's posture obtained.

[0050] Figure 1 This is an exemplary framework diagram of a human posture recognition system according to some embodiments of this specification.

[0051] The human body posture recognition system 100 may refer to a system for recognizing the posture of a human body. For example, when the posture of a human body changes, such as when a user bends his arm, the human body posture recognition system 100 may recognize that the user's arm has been bent based on the bending of the user's arm. Figure 1As shown, the human gesture recognition system 100 may include at least one set of ultrasonic sensors 110 and a processor 120. In some embodiments, the human gesture recognition system 100 may include multiple sets of ultrasonic sensors 110, which may be distributed across different parts of the user. For example, the multiple sets of ultrasonic sensors may be located at any one or more of the user's limbs (e.g., arms, legs, etc.), torso (e.g., shoulders, chest, back, waist, etc.), or head.

[0052] In some embodiments, each set of ultrasonic sensors 110 may include an ultrasonic transmitter 131 for transmitting ultrasonic waves and an ultrasonic receiver 132 for receiving ultrasonic waves. In some embodiments, the ultrasonic transmitter 131 and the ultrasonic receiver 132 are located at different parts of the user's body. For example, the ultrasonic transmitter and the ultrasonic receiver may be located on the upper arm and lower arm on either side of the user's elbow, respectively. In another example, the ultrasonic transmitter and the ultrasonic receiver may be located on the thigh and lower leg on either side of the user's knee, respectively. In another example, the ultrasonic transmitter and the ultrasonic receiver may be located on the upper arm and torso, respectively. In another example, the ultrasonic transmitter and the ultrasonic receiver may be located on both legs, respectively. In some embodiments, each set of ultrasonic sensors 110 may include at least one ultrasonic transmitter 131 and at least one ultrasonic receiver 132. For example, each set of ultrasonic sensors 110 may include one ultrasonic transmitter 131 and multiple ultrasonic receivers 132. In another example, each set of ultrasonic sensors 110 may include multiple ultrasonic transmitters 131 and one ultrasonic receiver 132. In another example, each set of ultrasonic sensors 110 may include multiple ultrasonic transmitters 131 and multiple ultrasonic receivers 132. In some embodiments, when the human gesture recognition system 100 includes multiple sets of ultrasonic sensors 110, the number of ultrasonic transmitters 131 and ultrasonic receivers 132 corresponding to the multiple sets of ultrasonic sensors 110 may be the same or different. For example, the number of ultrasonic transmitters 131 and ultrasonic receivers 132 distributed on the upper arms and forearms on both sides of the user's elbow joints may be smaller than the number of ultrasonic transmitters 131 and ultrasonic receivers 132 distributed on the inner sides of the upper arms and torso of the user.

[0053] Ultrasonic transmitter 131 may refer to a device capable of transmitting ultrasonic waves. Ultrasonic transmitter 131 may convert electrical signals into ultrasonic waves and transmit them. In some embodiments, ultrasonic transmitter 131 may include, but is not limited to, a magnetostrictive transmitter, a piezoelectric ultrasonic transmitter, a micromechanical ultrasonic transmitter, and the like.

[0054] Ultrasonic receiver 132 may refer to a device capable of receiving ultrasonic waves. Ultrasonic receiver 132 may receive ultrasonic waves and convert them into electrical signals. In some embodiments, ultrasonic receiver 132 may include, but is not limited to, a magnetostrictive receiver, a piezoelectric ultrasonic receiver, a micromechanical ultrasonic receiver, and the like.

[0055] The processor 120 can be used to process information and / or data related to the human gesture recognition system 100, for example, processing the position information of the ultrasonic transmitter and the ultrasonic receiver, the information of ultrasonic transmission of ultrasonic waves by the ultrasonic transmitter, and the information of ultrasonic reception of ultrasonic waves by the ultrasonic receiver. The processor 120 can process data, information, and / or processing results obtained from other devices or system components, and execute program instructions based on these data, information, and / or processing results to perform one or more functions described in this specification. By way of example only, the processor 120 may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), an application-specific instruction set processor (ASIP), a reduced instruction set computer (RISC), a microprocessor, or any combination thereof.

[0056] In some embodiments, the processor 120 may be configured to recognize the user's gesture based on the position information of the ultrasonic transmitter and the ultrasonic receiver, the information of ultrasonic waves transmitted by the ultrasonic transmitter, and the information of ultrasonic waves received by the ultrasonic receiver.

[0057] The position information of the ultrasonic transmitter and the ultrasonic receiver may refer to the spatial distribution information of the ultrasonic transmitter and the ultrasonic receiver on the user's body. For example, the position of the ultrasonic transmitter on the user's body, the position of the ultrasonic receiver on the user's body, etc. The position information of the ultrasonic transmitter and the ultrasonic receiver may be represented by coordinates or the like. For example, the positions of the ultrasonic transmitter and the ultrasonic receiver on the user's body are fixed, and a spatial coordinate system may be established with any point of the human body in a standing posture as the origin to obtain the position information (for example, three-dimensional coordinates) of the ultrasonic transmitter and the ultrasonic receiver at different parts of the user's body. It should be noted that the position information of the ultrasonic transmitter and the ultrasonic receiver may be fixed, or may be adaptively adjusted according to factors such as the user's age, gender, height, weight, and body shape.

[0058] The information about the ultrasonic transmitter transmitting the ultrasonic wave may refer to information related to the process of transmitting the ultrasonic wave by the ultrasonic transmitter. For example, the time when the ultrasonic transmitter transmits the ultrasonic wave, the frequency at which the ultrasonic transmitter transmits the ultrasonic wave, and the code used by the ultrasonic transmitter to transmit the ultrasonic wave. The information about the ultrasonic receiver receiving the ultrasonic wave may refer to information related to the process of receiving the ultrasonic wave by the ultrasonic receiver. For example, the time when the ultrasonic receiver receives the ultrasonic wave, the frequency at which the ultrasonic receiver receives the ultrasonic wave, and the code used by the ultrasonic receiver to receive the ultrasonic wave.

[0059] In some embodiments, the user's posture can be recognized by designing the layout position of the ultrasonic transmitter and the ultrasonic receiver in the ultrasonic sensor. For example, the ultrasonic transmitter and the ultrasonic receiver are respectively set on the user's right forearm and right upper arm, and the processor 120 can identify the bending angle between the two limbs corresponding to the elbow joint or knee joint based on the position information of the ultrasonic transmitter and the ultrasonic receiver, as well as the information of the ultrasonic transmitter transmitting ultrasonic waves and the information of the ultrasonic receiver receiving ultrasonic waves. For another example, the ultrasonic receiver is set at the upper arm of the human body, and the ultrasonic transmitter is set at the torso of the human body. The processor 120 can identify the posture of the upper arm relative to the torso (for example, the upper arm swings back and forth, up and down, left and right relative to the torso) based on the position information of the ultrasonic transmitter and the ultrasonic receiver, as well as the information of the ultrasonic transmitter transmitting ultrasonic waves and the information of the ultrasonic receiver receiving ultrasonic waves. For more information about identifying the bending angle between the two limbs corresponding to the elbow joint or knee joint and identifying the posture of the upper arm relative to the torso, please refer to Figure 2-Figure 5 、 Figure 6A 、 Figure 6B 、 Figure 6C 、 Figures 9-14 and related instructions.

[0060] In some embodiments, the human gesture recognition system 100 may further include a garment 130, which can be worn by a user and into which at least one set of ultrasonic sensors 110 is integrated. In some embodiments, the garment 130 may include any one or more of a top, pants, or bodysuit. In some embodiments, the ultrasonic sensors 110 may be integrated into the garment in a variety of ways. For example, the ultrasonic sensors may be sewn to the garment using sewing thread such as cotton or nylon thread. In another example, the ultrasonic sensors 110 may be bonded to the garment using adhesives such as acrylic adhesive, composite structural adhesive, or polymer adhesive. In another example, the ultrasonic sensors 110 may be attached to the garment using a detachable structure such as a buckle or Velcro. In some embodiments, the garment 130 may be a single-layer structure, and the ultrasonic sensors 110 may be located on the upper or lower surface of the single-layer structure. In some embodiments, the garment 130 may be a multi-layer structure, and the ultrasonic sensors 110 may be located on the upper or lower surface of the garment 130, or between two adjacent layers.

[0061] In some embodiments, the processor 120 can be integrated into the garment 130. For example, the processor 120 can be mounted on the garment 130 by sewing, gluing, or the like. In some embodiments, the processor 120 can be mounted separately from the garment 130. For example, the processor 120 can communicate with the ultrasonic sensor 110 mounted on the garment 130 or the user's body via a wired network and / or a wireless network. Exemplary wired networks can include cable networks, fiber optic networks, and the like. Exemplary wireless networks can include wireless local area networks (WLANs), Bluetooth networks, and global systems for mobile communications (GSM) networks.

[0062] In some embodiments, the ultrasonic sensor in the human gesture recognition system can also be directly mounted on the user's body. For example, the ultrasonic transmitter and ultrasonic receiver corresponding to the ultrasonic sensor can be fixed to the user's body or clothing via a fastener (e.g., a strap, pin, buckle), glue, or adhesive.

[0063] In order to improve the accuracy of the human body posture recognition system in recognizing user postures, in some embodiments, the ultrasonic sensors 110 can be set at different parts of the user's body. Furthermore, the ultrasonic transmitters 131 and ultrasonic receivers 132 corresponding to each group of ultrasonic sensors 110 are distributed at different parts of the user's body. The following describes the postures of different parts in combination with the distribution of the ultrasonic transmitters 131 and the ultrasonic receivers 132.

[0064] In some embodiments, the ultrasonic transmitter and the ultrasonic receiver can be respectively located at the two sides of the user's limbs corresponding to the elbow or knee joints to identify the movement posture of the user's arms or legs. For example, the bending angle or rotation angle between the two sides of the limbs corresponding to the elbow or knee joints of the user can be identified. The two sides of the limbs corresponding to the elbow joints can refer to the upper arm and the forearm, and the two sides of the limbs corresponding to the knee joints can refer to the thigh and the calf. For example, one of the ultrasonic transmitter and the ultrasonic receiver is located at the upper arm, and the other is located at the forearm. For another example, one of the ultrasonic transmitter and the ultrasonic receiver is located at the thigh, and the other is located at the calf. In some embodiments, when the ultrasonic sensor is integrated into clothing, the ultrasonic transmitter and the ultrasonic receiver are arranged on the sleeves or trouser legs of the clothing, so that when the user wears the clothing, the ultrasonic transmitter and the ultrasonic receiver can be respectively located at the two sides of the limbs corresponding to the elbow or knee joints of the user.

[0065] Figure 2 This is an exemplary schematic diagram of an ultrasonic transmitter and an ultrasonic receiver distributed on both sides of the user's limbs corresponding to the elbow joint or knee joint according to some embodiments of this specification. Figure 2As shown, the ultrasonic transmitter 210 and the ultrasonic receiver 220 are respectively located on the two sides of the user's limbs corresponding to the elbow joint, wherein the ultrasonic transmitter 210 is located on the user's left upper arm, and the ultrasonic receiver 220 is located on the user's left forearm; the ultrasonic transmitter 230 and the ultrasonic receiver 240 are respectively located on the two sides of the user's limbs corresponding to the elbow joint, wherein the ultrasonic transmitter 230 is located on the user's right upper arm, and the ultrasonic receiver 240 is located on the user's right forearm; the ultrasonic transmitter 250 and the ultrasonic receiver 260 are respectively located on the two sides of the user's limbs corresponding to the knee joint, wherein the ultrasonic transmitter 250 is located on the user's left thigh, and the ultrasonic receiver 260 is located on the user's left calf; the ultrasonic transmitter 270 and the ultrasonic receiver 280 are respectively located on the two sides of the user's limbs corresponding to the knee joint, wherein the ultrasonic transmitter 270 is located on the user's right thigh, and the ultrasonic receiver 280 is located on the user's right calf.

[0066] Figure 2 The arrangement positions of the ultrasonic sensors shown are only examples. It is understood that Figure 2 The positions of the ultrasonic transmitter and the ultrasonic receiver shown can be interchanged. For example, the ultrasonic transmitter 210 is located on the user's left forearm, and the ultrasonic receiver 220 is located on the user's left upper arm. For another example, the ultrasonic transmitter 270 is located on the user's right calf, and the ultrasonic receiver 280 is located on the user's right thigh. For more information on the placement of ultrasonic sensors, please refer to Figure 7 and Figure 8 and its description.

[0067] In order to further improve the recognition accuracy of the human posture recognition system, reduce the measurement blind spot, and prevent the influence of human body occlusion, each group of ultrasonic sensors can include multiple ultrasonic transmitters or ultrasonic receivers. For example, one ultrasonic transmitter is set on the user's left forearm, and two ultrasonic receivers are set on the user's left upper arm. For another example, three ultrasonic transmitters are set on the user's left forearm, and three ultrasonic receivers are set on the user's left upper arm. More information about setting multiple ultrasonic transmitters and ultrasonic receivers can be found elsewhere in this manual, for example, Figure 5-6C 、 Figure 11-14 and related instructions.

[0068] Figure 3 FIG3 is an exemplary flow chart of a method for recognizing a user's gesture according to some embodiments of this specification. In some embodiments, process 300 may be executed by processor 120. Figure 3 As shown, process 300 may include the following steps:

[0069] Step 310: Acquire the distance between the ultrasonic transmitter and the ultrasonic receiver based on the information of the ultrasonic transmitter emitting the ultrasonic wave and the information of the ultrasonic receiver receiving the ultrasonic wave.

[0070] In some embodiments, the processor can obtain the distance between the ultrasonic transmitter and the ultrasonic receiver based on the time when the ultrasonic transmitter transmits the ultrasonic wave and the time when the ultrasonic receiver receives the ultrasonic wave. For example, the ultrasonic transmitter and the ultrasonic receiver are distributed on the upper arm and the lower arm on both sides of the elbow joint. Figure 4 As shown, the ultrasonic transmitter 410 is located at the user's upper arm, and the ultrasonic receiver 420 is located at the user's lower arm. When the human body gesture recognition system is working, the ultrasonic transmitter 410 transmits ultrasonic waves that can be received by the ultrasonic receiver 420. Specifically, the time when the ultrasonic transmitter 410 transmits the ultrasonic wave is t1, and the time when the ultrasonic receiver 420 receives the ultrasonic wave is t2. The processor can multiply the difference Δt between t1 and t2 and the propagation speed of ultrasonic waves in air is C (approximately 340 m / s under conditions of 1 standard atmospheric pressure and 15°C) by C to determine the multiplication result as the distance X between the ultrasonic transmitter 410 and the ultrasonic receiver 420.

[0071] Step 320: Based on the distance between the ultrasonic transmitter and the ultrasonic receiver and the position information of the ultrasonic transmitter and the ultrasonic receiver, the bending angle between the two limbs corresponding to the elbow joint or the knee joint is obtained.

[0072] The bending angle between the two limbs corresponding to the elbow joint or knee joint may refer to the bending angle between the left (right) forearm and the left (right) upper arm or the bending angle between the left (right) calf and the left (right) thigh. For example, the ultrasonic transmitter and the ultrasonic receiver are distributed on the upper arm and the forearm on both sides of the elbow joint. Figure 4 As shown, you can Figure 4 The point O shown in the figure is considered to be the location of the elbow joint, and the angle α can be considered to be the bending angle (i.e., the angle between the upper arm and the forearm). It should be noted that the bending angle here can be considered to be the angle between the extension direction of the upper arm and the extension direction of the forearm, and the point O can be considered to be the intersection of the extension direction of the upper arm and the extension direction of the forearm.

[0073] In some embodiments, the processor can obtain the bending angle α between the two limbs corresponding to the elbow joint or knee joint based on the distance between the ultrasonic transmitter and the ultrasonic receiver and the position information of the ultrasonic transmitter and the ultrasonic receiver through formula (1):

[0074]

[0075] Among them, c is a length value that can characterize the dimension of the user's forearm where the ultrasonic transmitter is located, and d is a length value that can characterize the dimension of the user's upper arm where the ultrasonic receiver is located. The value of c or d is positively correlated with the dimension of the user's arm. For example, the thicker the user's upper arm or forearm, the larger c or d. Just as an example, the circumference of the cross-sectional shape of the forearm or upper arm taken along the extension direction of the vertical forearm or upper arm and through the geometric center of the ultrasonic transmitter is used as the dimension of the user's forearm or upper arm. The cross-sectional shape can be approximately regarded as an ellipse, and the major axis radius or minor axis radius of the ellipse is used as c or d. a can characterize the distance between the center point of the cross-sectional shape and the joint (for example, the elbow joint, in Figure 4 In the figure, a is the distance between the elbow joint and the center of the cross-sectional shape (represented by point O), and b can represent the distance between the center of the cross-sectional shape and the joint (for example, the elbow joint). a, b, c, and d are all known parameters. c and d can be determined based on the dimensions of the user's arms / legs. For example, based on the dimensions of the user's arms / legs, c and d can be determined by measurement. a and b can be determined based on the position information of the ultrasonic transmitter and the ultrasonic receiver. For example, a three-dimensional space coordinate system is established with a certain position of the human body as an element, and the position information (coordinate information) of the ultrasonic transmitter and the ultrasonic receiver, the coordinate information of the elbow joint (for example, point O), the dimensions of the user's arm, etc. are obtained to determine a and b. The ultrasonic transmitter and ultrasonic receiver corresponding to the ultrasonic sensor are set on the limbs corresponding to both sides of the knee joint or elbow joint, and the bending angle between the limbs on both sides corresponding to the elbow joint or knee joint can be determined through the information of the ultrasonic sensor (for example, the ultrasonic transmitter and the ultrasonic sensor) to identify the local movement posture of the user's limb. The human posture recognition system is stable, the algorithm is relatively simple, and it can more accurately identify the movement posture of the user's limb.

[0076] The motion state of the limbs on both sides corresponding to the elbow or knee joint is not a bending motion centered on the elbow or knee joint. The motion state of the limbs on both sides of the joint can also be a rotation motion centered on the joint. For example, the forearm can rotate relative to the upper arm about the elbow joint. In order to more accurately identify the posture of the user's limbs, in some embodiments, by respectively providing an ultrasonic transmitter and an ultrasonic receiver on the limbs on both sides corresponding to the elbow or knee joint, the rotation angle between the limbs on both sides corresponding to the elbow or knee joint of the user can also be identified. Figure 5 FIG5 is an exemplary flow chart of a method for recognizing a user's gesture according to other embodiments of this specification. In some embodiments, process 500 may be executed by processor 120. Figure 5 As shown, process 500 may include the following steps:

[0077] Step 510: Determine a first distance between the ultrasonic transmitter and the first ultrasonic receiver and a second distance between the ultrasonic transmitter and the second ultrasonic receiver based on information of ultrasonic transmission by the ultrasonic transmitter, information of ultrasonic reception by the first ultrasonic receiver, and information of ultrasonic reception by the second ultrasonic receiver.

[0078] To further improve the recognition accuracy of the human gesture recognition system and reduce measurement blind spots, each set of ultrasonic sensors can include multiple ultrasonic transmitters or receivers. For example, each set of ultrasonic sensors can include at least two ultrasonic transmitters and one ultrasonic receiver. Another example is that each set of ultrasonic sensors can include at least two ultrasonic transmitters and at least two ultrasonic receivers. Another example is that each set of ultrasonic sensors can include one ultrasonic transmitter and at least two ultrasonic receivers. When determining the rotation angle between two limbs corresponding to an elbow or knee joint, each set of ultrasonic sensors can include multiple ultrasonic transmitters or receivers. For illustrative purposes, the following example uses the case where each set of ultrasonic sensors can include one ultrasonic transmitter and at least two ultrasonic receivers.

[0079] In some embodiments, each set of ultrasonic sensors includes an ultrasonic transmitter and at least two ultrasonic receivers, wherein the at least two ultrasonic receivers include a first ultrasonic receiver and a second ultrasonic receiver. Figure 6A As shown in , each group of ultrasonic sensors includes an ultrasonic transmitter 610 , a first ultrasonic receiver 620 and a second ultrasonic receiver 630 .

[0080] The first distance may refer to the distance between the ultrasonic transmitter and the first ultrasonic receiver. Figure 6A As shown in FIG, the distance X1 between the ultrasonic transmitter 610 and the first ultrasonic receiver 620. The second distance may refer to the distance between the ultrasonic transmitter and the second ultrasonic receiver. Figure 6A As shown in , the distance X2 between the ultrasonic transmitter 610 and the second ultrasonic receiver 630 is.

[0081] In some embodiments, the processor may determine a first distance between the ultrasonic transmitter and the first ultrasonic receiver and a second distance between the ultrasonic transmitter and the second ultrasonic receiver based on the time when the ultrasonic transmitter transmits the ultrasonic wave, the time when the first ultrasonic receiver receives the ultrasonic wave, and the time when the second ultrasonic receiver receives the ultrasonic wave. For example, the ultrasonic transmitter and the ultrasonic receiver are distributed on the upper arm and the forearm on both sides of the elbow joint. Figure 6AAs shown, the ultrasonic transmitter 610 is located at the user's upper arm, and the first ultrasonic receiver 620 and the second ultrasonic receiver 630 are located at the user's forearm. When the human body gesture recognition system is working, the ultrasonic transmitter 610 transmits ultrasonic waves that can be received by the first ultrasonic receiver 620 and the second ultrasonic receiver 630. Specifically, the time when the ultrasonic transmitter 610 transmits the ultrasonic wave is t1, the time when the first ultrasonic receiver 620 receives the ultrasonic wave is t2, and the time when the second ultrasonic receiver 630 receives the ultrasonic wave is t3. The processor can multiply the above Δt12 and Δt13 by C based on the difference Δt12 between t1 and t2, the difference Δt13 between t1 and t3, and the propagation speed of ultrasonic waves in air is C (approximately 340 m / s under the conditions of 1 standard atmospheric pressure and 15°C). The multiplication results can be determined as the distance X1 between the ultrasonic transmitter 610 and the first ultrasonic receiver 620, and the distance X2 between the ultrasonic transmitter 610 and the second ultrasonic receiver 630.

[0082] Taking the right arm of the human body as an example, when the arm is normally extended, the forearm does not rotate relative to the upper arm. Due to the characteristics of the human elbow joint, the forearm is usually in the counterclockwise direction of the arm extension direction (the direction from the forearm to the hand) ( Figure 6A Based on this, in some embodiments, the first ultrasonic receiver 620 can be located on the same side of the user's forearm and palm, and the second ultrasonic receiver 630 can be located on the same side of the user's forearm and back of the hand, so as to more accurately identify the rotation angle of the user's forearm relative to the upper arm.

[0083] Step 520: Determine the rotation angle between the two limbs corresponding to the elbow joint or the knee joint according to the first distance and the second distance.

[0084] The rotation angle between the two sides of the limbs corresponding to the elbow joint can refer to the angle of rotation of the forearm relative to the upper arm. The rotation angle between the two sides of the limbs corresponding to the knee joint can refer to the angle of rotation of the calf relative to the thigh.

[0085] In order to explain more clearly, the following will be combined with Figure 6B and Figure 6C Here, Figure 6B is a side view of the distribution of exemplary ultrasound transmitters and ultrasound receivers on both sides of the limbs corresponding to the elbow joint or knee joint according to some embodiments of this specification, Figure 6C This is a top view of the distribution of exemplary ultrasound transmitters and ultrasound receivers on both sides of the limbs corresponding to the elbow joints or knee joints shown in some embodiments of this specification.

[0086] like Figure 6B and Figure 6CAs shown, in some embodiments, the first ultrasonic receiver 620 and the second ultrasonic receiver 630 can be arranged relative to a straight line (eg, Figure 6B and Figure 6C The straight line L) shown in FIG is approximately symmetrical. When the arm is normally straightened, the forearm does not rotate relative to the upper arm. Figure 6B or Figure 6C The first distance X1 and the second distance X2 shown can be considered to be approximately equal. In this case, the line connecting the first ultrasonic receiver 620 and the second ultrasonic receiver 630 is a first straight line L1.

[0087] When the forearm rotates relative to the upper arm, the first ultrasonic receiver 620 and the second ultrasonic receiver 630 will also rotate relative to the ultrasonic transmitter 610. At this time, the line connecting the first ultrasonic receiver 620 and the second ultrasonic receiver 630 is the second straight line L2. When the user's forearm and upper arm change from a straight state to a state where the forearm is rotated relative to each other, the angle β formed by the first straight line L1 and the second straight line L2 can be regarded as the rotation angle of the forearm relative to the upper arm. When the forearm rotates relative to the upper arm, the first distance X1 and the second distance X2 will change. For example, Figure 6B or Figure 6C As shown, when the lower arm rotates relative to the upper arm toward the side of the arm where the first ultrasonic receiver 620 is located (ie, Figure 6B or Figure 6C When the forearm rotates relative to the upper arm toward the arm where the second ultrasonic receiver 630 is located, the first distance X1 decreases and the second distance X2 increases, resulting in the first distance X1 being smaller than the second distance X2.

[0088] In some embodiments, the processor may determine the rotation angle β between the two limbs corresponding to the elbow joint or knee joint based on the first distance and the second distance, which can be specifically obtained by formula (2):

[0089] β∝±(X1-X2) (2)

[0090] Here, ± represents the rotation direction, + represents that the forearm rotates relative to the upper arm toward the side of the arm where the first ultrasonic receiver 620 is located, and - represents that the forearm rotates relative to the upper arm toward the side of the arm where the second ultrasonic receiver 630 is located.

[0091] It should be noted that the rotation angle β determined by formula (2) is a relative quantity, which is only used to characterize the degree of rotation of the forearm or calf, and is not the precise rotation angle between the two limbs corresponding to the elbow joint or knee joint.

[0092] In some embodiments, when the number of ultrasonic receivers in a set of ultrasonic sensors is two, the processor may determine the bending angle between the two limbs corresponding to the elbow joint or knee joint based on the first distance and the second distance. Figure 6B and Figure 6C As shown, the processor can add the first distance X1 and the second distance X2 and divide them by 2 to obtain the distance X between the midpoint M of the line connecting the first ultrasonic receiver 620 and the second ultrasonic receiver 630 and the ultrasonic transmitter 610, and then determine the bending angle α of the elbow joint based on formula (1). For more information about formula (1), please refer to Figure 3 It should be noted that the calculated value of X here is approximately equal because the distance between the first ultrasonic receiver 620 and the second ultrasonic receiver 630 is much smaller than the distance between the ultrasonic transmitter 610 and the first ultrasonic receiver 620 or the second ultrasonic receiver 630. Therefore, in this case, the calculated result X can be approximately equal to the distance between the ultrasonic transmitter and the ultrasonic receiver.

[0093] Ultrasonic sensors have a limited operating range. For example, an ultrasonic transmitter can only emit ultrasound waves in a certain direction, and an ultrasonic receiver can only receive ultrasound waves in that direction. The operating range of an ultrasonic sensor can be determined to some extent by the sensor's manufacturing process. Therefore, before designing the placement of an ultrasonic sensor, it is important to understand its operating range. The following describes the structure of an ultrasonic transmitter and receiver.

[0094] In some embodiments, the ultrasonic transmitter may include a vibration unit and a shell. The vibration unit may convert an electrical signal into an ultrasonic wave. The shell may provide fixed support for the ultrasonic transmitter, isolate the ultrasonic transmitter from the external environment, and other functions. In addition, the shell may also limit the direction of ultrasonic wave emission through various settings. For example, the shell may be provided with a hole to allow ultrasonic waves to be emitted from the hole. For another example, the shell may be provided with a material (e.g., the type or thickness of the material is set so that the material on this side is different from the material on the other sides) or a structure (e.g., the structure is deformed or shaped so that the structure on this side is different from the structure on the other sides) on one side to allow ultrasonic waves to be emitted from this side. In some embodiments, the side that emits ultrasonic waves may be referred to as the output end. For example, the side of the shell with a hole or the side with a material different from the other sides may be referred to as the output end.

[0095] In some embodiments, an ultrasonic receiver may include a transducer unit and a housing. The transducer unit can convert ultrasonic waves into electrical signals. The housing can provide fixed support for the ultrasonic receiver, isolate it from the external environment, and other functions. Furthermore, similar to the housing of the ultrasonic transmitter described above, the housing of the ultrasonic receiver can be configured to restrict the direction of ultrasonic wave reception. For example, one side may have a hole, or the material or structure of one side may be modified. In some embodiments, the side that receives ultrasonic waves can be referred to as a receiving end. For example, the side of the housing with the hole or the side made of a different material from the other sides can be referred to as the receiving end.

[0096] To ensure that the ultrasonic waves emitted by the ultrasonic transmitter can always be received by the ultrasonic receiver, in some embodiments, the ultrasonic transmitter and receiver are located on the inner sides of the limbs corresponding to the elbow or knee joints. When the elbow or knee joint is flexed, the sides of the limbs that face each other are referred to as the inner sides of the limbs corresponding to the elbow or knee joints. Since the forearm of an arm can rotate relative to the upper arm, using the arm as an example, the inner sides of the limbs corresponding to the elbow joints can also be the areas on the same side of the arm as the palm when the arm is naturally extended. Figure 7 This is a schematic diagram of the distribution of ultrasonic transmitters and ultrasonic receivers on both sides of the limbs corresponding to the elbow joint or knee joint according to some embodiments of this specification. Figure 7 As shown, the ultrasonic transmitter 710 and the ultrasonic receiver 720 are located on the inner sides of the limbs corresponding to the elbow joint or the knee joint.

[0097] If the distance between the ultrasonic transmitter and the ultrasonic receiver is too small, the output end of the ultrasonic transmitter will be too close to the receiving end of the ultrasonic receiver, and the measured difference in the distance change between the ultrasonic transmitter and the ultrasonic receiver will be too small, making it impossible to accurately calculate the bending angle of the joint. To improve the accuracy of the human posture recognition system, in some embodiments, when the user straightens their arm or leg, the distance between the ultrasonic transmitter and the ultrasonic receiver is not less than 10 cm. Preferably, when the user straightens their arm or leg, the distance between the ultrasonic transmitter and the ultrasonic receiver is not less than 15 cm. Further preferably, when the user straightens their arm or leg, the distance between the ultrasonic transmitter and the ultrasonic receiver is not less than 20 cm. To allow the ultrasonic transmitter and the ultrasonic receiver to be distributed on the user's limbs without affecting the normal operation of other parts of the user, in some embodiments, the distance between the ultrasonic transmitter and the ultrasonic receiver is not greater than the length of the user's limb (e.g., leg or arm). It should be noted that different users have different limb lengths. In actual application scenarios, adaptive adjustments can be made based on the user's actual situation (e.g., limb length).

[0098] When the ultrasonic sensor is integrated on the clothing, in some embodiments, the distance between the ultrasonic transmitter and the ultrasonic receiver is no greater than the length of the sleeve or trouser leg in the clothing. In some embodiments, when the ultrasonic sensor is integrated on the clothing and the clothing is in a flat-laid out state, the distance between the ultrasonic transmitter and the ultrasonic receiver may be no less than 10 cm. Preferably, when the ultrasonic sensor is integrated on the clothing and the clothing is in a flat-laid out state, the distance between the ultrasonic transmitter and the ultrasonic receiver may be no less than 15 cm. Further preferably, when the ultrasonic sensor is integrated on the clothing and the clothing is in a flat-laid out state, the distance between the ultrasonic transmitter and the ultrasonic receiver may be no less than 20 cm. It is understandable that the length of each person's limbs may be different, so clothing can be designed for different users so that the arrangement position of the ultrasonic sensor achieves the desired effect. Children's limbs are generally shorter. For example, the arm length of children aged 4 to 12 is generally greater than 30 cm, and the leg length is generally greater than 60 cm. The distance between the ultrasonic transmitter and the ultrasonic receiver at the sleeve of children's clothing can be 10 cm to 30 cm, and the distance between the ultrasonic transmitter and the ultrasonic receiver at the sleeve of children's clothing can be 20 cm to 50 cm. For example, the length of an adult's limbs is generally relative to that of a child's limbs. Therefore, the distance between the ultrasonic transmitter and the ultrasonic receiver at the arm or leg of an adult's clothing can be 20 cm to 50 cm.

[0099] In some embodiments, the ultrasonic transmitter includes an output end for emitting ultrasonic waves, and the output direction of the output end is toward the outside of the clothing, or away from the part where the ultrasonic transmitter directly or indirectly contacts the user's limbs. Figure 7 As shown, the ultrasonic transmitter 710 and the ultrasonic receiver 720 are located on the surface of clothing or the user's skin as an example for explanation. The ultrasonic transmitter 710 includes an output end 711, and the output direction of the output end 711 is toward the outside of the clothing, or away from the part where the ultrasonic transmitter 710 directly or indirectly contacts the user's limbs. In some embodiments, the ultrasonic receiver includes a receiving end for receiving ultrasonic waves, and the receiving end is toward the outside of the clothing, or away from the part where the ultrasonic receiver directly or indirectly contacts the user's limbs. Figure 7 As shown, the ultrasonic receiver 720 includes a receiving end 721, which faces the outside of the garment, or faces away from the part where the ultrasonic receiver 720 directly or indirectly contacts the user's limbs. In some embodiments, when the garment is a multi-layer structure, the ultrasonic transmitter and the ultrasonic receiver may be located between two layers of the garment, wherein the output directions of the ultrasonic transmitter and the ultrasonic receiver face the outside of the garment. That is, when the user wears the garment, the output end of the ultrasonic transmitter and the receiving end of the ultrasonic receiver face away from the part where the user directly or indirectly contacts the user's limbs.

[0100] As mentioned above, the ultrasonic wave emitted by the ultrasonic transmitter is radiated to the outside world through its output end, and the ultrasonic receiver needs to receive the ultrasonic wave through its receiving end. Since the ultrasonic transmitter and the ultrasonic receiver are respectively located at the upper arm and forearm of the user, if the user straightens his arms or legs, the ultrasonic receiver may not be able to receive the ultrasonic wave emitted by the ultrasonic transmitter. In order to ensure that the ultrasonic wave emitted by the ultrasonic transmitter can be received by the ultrasonic receiver and improve the reliability of the human body posture recognition system, in some embodiments, the angle between the plane where the output end is located and the plane where the receiving end is located may be less than 180°. Preferably, the angle between the plane where the output end is located and the plane where the receiving end is located is not greater than 170°. More preferably, the angle between the plane where the output end is located and the plane where the receiving end is located is not greater than 150°. Further preferably, the angle between the plane where the output end is located and the plane where the receiving end is located is not greater than 130°. Figure 7 As shown, you can Figure 7 The angle θ shown in the figure is the angle between the plane containing the output end of the ultrasonic transmitter 710 and the plane containing the receiving end of the ultrasonic receiver 720. It should be noted that the plane containing the output end is the plane containing the side wall of the ultrasonic transmitter housing that emits ultrasonic waves. For example, an ultrasonic transmitter may include a housing and a vibrating unit. The vibrating unit generates ultrasonic waves based on electrical signals. The ultrasonic waves can be radiated externally through a hole provided in the housing. In this case, the side wall containing the hole can be considered the output end of the ultrasonic transmitter. For another example, the housing may not have a hole, and ultrasonic waves can be radiated directly from a side wall of the housing. This side wall can also be considered the output end of the ultrasonic transmitter. Correspondingly, the plane containing the receiving end is the plane containing the side wall of the ultrasonic receiver housing that is used to receive ultrasonic waves. For example, an ultrasonic receiver may include a housing and a transducer unit. The transducer unit generates electrical signals based on sound signals. The ultrasonic waves emitted by the ultrasonic transmitter can be transmitted through a hole provided in the housing to the interior of the ultrasonic receiver housing and act on the transducer unit. In this case, the side wall containing the hole can be considered the receiving end of the ultrasonic receiver. For another example, the shell may not be provided with a hole, and the ultrasonic wave may be directly received from a side wall of the shell, and the side wall may also be regarded as a receiving end of the ultrasonic receiver.

[0101] In order to make the angle formed by the plane where the output end is located and the plane where the receiving end is located within a specific angle range (for example, less than 170°), in some embodiments, the ultrasonic transmitter is tilted relative to the user's limb contact part below it and toward the ultrasonic receiver, and the angle between the normal direction of the output end and the normal direction of the user's limb contact part below the ultrasonic transmitter is not less than 15°. Preferably, the angle between the normal direction of the output end and the normal direction of the user's limb contact part below the ultrasonic transmitter is not less than 10°. Further preferably, the angle between the normal direction of the output end and the normal direction of the user's limb contact part below the ultrasonic transmitter is not less than 5°. The user's limb contact part can be the user's skin or clothing. For example, when the user wears clothing with an integrated ultrasonic sensor, the user's limb contact part can be clothing. For another example, when the ultrasonic sensor is directly set together with the user's skin, the user's limb contact part can be the user's skin. As Figure 7 As shown, the ultrasonic transmitter 710 is tilted relative to the user's limb contact portion below it and toward the ultrasonic receiver 720. The normal line of the output end is p, and the normal line of the user's limb contact portion below the ultrasonic transmitter is q. Figure 7 The angle β1 shown can be considered as the angle between the normal line p and the normal line q.

[0102] In some embodiments, the ultrasonic receiver is tilted relative to the part of the user's limb that contacts it and toward the ultrasonic transmitter, and the angle between the normal direction of the receiving end and the normal direction of the part of the user's limb that contacts it is not less than 15°. Preferably, the angle between the normal direction of the receiving end and the normal direction of the part of the user's limb that contacts it is not less than 10°. Further preferably, the angle between the normal direction of the receiving end and the normal direction of the part of the user's limb that contacts it is not less than 5°. Figure 7 As shown, the ultrasonic receiver 720 is tilted relative to the user's limb contact portion below it and toward the ultrasonic transmitter 710. The normal line of the receiving end is s, and the normal line of the user's limb contact portion below the ultrasonic receiver is r. Figure 7 The angle β2 shown can be considered as the angle between the normal line s and the normal line r.

[0103] By setting β1 and β2 in the above manner, the angle between the plane where the output end is located and the plane where the receiving end is located is no more than 170°, which can ensure that the ultrasonic receiver can always receive the ultrasonic wave emitted by the ultrasonic transmitter, thereby improving the reliability of the system.

[0104] Figure 7 The exemplary ultrasonic transmitter 710 and ultrasonic receiver 720 shown are rectangular parallelepipeds, but the ultrasonic transmitter and ultrasonic receiver may also have other structures and / or shapes.

[0105] like Figure 8As shown, the ultrasonic transmitter 810 and the ultrasonic receiver 820 may include a right-angled trapezoid. The right-angled trapezoid may be understood as a rectangular parallelepiped structure with one edge cut off along its length, width, or thickness direction. The surface formed by cutting off the edge is the trapezoidal surface of the right-angled trapezoid. In some embodiments, the trapezoidal surface ( Figure 8 The ultrasonic transmitter 810 is tilted relative to the part of the ultrasonic transmitter that contacts the user's limb and toward the ultrasonic receiver 820. The normal of the output end 811 is p, and the normal of the part of the ultrasonic transmitter that contacts the user's limb is q. Figure 8 The angle β1 shown can be considered as the angle between the normal line p and the normal line q. The ultrasonic receiver 820 is tilted relative to the part of the user's body that contacts it and toward the ultrasonic transmitter 810. The normal line of the receiving end 821 is s, and the normal line of the part of the user's body that contacts it is r. Figure 8 The angle β2 shown can be considered as the angle between the normal line s and the normal line r.

[0106] It is understood that the ultrasonic transmitter and the ultrasonic receiver may also be other than Figure 7 and Figure 8 As shown, the angle between the normal direction of the output end and the normal direction of the part of the user's body that contacts the ultrasonic transmitter, and the angle between the normal direction of the receiving end and the normal direction of the part of the user's body that contacts the ultrasonic receiver can be satisfied. Other structures and / or shapes, such as triangular prisms, hexagonal prisms, and cylinders, can be used.

[0107] Figure 7 and Figure 8 The ultrasonic transmitter and ultrasonic receiver shown can be fixed to the human body or clothing by means of glue or adhesives. In some embodiments, the ultrasonic transmitter and ultrasonic receiver can also be fixed by setting a fixing part, so that the angle between the normal direction of the output end and the normal direction of the part of the user's limb in contact with the ultrasonic transmitter is not less than a specific angle (for example, 5°), and the angle between the normal direction of the receiving end and the normal direction of the part of the user's limb in contact with the ultrasonic receiver is not less than a specific angle (for example, 5°). In some embodiments, the fixing part can be integrally formed with the ultrasonic sensor (for example, the ultrasonic transmitter or ultrasonic receiver), or a structure independent of the ultrasonic sensor. Figure 7 As an example, the fixing member may be a wedge-shaped structure, which is inserted between the ultrasound transmitter and the clothing in contact with the user's skin and between the ultrasound receiver and the clothing in contact with the user's skin.

[0108] The ultrasonic sensor can not only obtain the bending angle between the two limbs corresponding to the elbow joint or the knee joint and the rotation angle between the two limbs corresponding to the elbow joint or the knee joint, but also be used to identify the posture of the upper arm relative to the torso.

[0109] In some embodiments, the ultrasonic transmitter may be located on the upper arm of the garment corresponding to the human body, and the ultrasonic receiver may be located on the torso of the garment corresponding to the human body. The ultrasonic transmitter and the ultrasonic receiver cooperate to identify the posture of the upper arm relative to the torso. In some embodiments, the ultrasonic receiver may be located on the upper arm of the garment corresponding to the human body, and the ultrasonic transmitter may be located on the torso of the garment corresponding to the human body. Figure 9 FIG is a schematic diagram of the distribution of ultrasonic transmitters and ultrasonic receivers on the upper arms and torso according to some embodiments of this specification. Figure 9 As shown, the ultrasonic transmitter 910 and the ultrasonic transmitter 930 are located at the upper arms of the clothing corresponding to the human body, and the ultrasonic receiver 920 and the ultrasonic receiver 940 are located at the torso of the clothing corresponding to the human body.

[0110] In some embodiments, the ultrasonic receivers can be located at multiple locations on the garment corresponding to the torso of the human body. Figure 10 Position shown.

[0111] It is understandable that the body parts corresponding to the shoulder joint (e.g., the upper arm and torso) have more complex movements than the limbs corresponding to the elbow or knee joints. For example, the upper arm can move up and down, forward and backward, rotate, and swing relative to the torso. The range of action of the ultrasonic sensor is limited. Therefore, for an ultrasonic transmitter, if only one ultrasonic receiver is set, there is often a measurement blind spot. Under certain conditions, it is impossible to receive the ultrasonic wave emitted by the ultrasonic transmitter for accurate positioning. In order to improve the accuracy of the ultrasonic sensor in identifying the movement state of the body parts corresponding to the shoulder joint (e.g., the upper arm and torso), in some embodiments, there can be multiple groups of ultrasonic sensors set corresponding to the shoulder joint. In some implementations, a group of ultrasonic sensors can include at least one ultrasonic generator and multiple ultrasonic receivers.

[0112] Figure 10 is an exemplary schematic diagram of the arrangement position of the ultrasonic sensor according to some embodiments of this specification, wherein: Figure 10 FIG. 1 (a) is an exemplary schematic diagram of the arrangement position of the ultrasonic sensor on the front of the human body, Figure 10 Figure (b) in FIG is an exemplary schematic diagram of the arrangement position of the ultrasonic sensor on the back of the human body.

[0113] like Figure 10As shown in Figures (a) and (b) in the figure, the ultrasonic transmitter 1010 and the ultrasonic transmitter 1020 can be respectively located at the two upper arms of the human body, and the ultrasonic receiver (the square black area shown in Figure (a)) can be located at any one or multiple locations of the human body such as the front of the left shoulder, the front of the right shoulder, the left waist, the right waist, and the chest. Furthermore, the ultrasonic receivers located at the front of the left shoulder and the left waist can receive the ultrasonic waves from the ultrasonic transmitter on the left upper arm (i.e., Figure 10 The ultrasonic transmitter 1010 shown in FIG. 10 can receive the ultrasonic wave from the right upper arm (ie, the ultrasonic receiver located at the front of the right shoulder and the right waist). Figure 10 In the embodiment of the present invention, the ultrasonic transmitter 1010 and the ultrasonic transmitter 1020 are shown in the figure. The ultrasonic receiver located on the chest can receive the ultrasonic waves from the ultrasonic transmitter on the left upper arm and the ultrasonic waves from the ultrasonic transmitter on the right upper arm. When the ultrasonic sensor is integrated into the clothing, the ultrasonic transmitter 1010 and the ultrasonic transmitter 1020 can be respectively located at the two upper arms of the corresponding human body in the clothing, and the ultrasonic receiver can be located at least one of the front side of the left shoulder, the front side of the right shoulder, the left waist, the right waist, and the chest of the corresponding human body in the clothing. Such an arrangement can ensure that when the user's arm makes any movement centered on the shoulder joint, there is at least one ultrasonic receiver that can receive the ultrasonic waves from the ultrasonic transmitter, thereby performing human posture recognition. In some embodiments, in order to further improve the recognition accuracy of the human posture recognition system for human movements, one or more ultrasonic transmitters can be set at each upper arm of the user, and one or more ultrasonic receivers can be set at any part of the user's left shoulder front, right shoulder front, left waist, right waist, and chest at the same time. Preferably, at least two ultrasonic transmitters can be installed on each upper arm of the user, and at least three ultrasonic receivers can be installed on any of the user's left shoulder front, right shoulder front, left waist, right waist, or chest. The multiple ultrasonic receivers are not located on the same straight line. For example, the number of ultrasonic receivers can be three, distributed in a triangular pattern. For another example, the number of ultrasonic receivers can be four, distributed in a triangular or quadrilateral pattern. To ensure measurement accuracy, while taking into account the limited size of the human body and the angle of the ultrasonic sensor, it is necessary to set a limit on the distance between the ultrasonic sensors. In some embodiments, the spacing between the at least two ultrasonic transmitters is no less than 0.5 cm, and the spacing between the at least three ultrasonic receivers is no less than 0.5 cm. Preferably, the spacing between the at least two ultrasonic transmitters is no less than 0.8 cm, and the spacing between the at least three ultrasonic receivers is no less than 0.8 cm. More preferably, the spacing between the at least two ultrasonic transmitters is no less than 1 cm, and the spacing between the at least three ultrasonic receivers is no less than 1 cm.

[0114] In some embodiments, the posture of the upper arm relative to the torso may include the angle of the upper arm relative to the torso. The angle of the upper arm relative to the torso can be understood as the angle formed by the extension direction of the upper arm and the side of the torso. Figure 11 To clearly describe the angles that determine the position of the upper arm relative to the torso. Figure 11 FIG1 is an exemplary flow chart of a method for determining the angle of the upper arm relative to the torso according to some embodiments of this specification. In some embodiments, process 1100 may be executed by processor 120. Figure 11 As shown, process 1100 may include the following steps:

[0115] Step 1110 : Based on the information of ultrasonic waves transmitted by the ultrasonic transmitter and the information of ultrasonic waves received by the ultrasonic receiver, the distance between the ultrasonic transmitter and the ultrasonic receiver is acquired.

[0116] Based on the information of ultrasonic transmitter transmitting ultrasonic waves and ultrasonic receiver receiving ultrasonic waves, the distance between the ultrasonic transmitter and the ultrasonic receiver can be obtained by referring to Figure 3 Get the relevant content in .

[0117] Step 1120 : Determine the angle of the upper arm relative to the torso based on the distance between the ultrasonic transmitter and the ultrasonic receiver and the position information of the ultrasonic transmitter and the ultrasonic receiver.

[0118] The angle of the upper arm relative to the trunk is related to the placement of the ultrasonic sensor. When the ultrasonic transmitter is located on the upper arm and the ultrasonic receiver is located on the side of the trunk (such as the waist, ribs, etc.), the angle of the upper arm relative to the trunk can refer to the angle between the upper arm and the side of the trunk, such as Figure 13 As shown, the ultrasonic transmitter 1310 is located on the upper arm and the ultrasonic receiver 1320 is located at the waist. The angle of the upper arm relative to the torso can refer to the angle between the upper arm and the waist. When the ultrasonic transmitter is located on the upper arm and the ultrasonic receiver is located on the chest or back, the angle of the upper arm relative to the torso can refer to the angle of the arm when it is extended or swung forward.

[0119] Based on the distance between the ultrasonic transmitter and the ultrasonic receiver and the position information of the ultrasonic transmitter and the ultrasonic receiver, the angle of the upper arm relative to the torso can be determined. Figure 3 Determine the relevant content in.

[0120] Figure 12This is an exemplary flow chart of a method for identifying the motion state of an upper arm relative to a torso according to some embodiments of this specification. The motion state of an upper arm relative to a torso is relatively complex. In order to improve the recognition accuracy of the human posture recognition system for this motion state, in some embodiments, each group of ultrasonic sensors may include at least two ultrasonic transmitters and at least three ultrasonic receivers. In some embodiments, process 1200 may be executed by processor 120. Figure 12 As shown, process 1200 may include the following steps:

[0121] Step 1210: Determine position change information of at least two ultrasonic transmitters based on position information of at least three ultrasonic receivers.

[0122] In some embodiments, at least three ultrasound receivers are not in the same straight line. Figure 14 As shown, the lines connecting the positions of the ultrasonic receiver 1421 , the ultrasonic receiver 1422 , and the ultrasonic receiver 1423 may form a triangle, that is, the three ultrasonic receivers are not on the same straight line.

[0123] To ensure measurement accuracy, while also taking into account the limited size of the human body and the angle of the ultrasonic sensors, it is necessary to limit the distance between ultrasonic sensors. In some embodiments, the spacing between at least two ultrasonic transmitters is no less than 0.5 cm, and the spacing between at least three ultrasonic receivers is no less than 0.5 cm. Preferably, the spacing between at least two ultrasonic transmitters is no less than 0.8 cm, and the spacing between at least three ultrasonic receivers is no less than 0.8 cm. Further preferably, the spacing between at least two ultrasonic transmitters is no less than 1 cm, and the spacing between at least three ultrasonic receivers is no less than 1 cm.

[0124] In some embodiments, determining the position change information of at least two ultrasonic transmitters based on the position information of at least three ultrasonic receivers may include: determining the position information of the ultrasonic transmitter based on the position information of at least three ultrasonic receivers, specifically, through formula (3),

[0125]

[0126] Among them, the coordinates of the three ultrasonic receivers are (x1, y1, z1), (x2, y2, z2), and (x3, y3, z3) which are known, the coordinates of the ultrasonic transmitter are (x, y, z), and R1, R2, and R3 represent the distances between the three ultrasonic sensors and the ultrasonic receivers.

[0127] like Figure 14As shown, the ultrasonic transmitter 1410 includes an ultrasonic transmitter 1411 and an ultrasonic transmitter 1412, and the ultrasonic receiver 1420 includes an ultrasonic receiver 1421, an ultrasonic receiver 1422, and an ultrasonic receiver 1423. The processor can determine the coordinates of the ultrasonic transmitter 1411 and the ultrasonic transmitter 1412 respectively based on the position information of the three ultrasonic receivers using formula (3).

[0128] In some embodiments, the position information of the ultrasonic transmitter may be the position information of the ultrasonic transmitter corresponding to the movement of the user's upper arm. In other words, the coordinates of the ultrasonic transmitter 1411 and the ultrasonic transmitter 1412 when the upper arm of the human body is in a certain posture can be determined by the above method.

[0129] In some embodiments, determining the position change information of at least two ultrasonic transmitters based on the position information of at least three ultrasonic receivers may include: determining the position information of a reference point based on the position information of the at least three ultrasonic receivers, and determining the position change information of the at least two ultrasonic transmitters based on the position information of the reference point and the position information of the ultrasonic transmitter corresponding to the movement of the user's upper arm. The reference point may refer to any point in the graph formed by the at least three ultrasonic receivers. For example, the reference point may be the geometric center of the graph formed by the at least three ultrasonic receivers. Figure 14 As shown, the processor can determine the coordinates of the geometric center of the triangle formed by the three ultrasonic receivers based on the coordinate information of the ultrasonic receiver 1421, the ultrasonic receiver 1422 and the ultrasonic receiver 1423, and use the geometric center of the triangle as the reference point. The position change information refers to the difference in the distance from the reference point to different ultrasonic transmitters when the user is in a certain motion posture. For example, the ultrasonic transmitter includes Figure 14 The ultrasonic transmitter 1411 and the ultrasonic transmitter 1412 shown, when the user's upper arm is in the chest expansion action with the arm extended, the position change information of the two ultrasonic transmitters includes the difference between the distance between the reference point and the ultrasonic transmitter 1411 and the distance between the reference point and the ultrasonic transmitter 1412. For example only, when the user stands naturally with the arm stretched downward, the distance between the reference point and the two ultrasonic transmitters (for example, X1' and X2') can be considered to be approximately equal. When the user's upper arm rotates with the shoulder joint as the center, the two ultrasonic transmitters rotate relative to the reference point. At this time, the distances X1' and X2' between the two ultrasonic transmitters and the reference point change, and the values ​​corresponding to X1' and X2' are not equal. The degree of rotation and direction of movement of the upper arm relative to the torso can be determined by the size and positive and negative of the difference between X1' and X2' (i.e., the position change information of the two ultrasonic transmitters). For specific details about the rotation angle of the upper arm relative to the torso, please refer to Figures 5 to 6C Related content.

[0130] Step 1220: Identify the motion state of the upper arm relative to the torso based on the position change information of at least two ultrasonic transmitters.

[0131] The motion state of the upper arm relative to the torso may include the rotation angle, motion direction, etc. of the upper arm relative to the torso. The rotation angle can be used to represent the degree of rotation of a specific upper arm position (e.g., the upper arm area corresponding to the armpit) relative to the torso with the shoulder joint as the center.

[0132] In some embodiments, the processor can determine the distances (e.g., X1' and X2') between the reference point and the two ultrasonic transmitters based on the position (coordinates) of the reference point and the position information of the two ultrasonic transmitters, and determine the rotation angle of the upper arm relative to the torso using the above formula (2). For example, the processor can use the distance between ultrasonic transmitter 1411 and the reference point as X1' and the distance between ultrasonic transmitter 1412 and the reference point as X2', and then substitute X1=X1' and X2=X2' into formula (2) to determine the rotation angle of the upper arm relative to the torso.

[0133] It should also be noted that the result determined by formula (2) is a relative quantity that can represent the degree of rotation of the upper arm, but is not the precise rotation angle of the upper arm relative to the trunk.

[0134] In some embodiments, the processor can determine the motion state of the upper arm relative to the torso based on the rotation angle of the upper arm relative to the torso, the position information of the ultrasonic transmitter and the ultrasonic receiver, the information of the ultrasonic transmitter transmitting the ultrasonic wave, and the information of the ultrasonic receiver receiving the ultrasonic wave. For example, when the ultrasonic receiver located on the chest receives the ultrasonic wave emitted by the ultrasonic transmitter located on the upper arm, the processor can determine that the upper arm has extended forward relative to the torso. Furthermore, when the ultrasonic transmitter emits a sound wave that is received by the ultrasonic receiver on the chest, the information of the ultrasonic transmitter transmitting the ultrasonic wave and the ultrasonic receiver receiving the ultrasonic wave can be used to determine the distance between the ultrasonic transmitter and the ultrasonic receiver on the chest, thereby determining the angle of the arm extending forward. Furthermore, based on the rotation angle of the upper arm relative to the torso, the processor can determine the rotation angle of the upper arm (e.g., the area corresponding to the armpit) relative to the torso (the waist area), thereby determining the motion state of the upper arm relative to the torso.

[0135] In some embodiments, in order to avoid signal crosstalk between different ultrasonic transmitters, different ultrasonic transmitters can be time-division multiplexed so that there is a certain time interval between the transmission signals of different ultrasonic transmitters. In this case, if the number of ultrasonic transmitters is too large, the sampling rate will decrease, that is, the more ultrasonic transmitters there are, the fewer signals the ultrasonic receiver will collect from the same ultrasonic transmitter per second. Therefore, in order to improve the sampling rate of the same ultrasonic transmitter, the fewer ultrasonic transmitters used in the overall layout, the better. Placing the ultrasonic transmitter on the upper arm can save the most ultrasonic transmitters while ensuring that a variety of human postures can be recognized. For more information about time-division multiplexing of ultrasonic transmitters, please refer to Figure 17 and its description.

[0136] At the same time, the above design can make the calculation results more accurate. When using formula (2), its positioning accuracy requires that R1, R2, and R3 are the distances between the ultrasonic transmitter and multiple ultrasonic receivers at the same time, that is, R1, R2, and R3 are determined at the same time as much as possible. The larger the time difference, the greater the error. One ultrasonic transmitter transmits ultrasonic waves, and three ultrasonic receivers receive them, which can maximize the simultaneity of ultrasonic waves. On the contrary, if three ultrasonic transmitters transmit ultrasonic waves and one ultrasonic receiver receives them, under the influence of time-sharing multiplexing of the ultrasonic transmitter, there is a large time difference in the transmission time of the three ultrasonic transmitters. Then, the obtained R1, R2, and R3 are not the data at the same time, and the calculation error will be large.

[0137] By using ultrasonic sensors and designing their placement, they can also be used to determine the posture of the user's legs.

[0138] Figure 16 is an exemplary flow chart of a method for determining the posture of a user's legs according to some embodiments of this specification. In some embodiments, process 1600 may be executed by processor 120. Figure 16 As shown, process 1600 may include the following steps:

[0139] Step 1610: Based on the information of ultrasonic waves transmitted by the ultrasonic transmitter and the information of ultrasonic waves received by the ultrasonic receiver, obtain the distance between the ultrasonic transmitter and the ultrasonic receiver.

[0140] In some embodiments, the ultrasonic transmitter and ultrasonic receiver may be located on the user's thighs or calves, respectively, to identify the user's leg posture. For example, one ultrasonic transmitter and one ultrasonic receiver may be located on the left thigh, and the other on the right thigh. For another example, one ultrasonic transmitter and one ultrasonic receiver may be located on the left calf, and the other on the right calf.

[0141] In some embodiments, when the ultrasonic sensor is integrated into clothing, the ultrasonic transmitter and the ultrasonic receiver can be respectively located at the two trouser legs of the clothing to identify the posture of the user's legs. For example, one of the ultrasonic transmitter and the ultrasonic receiver is located at the left trouser leg corresponding to the left thigh, and the other is located at the right trouser leg corresponding to the right thigh. For another example, one of the ultrasonic transmitter and the ultrasonic receiver is located at the left trouser leg corresponding to the left calf, and the other is located at the right trouser leg corresponding to the right calf. Figure 15 As shown, the ultrasonic transmitter 1510 and the ultrasonic receiver 1520 may be located on the trouser legs of the garment corresponding to the thigh, and the ultrasonic transmitter 1530 and the ultrasonic receiver 1540 may be located on the trouser legs of the garment corresponding to the calf.

[0142] Based on the information of ultrasonic transmitter transmitting ultrasonic waves and ultrasonic receiver receiving ultrasonic waves, the distance between the ultrasonic transmitter and the ultrasonic receiver can be obtained by referring to Figure 3 Get the relevant content in .

[0143] Step 1620 : Determine the posture of the user's legs based on the distance between the ultrasonic transmitter and the ultrasonic receiver and the position information of the ultrasonic transmitter and the ultrasonic receiver.

[0144] The leg posture refers to information related to the position and posture of the two legs, such as the angle between the extension directions of the two legs, the movement state of the two legs (such as standing, walking, running, etc.), and the posture of the two legs (legs crossed, legs together, etc.).

[0145] In some embodiments, the processor can determine the posture of the user's legs based on the position information of the ultrasonic transmitter and the ultrasonic receiver, and the comparison of the distance between the ultrasonic transmitter and the ultrasonic receiver with a threshold. For example, a group of ultrasonic sensors (e.g., a first ultrasonic transmitter and a first ultrasonic receiver) are set at the user's left thigh and right thigh, and another group of ultrasonic sensors (e.g., a second ultrasonic transmitter and a second ultrasonic receiver) are set at the user's left calf and right calf. The processor can determine that the user's leg posture is legs together based on the distance between the first ultrasonic transmitter and the first ultrasonic receiver being less than a threshold (e.g., 4 cm), and the distance between the second ultrasonic transmitter and the second ultrasonic receiver being less than a threshold (e.g., 6 cm). It should be noted that the threshold can be set according to different users.

[0146] By placing ultrasonic transmitters and receivers at different locations on the human body, posture recognition can be performed on different parts of the body. As described above, by placing ultrasonic transmitters and receivers on the elbows or knees of a user, the movement posture of the user's arms or legs can be recognized. By placing ultrasonic transmitters on the upper arms and receivers on the torso, the posture of the upper arms relative to the torso can be recognized. By placing ultrasonic transmitters and receivers on the thighs or calves of a user, the posture of the legs can be recognized.

[0147] By simultaneously setting up ultrasonic transmitters and ultrasonic receivers at different parts of the human body, the movement posture of the user's arms or legs, the posture of the upper arms relative to the torso, and the posture of the user's legs can be obtained at the same time, thereby realizing the recognition of the user's whole body posture.

[0148] In this system, multiple ultrasonic transmitters need to be used simultaneously to transmit ultrasonic waves. In some embodiments, in order to avoid crosstalk between ultrasonic waves from different ultrasonic transmitters, time division multiplexing can be used to distinguish the emission times of different ultrasonic waves in the time domain.

[0149] Figure 17 This is an exemplary schematic diagram of implementing time-division multiplexing based on ultrasonic sensors according to some embodiments of this specification.

[0150] Time division multiplexing may refer to different devices (eg, ultrasonic transmitters) generating ultrasonic waves at different time periods. In some embodiments, time division multiplexing may be implemented based on different ultrasonic transmitters in each group of ultrasonic sensors emitting ultrasonic waves at different times.

[0151] In some embodiments, each group of ultrasonic sensors includes multiple ultrasonic transmitters, and the time points at which the multiple ultrasonic transmitters emit ultrasonic waves have time intervals. Figure 17 As shown, the microcontroller controls the working states of the ultrasonic transmitter 1710 and the ultrasonic transmitter 1720 respectively through control signals. When the ultrasonic transmitter 1710 transmits ultrasonic waves, the ultrasonic transmitter 1720 is in a dormant state and does not transmit ultrasonic waves. At this time, the ultrasonic receivers 1730, 1740, and 1750 all receive ultrasonic waves from the ultrasonic transmitter 1710. After a time Δt, the ultrasonic transmitter 1720 starts to transmit ultrasonic waves.

[0152] Due to the size of the human body, the distance L between the ultrasonic transmitter and the ultrasonic receiver is less than 1m, so the transmission time t of the ultrasonic wave is less than 2.9ms. Therefore, to prevent crosstalk between different signals, it is necessary to limit the time interval between the ultrasonic waves emitted by two adjacent ultrasonic transmitters. In some embodiments, the time interval between the ultrasonic waves emitted by two adjacent ultrasonic transmitters is greater than 2.9ms. With this configuration, even if all ultrasonic receivers are always in operation, they can receive and distinguish ultrasonic waves from different ultrasonic transmitters in real time. Based on the time difference of the ultrasonic waves transmitted by the ultrasonic receivers, the microcontroller can calculate the relative distance and position change of the corresponding ultrasonic transmitter.

[0153] In some embodiments, to prevent crosstalk between ultrasonic waves from different ultrasonic transmitters, the frequencies of the ultrasonic waves emitted by different ultrasonic transmitters can be set. For example, each group of ultrasonic sensors includes multiple ultrasonic transmitters, and the frequencies of the ultrasonic waves emitted by the multiple ultrasonic transmitters are different. For example, the frequency of the ultrasonic wave emitted by ultrasonic transmitter 1710 is 50kHz to 140kHz, and the frequency of the ultrasonic wave emitted by ultrasonic transmitter 1720 is 200kHz to 300kHz. When the ultrasonic receiver receives the ultrasonic wave, the processor can locate the ultrasonic transmitter that emitted the ultrasonic wave based on the frequency of the ultrasonic wave and further determine the distance and position of the ultrasonic transmitter and the ultrasonic receiver.

[0154] In some embodiments, to prevent crosstalk between ultrasonic waves from different ultrasonic transmitters, the encoding of the ultrasonic waves emitted by the ultrasonic transmitters can be configured. For example, each ultrasonic sensor group includes multiple ultrasonic transmitters, and the ultrasonic waves emitted by the multiple ultrasonic transmitters have different encodings. For example, the processor can encode the ultrasonic waves using a universal codec IC group, allowing multiple ultrasonic transmitters to emit ultrasonic waves with different encodings.

[0155] It is understandable that the human posture recognition system may use multiple groups of ultrasonic sensors. The multiple ultrasonic transmitters included in the multiple groups of ultrasonic sensors may cause signal crosstalk. In order to avoid this situation, different groups of ultrasonic transmitters can be set.

[0156] In some embodiments, each group of ultrasonic sensors includes multiple ultrasonic transmitters, and the ultrasonic transmitters in different groups emit ultrasonic waves at intervals. For example, if signals from a first ultrasonic transmitter in group A and a second ultrasonic transmitter in group B may crosstalk, the first ultrasonic transmitter in group A and the second ultrasonic transmitter in group B may emit ultrasonic waves at a predetermined interval.

[0157] In some embodiments, each group of ultrasonic sensors includes multiple ultrasonic transmitters, and the ultrasonic transmitters in different groups emit ultrasonic waves at different frequencies. For example, the ultrasonic transmitters in group A emit ultrasonic waves at a frequency of 50 kHz to 140 kHz, while the ultrasonic transmitters in group B emit ultrasonic waves at a frequency of 200 kHz to 300 kHz.

[0158] In some embodiments, each group of ultrasonic sensors includes multiple ultrasonic transmitters, and the ultrasonic waves emitted by different groups of ultrasonic transmitters have different codes. For example, the ultrasonic transmitters in group A emit codes I, II, and III, while the ultrasonic transmitters in group B emit codes V, VI, and VII.

[0159] By using the above-mentioned time-division multiplexing and setting the frequency and coding of the ultrasonic waves emitted by the ultrasonic transmitter, signal crosstalk can be avoided, and the incorrect calculation of the distance between the ultrasonic transmitter and the ultrasonic receiver can be prevented from affecting the recognition of the user's posture.

[0160] The basic concepts have been described above. It will be apparent to those skilled in the art that the detailed disclosure above is merely illustrative and does not limit the present application. Although not explicitly stated herein, those skilled in the art may make various modifications, improvements, and amendments to the present application. Such modifications, improvements, and amendments are suggested in the present application and remain within the spirit and scope of the exemplary embodiments of the present application.

[0161] At the same time, this application uses specific terms to describe the embodiments of this application. For example, "one embodiment," "an embodiment," and / or "some embodiments" refer to a certain feature, structure, or characteristic related to at least one embodiment of this application. Therefore, it should be emphasized and noted that "one embodiment," "an embodiment," or "an alternative embodiment" mentioned twice or multiple times in different locations in this specification does not necessarily refer to the same embodiment. In addition, certain features, structures, or characteristics in one or more embodiments of this application may be appropriately combined.

[0162] In addition, it will be understood by those skilled in the art that various aspects of the present application can be illustrated and described by a number of patentable categories or situations, including any new and useful process, machine, product or combination of substances, or any new and useful improvements thereto. Accordingly, various aspects of the present application can be performed entirely by hardware, entirely by software (including firmware, resident software, microcode, etc.), or by a combination of hardware and software. The above hardware or software may all be referred to as "data blocks", "modules", "engines", "units", "components" or "systems". In addition, various aspects of the present application may be represented as a computer product located in one or more computer-readable media, which includes computer-readable program code.

[0163] A computer storage medium may include a propagated data signal embodying the computer program code, for example, in baseband or as part of a carrier wave. The propagated signal may be in a variety of forms, including electromagnetic, optical, or any suitable combination thereof. A computer storage medium may be any computer-readable medium other than a computer-readable storage medium that can be connected to an instruction execution system, apparatus, or device to communicate, propagate, or transfer the program for use. The program code on the computer storage medium may be transmitted via any suitable medium, including radio, cable, fiber optic cable, RF, or similar media, or any combination of these.

[0164] The computer program code required for the operation of each part of the present application can be written in any one or more programming languages, including object-oriented programming languages ​​such as Java, Scala, Smalltalk, Eiffel, JADE, Emerald, C++, C#, VB.NET, Python, etc., conventional procedural programming languages ​​such as C language, Visual Basic, Fortran 2003, Perl, COBOL 2002, PHP, ABAP, dynamic programming languages ​​such as Python, Ruby and Groovy, or other programming languages. The program code can be run entirely on the user's computer, or as a separate software package on the user's computer, or partly on the user's computer and partly on a remote computer, or entirely on a remote computer or server. In the latter case, the remote computer can be connected to the user's computer through any network form, such as a local area network (LAN) or a wide area network (WAN), or connected to an external computer (e.g., via the Internet), or in a cloud computing environment, or used as a service such as software as a service (SaaS).

[0165] In addition, unless expressly stated in the claims, the order of the processing elements and sequences described in this application, the use of alphanumeric characters, or the use of other names are not intended to limit the order of the processes and methods of this application. Although the above disclosure discusses some of the invention embodiments currently considered useful through various examples, it should be understood that such details are only for illustrative purposes, and the attached claims are not limited to the disclosed embodiments. On the contrary, the claims are intended to cover all modifications and equivalent combinations that are consistent with the essence and scope of the embodiments of this application. For example, although the system components described above can be implemented by hardware devices, they can also be implemented only by software solutions, such as installing the described system on an existing server or mobile device.

[0166] Similarly, it should be noted that, in order to simplify the presentation of this application and thus facilitate understanding of one or more embodiments of the invention, the foregoing descriptions of the embodiments of this application sometimes combine multiple features into a single embodiment, figure, or description thereof. However, this disclosure method does not mean that the subject matter of this application requires more features than those recited in the claims. In fact, an embodiment may have fewer features than all of the features of a single embodiment disclosed above.

[0167] In some embodiments, numbers are used to describe the quantity of components and attributes. It should be understood that such numbers used in the description of the embodiments are modified by the modifiers "about", "approximately" or "substantially" in some examples. Unless otherwise stated, "about", "approximately" or "substantially" indicate that the numbers are allowed to vary by ±20%. Accordingly, in some embodiments, the numerical parameters used in the description and claims are approximate values, which may change according to the required features of individual embodiments. In some embodiments, the numerical parameters should take into account the specified significant digits and adopt the general method of retaining digits. Although the numerical domains and parameters used to confirm the breadth of their range in some embodiments of the present application are approximate values, in specific embodiments, the settings of such numerical values ​​are as accurate as possible within the feasible range.

[0168] Each patent, patent application, patent application disclosure, and other materials, such as articles, books, specifications, publications, documents, etc., cited in this application is hereby incorporated by reference in its entirety. This includes application history documents that are inconsistent with or conflict with the content of this application, as well as documents (currently or subsequently attached to this application) that limit the broadest scope of the claims of this application. It should be noted that if the descriptions, definitions, and / or use of terms in the accompanying materials of this application are inconsistent or conflicting with the content of this application, the descriptions, definitions, and / or use of terms in this application shall prevail.

[0169] Finally, it should be understood that the embodiments described in this application are merely illustrative of the principles of the embodiments of this application. Other variations may also fall within the scope of this application. Therefore, by way of example and not limitation, alternative configurations of the embodiments of this application may be considered consistent with the teachings of this application. Accordingly, the embodiments of this application are not limited to the embodiments explicitly introduced and described in this application.

Claims

1. A human posture recognition system, comprising: At least one set of ultrasonic sensors, each set of ultrasonic sensors comprising: an ultrasonic transmitter for transmitting ultrasonic waves and an ultrasonic receiver for receiving the ultrasonic waves, wherein the ultrasonic transmitter and the ultrasonic receiver are respectively located at different parts of the user's body; and a processor configured to recognize the user's posture based on position information of the ultrasonic transmitter and the ultrasonic receiver, information of the ultrasonic transmitter transmitting the ultrasonic wave, and information of the ultrasonic receiver receiving the ultrasonic wave, wherein the position information of the ultrasonic transmitter refers to spatial distribution information of the ultrasonic transmitter on the user's body, and the position information of the ultrasonic receiver refers to spatial distribution information of the ultrasonic receiver on the user's body; and a garment, wherein the at least one set of ultrasonic sensors is integrated into the garment; The at least one group of ultrasonic sensors includes a first group of ultrasonic sensors, wherein the ultrasonic transmitters and the ultrasonic receivers in the first group of ultrasonic sensors are distributed on the sleeves or trouser legs of the garment, and when the user wears the garment, the ultrasonic transmitters and the ultrasonic receivers in the first group of ultrasonic sensors are respectively located on both sides of the user's limbs corresponding to the elbow joints or knee joints; The identifying the user's gesture based on the position information of the ultrasonic transmitter and the ultrasonic receiver, the information of the ultrasonic transmitter transmitting the ultrasonic wave, and the information of the ultrasonic receiver receiving the ultrasonic wave includes: acquiring a distance between the ultrasonic transmitter and the ultrasonic receiver in the first group of ultrasonic sensors based on information that the ultrasonic transmitter in the first group of ultrasonic sensors transmits the ultrasonic wave and information that the ultrasonic receiver in the first group of ultrasonic sensors receives the ultrasonic wave; Based on the distance between the ultrasonic transmitter and the ultrasonic receiver in the first group of ultrasonic sensors and the position information of the ultrasonic transmitter and the ultrasonic receiver in the first group of ultrasonic sensors, the bending angle between the two sides of the limbs corresponding to the elbow joint or the knee joint is obtained.

2. The human body posture recognition system according to claim 1, wherein: The first group of ultrasonic sensors includes an ultrasonic transmitter and at least two ultrasonic receivers, the at least two ultrasonic receivers include a first ultrasonic receiver and a second ultrasonic receiver, and identifying the user's gesture based on position information of the ultrasonic transmitter and the ultrasonic receiver, information of the ultrasonic transmitter transmitting the ultrasonic wave, and information of the ultrasonic receiver receiving the ultrasonic wave includes: determining a first distance between the ultrasonic transmitter in the first group of ultrasonic sensors and the first ultrasonic receiver, and a second distance between the ultrasonic transmitter in the first group of ultrasonic sensors and the second ultrasonic receiver based on information that the ultrasonic transmitter in the first group of ultrasonic sensors transmits the ultrasonic wave, information that the first ultrasonic receiver receives the ultrasonic wave, and information that the second ultrasonic receiver receives the ultrasonic wave; The rotation angle between the two sides of the limbs corresponding to the elbow joint or the knee joint is determined according to the first distance and the second distance.

3. The human body posture recognition system according to claim 1, wherein: When the elbow joint or knee joint is in a bent state, the sides of the limbs on both sides that are facing each other are characterized as the inner sides of the limbs corresponding to the elbow joint or knee joint. In the first group of ultrasonic sensors, the ultrasonic transmitter and the ultrasonic receiver are located on the inner sides of the limbs corresponding to the elbow joint or knee joint.

4. The human body posture recognition system according to claim 1, wherein: When the user stretches out his arms or legs, the distance between the ultrasonic transmitter and the ultrasonic receiver in the first group of ultrasonic sensors is not less than 10 cm.

5. The human body posture recognition system according to any one of claims 1 to 4, wherein in the first group of ultrasonic sensors, The ultrasonic transmitter comprises an output end for emitting the ultrasonic wave, wherein the output end faces away from the clothing; The ultrasonic receiver comprises a receiving end for receiving the ultrasonic wave, wherein the receiving end faces away from the clothing; The angle between the plane where the output end is located and the plane where the receiving end is located is not greater than 170°.

6. The human gesture recognition system according to claim 5, wherein in the first group of ultrasonic sensors, the ultrasonic transmitter is tilted relative to the user's body contact point below it and toward the ultrasonic receiver, and the angle between the normal direction of the output end and the normal direction of the user's body contact point below the ultrasonic transmitter is not less than 5°.

7. The human gesture recognition system according to claim 5, wherein in the first group of ultrasonic sensors, the ultrasonic receiver is tilted relative to the user's body contact point below it and toward the ultrasonic transmitter, and the angle between the normal direction of the receiving end and the normal direction of the user's body contact point below the ultrasonic receiver is not less than 5°.

8. The human posture recognition system according to claim 1, wherein the at least one group of ultrasonic sensors includes a second group of ultrasonic sensors, wherein the ultrasonic transmitter in the second group of ultrasonic sensors is located at the upper arm of the clothing corresponding to the human body, and the ultrasonic receiver is located at the torso of the clothing corresponding to the human body, and the ultrasonic receiver in the second group of ultrasonic sensors cooperates with the ultrasonic receiver to recognize the posture of the upper arm relative to the torso.

9. The human body posture recognition system according to claim 8, wherein: The posture of the upper arm relative to the torso includes the angle of the upper arm relative to the torso, and identifying the posture of the upper arm relative to the torso includes: acquiring a distance between the ultrasonic transmitter and the ultrasonic receiver in the second group of ultrasonic sensors based on information that the ultrasonic transmitter in the second group of ultrasonic sensors transmits the ultrasonic wave and information that the ultrasonic receiver in the second group of ultrasonic sensors receives the ultrasonic wave; The angle of the upper arm relative to the torso is determined based on the distance between the ultrasonic transmitters in the second group of ultrasonic sensors and the position information of the ultrasonic transmitters and the ultrasonic receivers in the second group of ultrasonic sensors.

10. The human body posture recognition system according to claim 8, wherein: In the second group of ultrasonic sensors, the ultrasonic transmitter includes at least two ultrasonic transmitters, the ultrasonic receiver includes at least three ultrasonic receivers, and the identifying the posture of the user's upper arm relative to the torso includes: determining position change information of the at least two ultrasonic transmitters based on position information of the at least three ultrasonic receivers; Identifying the motion state of the upper arm relative to the trunk based on position change information of the at least two ultrasonic transmitters; The determining of the position change information of the at least two ultrasonic transmitters based on the position information of the at least three ultrasonic receivers includes: determining the position information of a reference point according to the position information of the at least three ultrasonic receivers, and determining the position change information of the at least two ultrasonic transmitters based on the position information of the reference point and the position information of the ultrasonic transmitter corresponding to the movement of the user's upper arm. The human body gesture recognition system according to claim 10 , wherein the at least three ultrasonic receivers are not on the same straight line.

12. The human body posture recognition system according to claim 10, wherein: The distance between the at least two ultrasonic transmitters is not less than 1 cm, and the distance between the at least three ultrasonic receivers is not less than 1 cm.

13. The human body posture recognition system according to any one of claims 8 to 12, wherein: The torso portion of the clothing corresponding to the human body includes at least one of the front side of the left shoulder, the front side of the right shoulder, the left waist, the right waist, and the chest.

14. The human gesture recognition system according to claim 1, wherein the at least one group of ultrasonic sensors comprises a third group of ultrasonic sensors, wherein: The ultrasonic transmitter and the ultrasonic receiver are respectively located at two trouser legs of the garment, and the identifying the user's posture based on position information of the ultrasonic transmitter and the ultrasonic receiver, information of the ultrasonic transmitter transmitting the ultrasonic wave, and information of the ultrasonic receiver receiving the ultrasonic wave includes: acquiring, based on information of the ultrasonic transmitter in the third group of ultrasonic sensors transmitting the ultrasonic wave and information of the ultrasonic receiver in the third group of ultrasonic sensors receiving the ultrasonic wave, a distance between the ultrasonic transmitter and the ultrasonic receiver in the third group of ultrasonic sensors; The posture of the user's legs is determined based on the distance between the ultrasonic transmitter and the ultrasonic receiver in the third group of ultrasonic sensors and the position information of the ultrasonic transmitter and the ultrasonic receiver in the third group of ultrasonic sensors.

15. The human body posture recognition system according to claim 1, wherein: Each group of ultrasonic sensors includes a plurality of ultrasonic transmitters, and the time points at which the plurality of ultrasonic transmitters emit ultrasonic waves have time intervals.

16. The human body posture recognition system according to claim 15, wherein: The time interval between two adjacent ultrasonic transmitters emitting ultrasonic waves is greater than 2.9 ms.

17. The human body posture recognition system according to claim 1, wherein: Each group of ultrasonic sensors includes a plurality of ultrasonic transmitters, and the ultrasonic waves emitted by the plurality of ultrasonic transmitters have different frequencies.

18. The human body gesture recognition system according to claim 1, wherein: Each group of ultrasonic sensors includes a plurality of ultrasonic transmitters, and the ultrasonic waves emitted by the plurality of ultrasonic transmitters have different codes.

Images