A method of gesture recognition and related devices

Abstract

The embodiment of the application discloses a posture recognition method, characterized in that the method comprises the following steps: collecting a first image of a target object at a first time through a camera; determining first joint information corresponding to the target object at the first time according to the first image, wherein the first joint information comprises coordinate values of a plurality of preset joints of the target object on a first coordinate plane; collecting pressure sensing data of the target object at the first time through a pressure sensing mat; determining first measurement pressure information of the target object at the first time according to the pressure sensing data, wherein the first measurement pressure information comprises coordinate values and corresponding pressure values of a plurality of measurement feature points of the target object on a second coordinate plane; and recognizing a first target posture of the target object at the first time based on the first joint information and the first measurement pressure information. The embodiment of the application can more accurately recognize the posture of a user and improve the user experience.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of terminal, in particular to a gesture recognition method and related device. BACKGROUND

[0002] With the continuous improvement of living standards, people pay more and more attention to the health status of the body, and the group of fitness enthusiasts gradually increases. If there is a professional guide in the fitness process, the exercise effect can be better achieved and serious sports injuries can be avoided. However, the current professional fitness trainers are not only few in number, but also high in price, which is difficult to meet the needs of all fitness enthusiasts. Therefore, intelligently recognizing the user's exercise posture and detecting the user's exercise effect during the user's exercise can help the user achieve an effective fitness state.

[0003] At present, in the prior art, the user's exercise posture can be recognized based on images, that is, the real-time action image of the user is obtained through the camera, and then the human body posture in the image is recognized using a pre-trained deep neural network, and then the coordinates of each limb joint of the user are obtained. However, this technology has low recognition accuracy for the occluded limb parts, especially for ground-based actions (such as yoga, abdominal muscle training, etc.). In these actions, the user's limbs will overlap and occlude each other, resulting in reduced recognition accuracy in the image recognition process, thereby bringing poor user experience to the user.

[0004] Therefore, how to more accurately recognize the user's posture is a problem to be solved. SUMMARY

[0005] The technical problem to be solved by the embodiments of the present application is to provide a gesture recognition method and related device, which can more accurately recognize the user's posture and improve the user experience.

[0006] In a first aspect, the embodiments of the present application provide a gesture recognition method, characterized in that the method comprises: collecting a first image of a target object at a first time through a camera; determining first joint information corresponding to the target object at the first time according to the first image, the first joint information comprising coordinate values of a plurality of preset joints of the target object on a first coordinate plane; collecting pressure sensing data of the target object at the first time through a pressure sensing mat; determining first measurement pressure information of the target object at the first time according to the pressure sensing data, the first measurement pressure information comprising coordinate values of a plurality of measurement feature points of the target object on a second coordinate plane and corresponding pressure values; and identifying a first target posture of the target object at the first time based on the first joint information and the first measurement pressure information.

[0007] In the prior art, due to the fact that some ground actions (such as yoga, abdominal muscle training, etc.) performed by a user will cause limbs to overlap or limbs to be blocked, when real-time action images of the user are acquired and an image recognition algorithm is used to recognize a human posture in the images, for the limbs overlapping or the limbs blocking parts, the image recognition algorithm will cause a large error between a predicted posture and an actual posture according to image information, resulting in a low posture recognition accuracy.

[0008] In the embodiment of the present application, by combining image information and pressure information to recognize a user posture, for the limbs overlapping or the limbs blocking parts, the prediction can be performed based on the image information in combination with the pressure information or based on the pressure information in combination with the image information, thereby avoiding the problem that when the limbs overlapping or the limbs blocking parts are predicted only by the image information, there is less effective information about the overlapping or blocking parts, resulting in a large difference between the predicted result and the actual situation. Specifically, the embodiment of the present application acquires images of the user by a camera and pressure sensing data by a pressure mat, and generates corresponding image information and pressure information according to the images and the pressure sensing data, and then combines the image information and the pressure information to recognize a target posture of the user. Since the pressure information is information obtained according to pressure sensing data of actual actions of the user, the prediction of the limbs overlapping or the limbs blocking parts is more accurate, so that the user posture recognized after the image information and the pressure information are combined is more accurate, the recognition accuracy is improved, and the user experience is improved.

[0009] In a possible implementation manner, the determining the first measurement pressure information of the target object at the first time according to the pressure sensing data comprises: performing normalization processing on the pressure sensing data, and clearing zero pressure sensing data lower than a first preset threshold in the normalized pressure sensing data to obtain the first measurement pressure information.

[0010] In the embodiment of the present application, the acquired pressure sensing data is normalized, so that the pressure sensing data is limited in a certain range, such as between 0 and 1, thereby eliminating the adverse effects caused by singular data and facilitating the comparison between pressure data of different postures. Further, the normalized pressure sensing data is preprocessed, and the pressure sensing data lower than the preset threshold in the data is cleared to zero to obtain measurement pressure information, and the processing efficiency of the data can be improved when the user posture is recognized according to the measurement pressure information.

[0011] In a possible implementation, the first target posture of the target object at the first time is identified based on the first joint node information and the first measured pressure information, including: generating first predicted pressure information according to the first joint node information, the first predicted pressure information including coordinate values of a plurality of predicted feature points matched with the first joint node information on the first coordinate plane and corresponding pressure values; calculating a first similarity between the first predicted pressure information and the first measured pressure information; and if the first similarity is greater than a second preset threshold, generating the first target posture based on the first joint node information.

[0012] In the embodiment of the present application, the measured pressure information and the image information at the same time are heterogeneous information and cannot be directly fused, and the image information is processed to obtain predicted pressure information according to human body measurement and physical principles, and the measured pressure information is further compared with the predicted pressure information, and if the similarity between the two is greater than a preset threshold, the target posture of the user can be obtained according to the image information. Through the method provided in the embodiment of the present application, it can be determined whether the target posture obtained according to the image information is close to the actual posture of the user through the pressure information, and if the target posture obtained according to the image information is close to the actual posture of the user, the target posture of the user obtained according to the image information is output, thereby improving the posture recognition accuracy of the user.

[0013] In a possible implementation, the first similarity between the first predicted pressure information and the first measured pressure information is calculated, including: processing coordinate values of the plurality of measured feature points in the first measured pressure information on the second coordinate plane to obtain second measured pressure information; the second measured pressure information includes corresponding coordinate values and corresponding pressure values of the plurality of measured feature points mapped on the first coordinate plane; and the first similarity between the first predicted pressure information and the first measured pressure information is calculated based on the second measured pressure information and the first predicted pressure information.

[0014] In the embodiment of the present application, the image information and the pressure information can be in different coordinate planes, so that the predicted pressure information obtained according to the image information and the measured pressure information are in different coordinate planes, and therefore the predicted pressure information and the measured pressure information are unified to the same coordinate plane for comparison, thereby avoiding the problem that data in different coordinate planes cannot be directly compared, and thus improving the posture recognition accuracy of the user.

[0015] In a possible implementation, after the first similarity between the first predicted pressure information and the first measured pressure information is calculated, the method further includes: if the first similarity is less than or equal to the second preset threshold, dividing the first measured pressure information into M first measured pressure sub-information, and dividing the first predicted pressure information into M first predicted pressure sub-information, to generate M groups of to-be-compared pressure information; wherein the M first measured pressure sub-information correspond to M parts of the target object, and the M first predicted pressure sub-information correspond to the M parts; each of the groups of to-be-compared pressure information includes the first measured pressure sub-information and the first predicted pressure sub-information corresponding to the same part of the M parts of the target object; a second similarity between the first measured pressure sub-information and the first predicted pressure sub-information in each of the groups of to-be-compared pressure information is calculated; L groups of to-be-compared pressure information whose second similarity is less than or equal to a third preset threshold are determined; L second predicted pressure sub-information are determined according to the first measured pressure sub-information in the L groups of to-be-compared pressure information respectively; and the first target posture is generated based on the L second predicted pressure sub-information and M-L first predicted pressure sub-information in M-L groups of to-be-compared pressure information; the M-L groups of to-be-compared pressure information are groups of to-be-compared pressure information other than the L groups of to-be-compared pressure information in the M groups of to-be-compared pressure information.

[0016] In the embodiment of the present application, when the similarity between the measured pressure information and the predicted pressure information is less than or equal to the preset threshold in the above method, it does not mean that the target posture obtained from the image information is completely not close to the actual posture of the user, but only means that some local postures in the target posture obtained from the image information are misrecognized. Therefore, the measured pressure information and the predicted pressure information are divided into M parts according to the M parts of the target object, and then L local postures corresponding to the misrecognized information are found. Then, the L local postures are predicted through the pressure information, and then a target posture closer to the actual posture of the user is obtained according to M-L predicted pressure sub-information obtained from the image information (excluding the predicted pressure sub-information corresponding to the L misrecognized local postures) and L predicted pressure sub-information obtained from the pressure information, thereby improving the posture recognition accuracy of the user.

[0017] In a possible implementation, the determining L second predicted pressure sub-information respectively according to the first measured pressure sub-information in the L groups of pressure information to be compared comprises: determining a local erroneous posture pressure information; the local erroneous posture pressure information comprises the first measured pressure sub-information in each of the L groups of pressure information to be compared; obtaining N preset local posture information, and generating N second predicted pressure sub-information respectively based on the N preset local posture information; calculating a third similarity between the N second predicted pressure sub-information and the local erroneous posture pressure information respectively, to obtain a local candidate posture information set; the local candidate posture information set comprises preset local posture information corresponding to one or more second predicted pressure sub-information in the N second predicted pressure sub-information, and the third similarity between the one or more second predicted pressure sub-information and the local erroneous posture pressure information is greater than a fourth threshold; dividing the first joint information into M sub-information matched with the M parts of the target object, and determining a first joint sub-information; the first joint sub-information is joint information corresponding to the local erroneous posture pressure information; calculating a fourth similarity between each preset local posture information in each local candidate posture information set and the first joint sub-information respectively, to obtain a target local posture information; the target local posture information is preset local posture information with the highest fourth similarity; and obtaining the L second predicted pressure sub-information according to the target local posture information.

[0018] In the embodiment of the present application, when the information corresponding to the L local postures that are identified as erroneous is found in the above method, the local erroneous posture pressure information is obtained according to the measured pressure sub-information in the L local postures, then the N preset local posture information is obtained, the preset local posture information is further converted into predicted pressure sub-information and compared with the local erroneous posture pressure information, the corresponding local candidate posture information set is determined, and then the information in the local candidate posture information set and the joint information obtained from the image information are combined to determine the local candidate posture information set closest to the real local posture of the user, thereby improving the posture recognition accuracy of the user.

[0019] In a possible implementation, the first target pose of the target object at the first time is identified based on the first set of joint coordinates and the first set of pressure values; the method comprises: obtaining S preset target pose information, and processing the S preset target pose information respectively to generate S second predicted pressure information; calculating a fifth similarity between the S second predicted pressure information and the first measured pressure information respectively, and adding the second predicted pressure information with the fifth similarity greater than a fifth preset threshold to a target candidate pose set; calculating a sixth similarity between the predicted target pose information in each target candidate pose set and the first joint information respectively, obtaining the predicted target pose information with the highest sixth similarity; and generating the first target pose based on the predicted target pose information with the highest sixth similarity.

[0020] In the embodiment of the present application, when the target pose obtained from image information is greatly different from the actual pose of the user, the pose of the user can be predicted based on pressure information and in combination with image information. Specifically, S preset target pose information can be obtained first, and then the preset target pose information is converted into predicted target pose information and compared with measured pressure information to determine a corresponding target candidate pose set, and then the predicted target pose information in the target candidate pose set is compared with predicted pressure information obtained from image information to determine the predicted target pose information with the highest similarity in the target candidate pose set, and then the target pose is identified according to the predicted target pose information, thereby improving the pose recognition accuracy of the user.

[0021] In a possible implementation, when the target candidate pose set comprises j predicted target pose information, before the sixth similarity between the predicted target pose information in each target candidate pose set and the first joint information is calculated, the method further comprises: comparing the j predicted target pose information in the target candidate pose set with second joint information corresponding to a second target pose output at a previous time of the first time, and removing the predicted target pose information with a similarity less than or equal to a seventh preset threshold from the target candidate pose set.

[0022] In the embodiment of the present application, when there are many predicted target pose information in the target candidate pose set under the above method, a plurality of predicted target pose information needs to be compared with measured pressure information respectively, and the calculation amount is large. Therefore, in combination with the target pose output at the previous time of the target object, the predicted target pose information in the target candidate pose set is screened, and the predicted target pose information with too low similarity is removed, thereby reducing the number of predicted target pose information to be compared, reducing the calculation overhead, improving the efficiency, and also improving the pose recognition accuracy of the user.

[0023] In a possible implementation, the method further includes: calculating a seventh similarity between the first target posture and a target posture at a neighboring time; and correcting the first target posture according to the target posture at the neighboring time if the seventh similarity is less than or equal to an eighth preset threshold.

[0024] In the embodiment of the present application, if posture recognition error occurs in an abnormal situation, the error posture can be corrected according to the target posture output at a neighboring time of the error posture, thereby improving the posture recognition accuracy of the user.

[0025] In a second aspect, the embodiment of the present application provides a device for posture recognition, and the device includes: a first receiving unit configured to collect a first image of a target object at a first time through a camera; a first processing unit configured to determine first joint information corresponding to the target object at the first time according to the first image, the first joint information including coordinate values of a plurality of preset joint nodes of the target object on a first coordinate plane; a second receiving unit configured to collect pressure sensing data of the target object at the first time through a pressure sensing mat; a second processing unit configured to determine first measured pressure information of the target object at the first time according to the pressure sensing data, the first measured pressure information including coordinate values of a plurality of measured feature points of the target object on a second coordinate plane and corresponding pressure values; and a third processing unit configured to recognize a first target posture of the target object at the first time based on the first joint information and the first measured pressure information.

[0026] In a possible implementation, the second processing unit is specifically configured to: perform normalization processing on the pressure sensing data, clear the pressure sensing data lower than a first preset threshold in the normalized pressure sensing data, and obtain the first measured pressure information.

[0027] In a possible implementation, the third processing unit is specifically configured to: generate first predicted pressure information according to the first joint information, the first predicted pressure information including coordinate values of a plurality of predicted feature points matched with the first joint information on the first coordinate plane and corresponding pressure values; calculate a first similarity between the first predicted pressure information and the first measured pressure information; and generate the first target posture based on the first joint information if the first similarity is greater than a second preset threshold.

[0028] In a possible implementation, the third processing unit is specifically configured to: process the coordinate values of the plurality of measurement feature points in the first measurement pressure information on the second coordinate plane to obtain second measurement pressure information; the second measurement pressure information comprises corresponding coordinate values and corresponding pressure values of the plurality of measurement feature points mapped on the first coordinate plane; and calculate the first similarity between the first predicted pressure information and the first measurement pressure information based on the second measurement pressure information and the first predicted pressure information.

[0029] In a possible implementation, the device further comprises: a fourth processing unit configured to, if the first similarity is less than or equal to the second preset threshold, divide the first measurement pressure information into M first measurement pressure sub-information, and divide the first predicted pressure information into M first predicted pressure sub-information, to generate M groups of to-be-compared pressure information; wherein the M first measurement pressure sub-information correspond to M parts of the target object, and the M first predicted pressure sub-information correspond to the M parts; each of the groups of to-be-compared pressure information comprises the first measurement pressure sub-information and the first predicted pressure sub-information corresponding to the same part of the M parts of the target object; the fourth processing unit is configured to calculate a second similarity between the first measurement pressure sub-information and the first predicted pressure sub-information in each of the groups of to-be-compared pressure information; the fourth processing unit is further configured to determine L groups of to-be-compared pressure information whose second similarity is less than or equal to a third preset threshold; the fourth processing unit is further configured to determine L second predicted pressure sub-information according to the first measurement pressure sub-information in the L groups of to-be-compared pressure information respectively; and the fourth processing unit is further configured to generate the first target pose based on the L second predicted pressure sub-information and M-L first predicted pressure sub-information in M-L groups of to-be-compared pressure information; the M-L groups of to-be-compared pressure information are groups of to-be-compared pressure information other than the L groups of to-be-compared pressure information in the M groups of to-be-compared pressure information.

[0030] In a possible implementation, the fourth processing unit is specifically configured to: determine local error posture pressure information; the local error posture pressure information includes the first measurement pressure sub-information of each of the L groups of to-be-compared pressure information; determine local error posture pressure information; the local error posture information includes the first measurement pressure sub-information of each of the L groups of to-be-compared pressure information; obtain N preset local posture information, and generate N second predicted pressure sub-information based on the N preset local posture information; calculate a third similarity between the N second predicted pressure sub-information and the local error posture pressure information respectively, to obtain a local candidate posture information set; the local candidate posture information set includes one or more second predicted pressure sub-information corresponding to the preset local posture information in the N second predicted pressure sub-information, and the third similarity between the one or more second predicted pressure sub-information and the local error posture pressure information is greater than a fourth threshold; divide the first joint information into M sub-information matched with the M parts of the target object, and determine a first joint sub-information; the first joint sub-information is the joint information corresponding to the local error posture pressure information; calculate a fourth similarity between each preset local posture information in each local candidate posture information set and the first joint sub-information respectively, to obtain a target local posture information; the target local posture information is the preset local posture information with the highest fourth similarity; and obtain the L second predicted pressure sub-information according to the target local posture information.

[0031] In a possible implementation, the fourth processing unit is specifically configured to: obtain S preset target posture information, and generate S second predicted pressure information based on the S preset target posture information; calculate a fifth similarity between the S second predicted pressure information and the first measurement pressure information respectively, and add the second predicted pressure information with a fifth similarity greater than a fifth preset threshold to a target candidate posture set; calculate a sixth similarity between the predicted target posture information in each target candidate posture set and the first joint information respectively, to obtain the predicted target posture information with the highest sixth similarity; and generate the first target posture based on the predicted target posture information with the highest sixth similarity.

[0032] In a possible implementation, when the target candidate posture set includes j pieces of the predicted target posture information, the device further includes a fifth processing unit configured to compare the j pieces of the predicted target posture information in the target candidate posture set with second node information corresponding to a second target posture output at a previous time of the first time, and remove the predicted target posture information with a similarity less than or equal to a seventh preset threshold from the target candidate posture set.

[0033] In a possible implementation, the device further includes a sixth processing unit configured to calculate a seventh similarity between the first target posture and a target posture at a neighboring time; and correct the first target posture according to the target posture at the neighboring time if the seventh similarity is less than or equal to an eighth preset threshold.

[0034] In a third aspect, an embodiment of the present application provides a chip system, which includes at least one processor, a memory and an interface circuit, the memory, the interface circuit and the at least one processor are interconnected through a circuit, and the at least one memory stores instructions; when the instructions are executed by the processor, any one of the methods in the first aspect is implemented.

[0035] In a fourth aspect, an embodiment of the present application provides a computer storage medium, which stores a computer program, and the computer program is executed by a processor to implement any one of the methods in the first aspect.

[0036] In a fifth aspect, an embodiment of the present application provides a computer program, which includes instructions, and when the computer program is executed by a computer, the computer executes any one of the methods in the first aspect.

[0037] In a sixth aspect, the present application provides a terminal device having a function of implementing any one of the posture recognition methods provided in the first aspect. The function can be implemented by hardware, or implemented by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions.

[0038] In a seventh aspect, the present application provides a smart device having a function of implementing any one of the posture recognition methods provided in the first aspect. The function can be implemented by hardware, or implemented by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions. BRIEF DESCRIPTION OF DRAWINGS

[0039] Figure 1 FIG. 1 is a system architecture schematic diagram of a posture recognition method provided by an embodiment of the present application.

[0040] Figure 2 is a flowchart of a posture recognition method in an embodiment of the present application.

[0041] Figure 3A is a user motion posture photo schematic diagram provided by an embodiment of the present application.

[0042] Figure 3B is a user motion posture pressure data schematic diagram provided by an embodiment of the present application.

[0043] Figure 3C is a flowchart of comparing image information with pressure information provided by an embodiment of the present application.

[0044] Figure 3D is a first coordinate plane and a second coordinate plane schematic diagram provided by an embodiment of the present application.

[0045] Figure 3E is an image coordinate system schematic diagram provided by an embodiment of the present application.

[0046] Figure 3F is a pressure coordinate system schematic diagram provided by an embodiment of the present application.

[0047] Figure 3G is an M groups of pressure information to be compared schematic diagram provided by an embodiment of the present application.

[0048] Figure 3H is a schematic diagram of selecting L groups of pressure information to be compared from M groups of pressure information to be compared provided by an embodiment of the present application.

[0049] Figure 3I is a second predicted pressure information schematic diagram provided by an embodiment of the present application.

[0050] Figure 3J is a schematic diagram of determining a local candidate posture set provided by an embodiment of the present application.

[0051] Figure 3K is a schematic diagram of determining L second predicted pressure sub-information provided by an embodiment of the present application.

[0052] Figure 3L is a schematic diagram of determining a target candidate posture set provided by an embodiment of the present application.

[0053] Figure 4A is an exemplary flowchart of a posture recognition method provided by an embodiment of the present application.

[0054] Figure 4B is an exemplary flowchart of another posture recognition method provided by an embodiment of the present application.

[0055] Figure 4C An example flowchart of a posture recognition method capable of error correction is provided in embodiments of the present application.

[0056] Figure 5 A schematic diagram of a posture recognition device is provided in embodiments of the present application. DETAILED DESCRIPTION

[0057] The terminology used in the following description of the embodiments herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used in the description of embodiments and the appended claims, the singular forms "a", "an" and "the" are intended to include both singular and plural forms, unless the context clearly indicates otherwise. It will be further understood that the terms "and / or", "comprises" and / or "comprising", when used in this specification, specify the presence of features, integers, steps, operations, elements, and / or components with one or more of them, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0058] Embodiments of electronic devices, user interfaces for such electronic devices, and associated processes for using such electronic devices are described below. In some embodiments, an electronic device can be a portable electronic device, such as a mobile phone, a tablet computer, a wearable electronic device (e.g., a smart watch) with wireless communication capabilities, or the like, that also contains other functionality such as personal digital assistant and / or music player functionality. Example embodiments of portable electronic devices include, but are not limited to, mobile phones or other operating systems. The portable electronic devices described above can also be other portable electronic devices, such as a laptop computer (Laptop) with a touch-sensitive surface or touch panel, or the like. It will also be appreciated that, in some other embodiments, the electronic devices described above can not be portable electronic devices, but rather can be desktop computers with a touch-sensitive surface or touch panel.

[0059] The term "user interface (UI)" in the specification and claims of the present application and the accompanying drawings is a medium interface for interaction and information exchange between an application or an operating system and a user, which realizes the conversion between the internal form of information and the form acceptable to the user. The user interface of an application is source code written in a specific computer language such as Java, extensible markup language (XML), etc. The interface source code is parsed, rendered, and finally presented as content recognizable by the user, such as pictures, text, buttons, etc. controls on the terminal device. Controls, also known as widgets, are the basic elements of the user interface. Typical controls include toolbars, menu bars, text boxes, buttons, scrollbars, pictures, and text. The properties and content of the controls in the interface are defined by tags or nodes, such as XML <textview> 、 <imgview> 、 <videoview>The interface is defined by nodes that specify the controls contained in the interface. One node corresponds to one control or property in the interface, and the nodes are parsed and rendered to present the content visible to the user. In addition, many applications, such as hybrid applications, also contain web pages in the interface. A web page, also referred to as a page, can be understood as a special control embedded in the interface of an application. The web page is a source code written in a specific computer language, such as hyper text markup language (HTML), cascading style sheets (CSS), JavaScript (JS), etc. The web page source code can be loaded and displayed by a browser or a web page display component similar to the function of a browser to present content recognizable to the user. The specific content contained in the web page is also defined by tags or nodes in the web page source code, such as HTML defines the content by tags, and the content is displayed by a browser or a web page display component similar to the function of a browser. 、 、 <video> 、 <canvas>To define the elements and attributes of a web page.

[0060] The commonly used form of user interface is a graphic user interface (GUI), which refers to a user interface related to computer operation displayed in a graphical manner. It can be an icon, window, control, etc. interface element displayed in the display screen of an electronic device, wherein the control can include an icon, button, menu, tab, text box, dialog box, status bar, navigation bar, Widget, etc. visual interface element.

[0061] First, some terms in the present application are explained and described to facilitate understanding by those skilled in the art.

[0062] (1) Target detection algorithm (Single Shot MultiBox Detector, SSD), target detection is the basis of many computer vision tasks, whether it is to realize the interaction between image and text or to identify fine categories, it provides reliable information. In recent years, with the hot development of deep learning technology, the target detection algorithm has also changed from the traditional algorithm based on manual features to the detection technology based on deep neural network.

[0063] The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0064] Based on the above-mentioned technical problems, in order to facilitate understanding of the embodiments of the present application, the system architecture based on the embodiments of the present application will be described first. Please refer to Figure 1 , Figure 1 is a system architecture diagram of a posture recognition provided by the embodiments of the present application. The system can more accurately recognize the user's posture and improve the user experience. The system architecture can include an image acquisition device 101, a pressure acquisition device 102 and a processor 103. Among them,

[0065] The image acquisition device 101 can be a separate camera, or a variety of devices with a camera, such as a mobile phone, a smart TV, a tablet computer, etc. The image acquisition device 101 has basic functions such as video shooting or still image capture. The image is processed and converted into a digital signal that can be recognized by the processor by the photosensitive component circuit and control component in the camera. For example, the image acquisition device 101 can be used to acquire image data when the user exercises.

[0066] The pressure acquisition device 102 can be a pressure pad or a pressure plate, etc. which can detect and record pressure sensing data. The pressure acquisition device 102 can measure the pressure values at each position on the surface of the device. The device can be composed of an array of flexible pressure sensors or a plurality of single-point pressure sensors, and the pressure sensing data at each position can be obtained by the sensors and processed by the processor. For example, the pressure acquisition device 102 can be a pressure pad which can be used to obtain the pressure sensing data generated on the pressure pad when the user exercises.

[0067] The processor 103 is the operation and control core of the computer system, and its main functions are to interpret computer instructions and process data in computer software. The processor is responsible for reading instructions, decoding instructions, and executing instructions. The processor mainly includes two parts, namely the controller and the arithmetic unit, and also includes a cache memory and a bus for realizing the connection between them. The main functions of the processor are to process instructions, perform operations, control time, and process data. For example, in the embodiment of the present application, the processor 103 can be used to receive image data from the image acquisition device 101 and pressure sensing data from the pressure acquisition device 102, and process the image data and the pressure sensing data.

[0068] Optionally, the posture recognition system architecture can also include an image display device 104, which is a display tool that displays certain electronic files to the screen through a specific transmission device and then reflects them to the human eye. For example, the image display device 104 can display images, pressure, or posture recognition results to the user in real time.

[0069] It can be understood that, Figure 1 The posture recognition system architecture in the above embodiment is only an exemplary embodiment of the present application, and the advertisement tracking system architecture in the embodiment of the present application includes but is not limited to the above system architecture.

[0070] The specific method architecture based on which the embodiment of the present application is described below. Referring to Figure 2 , Figure 2 is a flowchart of a posture recognition method in the embodiment of the present application, which will be described below in conjunction with the accompanying Figure 2 and based on the posture recognition system architecture in the above Figure 1 The posture recognition method in the embodiment of the present application is described from the interaction side of the image acquisition device, the pressure acquisition device, and the processor. It should be noted that, in order to more specifically describe the posture recognition method in the embodiment of the present application, the corresponding execution subject is described as an electronic device in each flow step, but it does not mean that the corresponding method flow can only be performed by the described execution subject in the embodiment of the present application.

[0071] Step S201: acquiring a first image of a target object at a first time through a camera.

[0072] Specifically, the electronic device acquires a motion posture photo of the user at the first time through the camera. The first time can be understood as a time when the user's posture needs to be recognized. The first image can be understood as a photo of the user's motion posture acquired by the camera at the first time. The camera can be understood as a separate camera, or a camera installed on other devices (such as a mobile phone, a smart TV, a tablet computer, etc.). For example, as shown in Figure 3A Figure 3A A user's motion posture photo schematic diagram provided by an embodiment of the present application is shown in the figure. The user lies on the pressure mat. Through the photo and in combination with an image recognition algorithm (such as a neural network algorithm), the motion posture of the right half of the user's body (such as the right hand and the right leg) can be recognized, but the motion posture of the left half of the user's body cannot be accurately recognized.

[0073] Step S202: determining first joint information corresponding to the target object at the first time according to the first image.

[0074] The first joint information includes coordinate values of a plurality of preset joints of the target object on a first coordinate plane. Specifically, according to the user's motion posture photo acquired by the camera, the coordinate values of a plurality of preset joints of the user on the image coordinate plane can be recognized. The preset joint can be understood as a joint of the human body such as the head, the left elbow, the right elbow, the left knee, and the right knee. Optionally, the user's motion posture photo is processed by using an image recognition algorithm to obtain the first joint information. For example, the first joint information can be n is the number of preset joints. It should be noted that the value of n may be different for different image recognition algorithms. n is generally 10, 14, 15, 30, etc. If a preset joint is blocked, the image recognition algorithm will guess a coordinate value for the blocked preset joint. For example, as shown in Figure 3A Suppose n is 10, the preset joints include the head, the neck, the left elbow joint, the right elbow joint, the left wrist joint, the right wrist joint, the left knee joint, the right knee joint, the left ankle joint, and the right ankle joint. According to Figure 3A Only the coordinates of the head, the neck, the right elbow joint, the right wrist joint, the right knee joint, and the right ankle joint can be accurately obtained. The image recognition algorithm will give a guessed coordinate value for the blocked left elbow joint, the left wrist joint, the left knee joint, and the left ankle joint. In the prior art, since there is little effective information about the blocked part, the coordinate value guessed by the image recognition algorithm has a large error from the actual value.

[0075] Step S203: acquiring pressure sensing data of the target object at the first time through a pressure sensing mat.

[0076] ​Specifically, when the user moves on the pressure sensing mat, since there are multiple pressure sensors on the pressure sensing mat, when the user moves on the pressure sensing mat, the pressure sensing data generated by the user at the first time can be collected by the pressure sensing mat. It can be understood that when the user's motion posture is identified, the first image and the pressure sensing data collected are data generated at the same time based on the user's motion posture.

[0077] Optionally, if the sampling frequencies of the image data obtained by the camera and the pressure data obtained by the pressure sensing mat are different, the high-frequency data can be down-sampled to make the sampling frequencies consistent; or only when the low-frequency data is refreshed, the high-frequency data can be obtained, and the high-frequency data at other times is directly discarded and does not participate in the calculation, so that the sampling frequencies of the image data and the pressure data are the same.

[0078] Step S204: determining first measurement pressure information of the target object at the first time according to the pressure sensing data.

[0079] The first measurement pressure information includes coordinate values and corresponding pressure values of multiple measurement feature points of the target object on a second coordinate plane. Specifically, after processing the collected pressure sensing data, coordinate values and pressure values of multiple measurement feature points on a pressure coordinate plane can be obtained. The multiple measurement feature points can be understood as points on the pressure pad that generate effective pressure values. For example, as shown in FIG. 2, the pressure sensing mat 200 is provided with a plurality of pressure sensors, and the user's hand 210 is placed on the pressure sensing mat 200. The user's hand 210 is in contact with the pressure sensing mat 200, and the pressure sensing mat 200 can collect pressure sensing data generated by the user's hand 210 at the first time. The pressure sensing data can be processed to obtain coordinate values and pressure values of multiple measurement feature points on a pressure coordinate plane. The multiple measurement feature points can be understood as points on the pressure sensing mat 200 that generate effective pressure values. Figure 3B Figure 3B A user's motion posture pressure data diagram provided by an embodiment of the present application is shown in FIG. 2, which is a pressure contour diagram obtained based on pressure sensing data. As can be seen from the diagram, there are areas with relatively dense pressure contours and areas with relatively sparse pressure contours. The multiple measurement feature points can be understood as points on the dense pressure contour area in the diagram, and the first measurement pressure information can be coordinate values and corresponding pressure values of the points on the pressure coordinate plane.

[0080] In a possible implementation, the determining first measurement pressure information of the target object at the first time according to the pressure sensing data comprises: performing normalization processing on the pressure sensing data, and clearing zero pressure sensing data in the normalized pressure sensing data that is lower than a first preset threshold to obtain the first measurement pressure information.

[0081] ​Specifically, the obtained pressure sensing data is normalized so that the pressure sensing data is limited in a certain range, such as between 0 and 1, so as to eliminate the adverse effects of singular data and facilitate comparison between pressure data of different postures. Further, the normalized pressure sensing data is preprocessed, and pressure sensing data below a preset threshold is cleared to obtain measurement pressure information, and then the measurement pressure information is used to identify the user posture, which can improve the processing efficiency of the data. For example, the pressure sensing data is normalized to be between 0 and 1, and the preset threshold T is set to 0.2, then the pressure values below 0.2 are cleared, only the region with larger pressure values is left, which can be understood as the region where the limb directly contacts the pressure pad. On this basis, the coordinate value of the last point in the region on the pressure coordinate plane and the pressure value thereof can be obtained, so as to obtain the first measurement pressure information.

[0082] Step S205: identifying a first target posture of the target object at the first time based on the first joint information and the first measurement pressure information.

[0083] According to the image information (such as the first joint information) and the pressure information (such as the first measurement pressure information) obtained above, the target posture of the target object at the first time is identified. Specifically, in the embodiments of the present application, two implementation manners are provided to obtain the first target posture according to the image information and the pressure information. One is to obtain the target posture of the target object based on the image information in combination with the pressure information, and the other is to obtain the target posture of the target object based on the pressure information in combination with the image information.

[0084] Implementation manner one: obtaining the target posture of the target object based on the image information in combination with the pressure information.

[0085] In a possible implementation manner, the identifying the first target posture of the target object at the first time based on the first joint information and the first measurement pressure information comprises: generating first predicted pressure information according to the first joint information, the first predicted pressure information comprising coordinate values of a plurality of predicted feature points matched with the first joint information on the first coordinate plane and corresponding pressure values; calculating a first similarity between the first predicted pressure information and the first measurement pressure information; and if the first similarity is greater than a second preset threshold, generating the first target posture based on the first joint information.

[0086] Specifically, the first predicted pressure information is information obtained according to a plurality of preset joints (such as head, neck, left and right elbow joints, left and right wrist joints, left and right knee joints, and left and right ankle joints) in the first joint information. Optionally, the plurality of preset joints (such as head, neck, left and right elbow joints, left and right wrist joints, left and right knee joints, and left and right ankle joints) in the first joint information are combined with a limb mass physiological model to obtain coordinate values of a plurality of predicted feature points in a pressure coordinate plane and corresponding pressure values. Optionally, the limb mass physiological model is constructed according to principles of anthropometry and physics. In the embodiment of the present application, as shown in Figure 3C Figure 3C A flowchart for comparing image information and pressure information is provided in the embodiment of the present application. In the flowchart, the first joint information is obtained according to the first image, and the first measured pressure information is obtained according to the pressure sensing data. However, the first measured pressure information and the first joint information at the same time are heterogeneous information and cannot be directly fused. Optionally, the first joint information is processed according to a limb mass physiological model (such as a model constructed according to principles of anthropometry and physics) to obtain the first predicted pressure information, so as to obtain the pressure distribution that may be generated by the user when performing the action according to the joint information. The first predicted pressure information can be understood as the pressure distribution information that may be generated by the user when performing the action according to the joint information. The first measured pressure information is further compared with the first predicted pressure information. If the similarity between the two is greater than a preset threshold, the target posture of the user can be obtained according to the first joint information. For example, after obtaining the first predicted pressure information, the similarity between the first predicted pressure information and the first measured pressure information is calculated, and the preset threshold is 0.9. If the similarity between the first predicted pressure information and the first measured pressure information is greater than 0.9, it can be considered that the target posture obtained according to the image information is close to the actual posture of the user, so that the target posture of the user obtained according to the image information can be output based on the first joint information. Through the method provided in the embodiment of the present application, whether the target posture obtained according to the image information is close to the actual posture of the user can be determined through the pressure information. If the target posture obtained according to the image information is close to the actual posture of the user, the target posture of the user obtained according to the image information is output, so as to improve the posture recognition accuracy of the user.

[0087] In a possible implementation manner, the calculating the first similarity between the first predicted pressure information and the first measured pressure information comprises: processing the coordinate values of the plurality of measured feature points in the first measured pressure information on the second coordinate plane to obtain second measured pressure information; the second measured pressure information comprises coordinate values and corresponding pressure values of the plurality of measured feature points mapped on the first coordinate plane; and the first similarity between the first predicted pressure information and the first measured pressure information is calculated based on the second measured pressure information and the first predicted pressure information.​

[0088] Specifically, since the first predicted pressure information is obtained according to the first joint information, the coordinate values of the plurality of predicted feature points in the first predicted pressure information are in an image coordinate plane (such as the first coordinate plane), but the first measured pressure information is in a pressure coordinate plane (such as the second coordinate plane), and the coordinates in different coordinate planes are difficult to directly compare the pressure values, therefore, the second measured pressure information is obtained based on the first measured pressure information, and the second measured pressure information can be understood as the coordinate values of the plurality of measured feature points and the corresponding pressure values after the coordinates are unified to the image coordinate plane. In the embodiment of the present application, since the image information and the pressure information are in different coordinate planes, the predicted pressure information obtained according to the image information and the measured pressure information are in different coordinate planes, therefore, the predicted pressure information and the measured pressure information can be unified to the same coordinate plane for comparison, thereby avoiding the problem that the data in different coordinate planes are difficult to directly compare, and thus improving the user posture recognition accuracy.

[0089] For example, as shown in Figure 3D , Figure 3D is a first coordinate plane and a second coordinate plane provided by an embodiment of the present application, XOZ in the figure belongs to the first coordinate plane, and XOY belongs to the second coordinate plane, and the plurality of predicted feature points in the first predicted pressure information are on the first coordinate plane; the plurality of measured feature points in the first measured pressure information are on the second coordinate plane, therefore, the points on different coordinate planes are difficult to directly compare the pressure values, so the plurality of measured feature points on the second coordinate plane are converted to the first coordinate plane. As shown in Figure 3E and Figure 3F , Figure 3E is an image coordinate system diagram provided by an embodiment of the present application, Figure 3F is a pressure coordinate system diagram provided by an embodiment of the present application, as shown in Figure 3E , the image coordinate system is established according to the first coordinate plane, such as the coordinate in the image coordinate system is As shown in Figure 3F , the pressure coordinate system is established according to the second coordinate plane, such as the coordinate in the pressure coordinate system is The coordinates are converted from the pressure coordinate system to the image coordinate system, and optionally, the transformation matrix from the pressure coordinate system to the image coordinate system is first solved, and then the coordinates in the pressure coordinate system are multiplied by the transformation matrix to obtain the coordinates in the image coordinate system, such as A p ·M=A G . For example, the transformation matrix is as follows:

[0090]

[0091] It should be noted that four points can be found on the pressure coordinate system to substitute A p • M = A G Solving the transformation matrix, so that the point in the pressure coordinate system and the image coordinate system can be converted by the transformation matrix M.

[0092] In a possible implementation, after the first similarity of the first predicted pressure information and the first measured pressure information is calculated, the method further includes: if the first similarity is less than or equal to the second preset threshold, dividing the first measured pressure information into M first measured pressure sub-information, and dividing the first predicted pressure information into M first predicted pressure sub-information, to generate M groups of to-be-compared pressure information; wherein the M first measured pressure sub-information correspond to M parts of the target object, and the M first predicted pressure sub-information correspond to the M parts; each of the groups of to-be-compared pressure information includes the first measured pressure sub-information and the first predicted pressure sub-information corresponding to the same part of the M parts of the target object; a second similarity of the first measured pressure sub-information and the first predicted pressure sub-information in each of the groups of to-be-compared pressure information is calculated; L groups of to-be-compared pressure information whose second similarity is less than or equal to a third preset threshold are determined; L second predicted pressure sub-information are determined according to the first measured pressure sub-information in the L groups of to-be-compared pressure information respectively; and the first target attitude is generated based on the L second predicted pressure sub-information and M-L first predicted pressure sub-information in M-L groups of to-be-compared pressure information; the M-L groups of to-be-compared pressure information are groups of to-be-compared pressure information other than the L groups of to-be-compared pressure information in the M groups of to-be-compared pressure information.

[0093] Specifically, as shown in Figure 3G , Figure 3G A group of to-be-compared pressure information provided by the embodiment of the present application is shown in the figure. The first measured pressure sub-information can be understood as dividing the first measured pressure information into one of the M information according to the M parts of the target object; the first predicted pressure sub-information can be understood as dividing the first predicted pressure information into one of the M information according to the M parts of the target object; the first measured pressure sub-information and the first predicted pressure sub-information corresponding to the same part are included in each of the groups of to-be-compared pressure information. As shown in Figure 3H , Figure 3H A schematic diagram for selecting L pressure information groups from M pressure information groups is provided in the embodiment of the present application. After the M pressure information groups are generated, the second similarity of the first measured pressure sub-information and the first predicted pressure sub-information in each pressure information group is calculated, and then L pressure information groups with the second similarity lower than or equal to a preset threshold value are determined. It should be noted that if the second similarity is greater than the preset threshold value, some local postures in the target posture can be determined according to the first predicted pressure sub-information in the corresponding pressure information group. The L pressure information groups can be understood as L local postures that are incorrectly recognized based on the image recognition algorithm. As shown in Figure 3I Figure 3I A schematic diagram of second predicted pressure information is provided in the embodiment of the present application. After the L pressure information groups are determined, L first measured pressure sub-information in the L pressure information groups is taken out, and then L second predicted pressure sub-information is obtained according to the L first measured pressure sub-information. Then, the target posture at the first time is obtained according to the L second predicted pressure sub-information and the first predicted pressure sub-information in the pressure information groups other than the L pressure information groups. It should be noted that L is less than or equal to M. The L second predicted pressure sub-information obtained according to the L first measured pressure sub-information can be understood as correcting the L image recognition errors of the local postures according to the pressure information generated by the actual posture of the user. The target posture at the first time obtained according to the L second predicted pressure sub-information and the first predicted pressure sub-information in the pressure information groups other than the L pressure information groups can be understood as obtaining the target posture according to the M-L correct local posture information and the L corrected local posture information recognized by the image information.

[0094] In the embodiment of the present application, when the similarity between the measured pressure information and the predicted pressure information is less than or equal to the preset threshold value, it does not mean that the target posture obtained according to the image information is completely not close to the actual posture of the user, but only means that some local postures in the target posture obtained according to the image information are incorrectly recognized. Therefore, the measured pressure information and the predicted pressure information are divided into M parts according to the M parts of the target object, and then the information corresponding to the L incorrectly recognized local postures is found. Then, the L local postures are predicted through the pressure information, and then a target posture more close to the actual posture of the user is obtained according to the M-L predicted pressure sub-information obtained from the image information (excluding the predicted pressure sub-information corresponding to the L incorrectly recognized local postures) and the L predicted pressure sub-information obtained from the pressure information, thereby improving the posture recognition accuracy of the user.

[0095] ​In a possible implementation, the determining L second predicted pressure sub-information respectively according to the first measured pressure sub-information in the L groups of to-be-compared pressure information comprises: determining a local erroneous attitude pressure information; the local erroneous attitude pressure information comprises the first measured pressure sub-information of each of the L groups of to-be-compared pressure information; obtaining N preset local attitude information, and generating N second predicted pressure sub-information respectively based on the N preset local attitude information; calculating a third similarity between the N second predicted pressure sub-information and the local erroneous attitude pressure information respectively, to obtain a local candidate attitude information set; the local candidate attitude information set comprises preset local attitude information corresponding to one or more second predicted pressure sub-information in the N second predicted pressure sub-information and having a third similarity greater than a fourth threshold with the local erroneous attitude pressure information; dividing the first joint information into M sub-information matched with the M parts of the target object, and determining a first joint sub-information; the first joint sub-information is the joint information corresponding to the local erroneous attitude pressure information; calculating a fourth similarity between each preset local attitude information in each local candidate attitude information set and the first joint sub-information respectively, to obtain a target local attitude information; the target local attitude information is the preset local attitude information with the highest fourth similarity; and obtaining the L second predicted pressure sub-information according to the target local attitude information.

[0096] Specifically, the N preset local attitude information can be understood as preset local attitude information obtained from a database. Since the N preset local attitude information and the first measured pressure sub-information are heterogeneous information and cannot be directly fused, the N preset local attitude information is processed to obtain N second predicted pressure sub-information. Figure 3J Figure 3J A schematic diagram for determining a local candidate attitude set is provided in an embodiment of the present application. In the diagram, after determining a local erroneous attitude pressure information according to the first measured pressure sub-information of each of the L groups of to-be-compared pressure information, N preset local attitude information is obtained. Optionally, the N preset local attitude information is processed to obtain N second predicted pressure sub-information according to a limb quality physiological model (such as a model constructed based on anthropometry and physical principles). Further, the N second predicted pressure sub-information is compared with the local erroneous attitude pressure information respectively, so as to find a local candidate attitude information set. The local candidate attitude information set can be understood as a set of information of some possible local candidate attitudes found for the erroneous local combined pressure information. As shown in Figure 3K Figure 3K ​​An embodiment of the present application provides a schematic diagram for determining L second predicted pressure sub-information. In the diagram, the local candidate posture set comprises one or more candidate posture information (such as one or more preset local posture information in N preset local posture information). Further, the first joint node information (information obtained according to image information) is divided into M sub-information matched with M parts of the target object, the first joint node sub-information is obtained for the part with recognition error, each local candidate posture information in the local candidate posture set is compared with the first joint node sub-information, the local candidate posture information with the highest similarity is found, and then the L second predicted pressure information corresponding to the local candidate posture information can be determined. The first joint node sub-information can be understood as the joint node information of the part with recognition error; it should be noted that even if the picture information of the local posture with recognition error has a certain gap with the actual posture information, it still has reference value in predicting the local posture. It can be understood that the L second predicted pressure information is information selected from N second predicted pressure information according to pressure information and image information.

[0097] In the embodiment of the present application, after the information corresponding to the L local postures with recognition error is found in the above method, the local error posture pressure information is obtained according to the measured pressure sub-information in the L local postures, then the N preset local posture information is obtained, the preset local posture information is further converted into predicted pressure sub-information and compared with the local error posture pressure information, the corresponding local candidate posture information set is determined, and then the information in the local candidate posture information set and the joint node information obtained from the image information are combined to determine the local candidate posture information set closest to the real local posture of the user, so that the posture recognition accuracy of the user is improved.

[0098] Embodiment two: obtaining the target posture of the target object based on the pressure information and the image information.

[0099] In a possible implementation, the first target posture of the target object at the first time is identified based on the first joint node coordinate set and the first pressure value set; the method comprises: obtaining S preset target posture information, and respectively processing the S preset target posture information to generate S second predicted pressure information; respectively calculating fifth similarity between the S second predicted pressure information and the first measured pressure information, and adding the second predicted pressure information with the fifth similarity greater than a fifth preset threshold to a target candidate posture set; respectively calculating sixth similarity between the predicted target posture information in each target candidate posture set and the first joint node information, obtaining the predicted target posture information with the highest sixth similarity; and generating the first target posture based on the predicted target posture information with the highest sixth similarity.

[0100] Specifically, the S preset target posture information can be understood as preset target posture information obtained from a database. Since the S preset target posture information and the first measured pressure information are heterogeneous information and cannot be directly fused, as shown in Figure 3L , Figure 3L An example of determining a target candidate posture set provided by the embodiment of the present application is shown in the figure. Optionally, S second predicted pressure information is obtained by processing S preset target posture information according to a limb quality physiological model (such as a model constructed according to principles of anthropometry and physics). Further, N second predicted pressure information is compared with the first measured pressure information, respectively, to find a target candidate posture set. The target candidate posture set can be understood as a candidate posture information set selected from the S preset target posture information and close to the actual posture of the user for the target object. Further, the candidate posture information in the target candidate posture set is compared with the first joint information to find the candidate posture information with the highest similarity to the first joint information. Then, the target posture can be obtained according to the candidate posture information. In the embodiment of the present application, in the case that the target posture obtained according to the image information is greatly different from the actual posture of the user, the posture of the user can be predicted based on the pressure information and in combination with the image information. Specifically, S preset target posture information can be obtained first. Then, the preset target posture information is converted into predicted target posture information and compared with the measured pressure information to determine a corresponding target candidate posture set. Then, the predicted target posture information in the target candidate posture set is compared with the predicted pressure information obtained from the image information to determine the predicted target posture information with the highest similarity in the target candidate posture set. Then, the target posture is recognized according to the predicted target posture information, thereby improving the posture recognition accuracy of the user.

[0101] In a possible implementation, when the target candidate posture set includes j predicted target posture information, before the sixth similarity of the predicted target posture information in each target candidate posture set to the first joint information is calculated, the method further includes: comparing the j predicted target posture information in the target candidate posture set with second joint information corresponding to a second target posture output at a previous time of the first time, and removing the predicted target posture information with a similarity less than or equal to a seventh preset threshold from the target candidate posture set.

[0102] Specifically, it can be understood that the second target pose is the target pose output at the previous moment of the first moment. In this embodiment of the invention, when there are many predicted target pose information in the target candidate pose set under the above method, multiple predicted target pose information needs to be compared with the measured pressure information separately, resulting in a large amount of computation. Therefore, by combining the target pose output by the target object at the previous moment, the predicted target pose information in the target candidate pose set is filtered, and predicted target pose information with too low similarity is eliminated, reducing the number of predicted target pose information that needs to be compared, thereby reducing computational overhead and improving efficiency. For example, when there is a lot of limb overlap and the pressure distribution is dense, the target candidate pose set {POSE}... t The predicted target pose information in the previous time step is too large, resulting in excessive computational overhead. Therefore, it is necessary to combine the target pose POSE from the previous time step. t-1 Calculate the current moment; optionally, compare with POSE. t-1 With {POSE t Each POSE in} t Calculate the distance difference {D} between each joint. i According to the following formula:

[0103] |POSE P -POSE t-1 |>α·{D i }

[0104] Remove predicted target pose information from the target candidate pose set that has a large difference from the target pose at the previous time step.

[0105] In one possible implementation, the method further includes: calculating a seventh similarity between the first target pose and the target pose at adjacent time points; if the seventh similarity is less than or equal to an eighth preset threshold, then correcting the first target pose based on the target pose at the adjacent time points.

[0106] Specifically, the target pose at adjacent time points can be understood as the target pose at the time point before the first time point and the target pose at the time point after the first time point, or it can be understood as multiple target poses within a time period adjacent to the first time point. In this embodiment of the invention, if a pose recognition error occurs under abnormal circumstances, the erroneous pose can be corrected based on the target pose output at the time point adjacent to the erroneous pose, thereby improving the accuracy of user pose recognition.

[0107] In the embodiment of the present application, the user posture can be recognized by combining image information and pressure information. For the body overlap or body occlusion part, the prediction can be made based on the image information combined with the pressure information or based on the pressure information combined with the image information, which can avoid the problem that the effective information about the overlap or occlusion part is less when the body overlap or body occlusion part is predicted only by the image information, and the prediction result is greatly different from the actual result. Specifically, the image of the user is collected by the camera, the pressure sensing data is obtained by the pressure pad, and the corresponding image information and pressure information are generated according to the image and the pressure sensing data, and then the target posture of the user is recognized according to the image information and the pressure information. Since the pressure information is obtained according to the pressure sensing data of the actual action of the user, when the body overlap or body occlusion part is predicted, the user posture can be more accurately predicted, thereby improving the posture recognition accuracy.

[0108] To describe the posture recognition method in the embodiment of the present application in more detail, the posture recognition system architecture is described below. Figure 4A Figure 4A An example flowchart of the posture recognition method provided in the embodiment of the present application is shown in the following.

[0109] 1. Image recognition of the pressure pad. The position of the pressure pad (image space) is detected by the image. This step can use the existing technology to detect the position of the pressure pad, such as a target detection algorithm.

[0110] 2. Position calibration of the pressure pad. The position of the pressure pad is calibrated, and the pressure coordinate system and the image coordinate system are unified.

[0111] Optionally, the preset pressure pad placement area is marked on the display device to guide the user to place the pressure pad at the preset position, so as to facilitate the conversion of the point on the pressure coordinate system to the image coordinate system.

[0112] 3. Time synchronization. The image data and the pressure pad data are time-synchronized. Since the sampling frequencies of the image data and the pressure pad data are different, in one implementation, the high-frequency data can be down-sampled to make the sampling frequencies consistent. In another implementation, the high-frequency data is only obtained when the low-frequency data is refreshed, and the high-frequency data at other times is directly discarded and does not participate in the calculation.

[0113] 4. Image recognition of the skeleton node. Based on the image information, the image recognition algorithm is used to recognize all the joint node coordinates of the user n is the number of joint nodes, and the value of n is different according to different image recognition algorithms, such as 10, 14, 15, 30, etc.

[0114] ​5、Pressure detection. Obtain pressure data F through pressure sensing device, in one implementation, the pressure data can be two-dimensional matrix information [F], mark the pressure value F of each position ij .

[0115] 6、Preprocessing. Optionally, pre-process the pressure data, normalize the pressure data, and clear the pressure values below the threshold T (for example, T can be 0.2). Then only the local area with larger pressure is left in the pressure data, which can be understood as the area directly contacting the pressure pad.

[0116] 7、Body inference based on body mass physiological model, pressure distribution model and pressure data. For a given arbitrary pose POSE (defined by a set of joint coordinates ), the mass distribution of each body part can be calculated according to the knowledge of anthropometry, and the pressure distribution F' on the pressure pad can be calculated according to the pose of each body part. For the current pressure distribution F, find the set of poses POSE that satisfy F' ≈ F (the similarity is higher than a threshold), as the candidate pose set {POSE}.

[0117] 8、Find the optimal match by combining body inference information and image-recognized skeleton node information. Transform the candidate pose set {POSE} to the image coordinate system through the transformation matrix M, to obtain {POSE G}, and compare {POSE G} with the image information-based Select the POSE G with the highest similarity S (which can be calculated by the cosine distance method) as the target pose.

[0118] In the embodiments of the present application, the user's pose can be recognized by combining image information and pressure information. For the body overlapping or body occlusion part, the pressure information can be combined with the image information or the image information can be combined with the pressure information based on the image information, which can avoid the problem that the effective information about the overlapping or occlusion part is less when only the image information is used to predict the body overlapping or body occlusion part, resulting in a large difference between the prediction result and the actual result.

[0119] To describe the pose recognition method in the embodiments of the present application in more detail, the system architecture based on the above-mentioned pose recognition will be described Figure 4B exemplarily, Figure 4B and another exemplary flowchart of the pose recognition method provided by the embodiments of the present application is shown in FIG. 2.

[0120] 1、Image recognition of pressure pad. The position of the pressure pad (image space) is detected through the image, and this step can use the existing technology to detect the position of the pressure pad, such as target detection algorithm.

[0121] 2. Calibration of the position of the pressure mat. The position of the pressure mat is calibrated, and the pressure coordinate system and the image coordinate system are unified.

[0122] Optionally, a preset pressure mat placement area is marked on the display device, guiding the user to place the pressure mat at the preset position, facilitating the conversion of points on the pressure coordinate system to the image coordinate system.

[0123] 3. Time synchronization. The image data and the pressure mat data are time synchronized. Since the sampling frequencies of the image data and the pressure mat data are different, in one implementation scheme, the high-frequency data can be down-sampled to make the sampling frequencies consistent; in another scheme, only when the low-frequency data is refreshed, the high-frequency data is acquired, and the high-frequency data at other times is directly discarded and does not participate in the calculation.

[0124] 4. Image recognition of the skeleton nodes. Based on the image information, an image recognition algorithm is used to recognize the coordinates of all the joint nodes of the user n is the number of the joint nodes, and the value of n is different according to different image recognition algorithms, such as 10, 14, 15, 30, etc.

[0125] 5. Pressure estimation based on the limb mass physiological model, the pressure distribution model, and the image-recognized skeleton node information. The pressure distribution F is calculated in combination with anthropometric information and physical principles G , and is normalized. Meanwhile, the contact limb part corresponding to each pressure value is recorded;

[0126] 6. Pressure detection. The pressure data F is acquired through the pressure sensing device. In one implementation, the pressure data can be two-dimensional matrix information [F], marking the pressure values F of each position ij .

[0127] 7. Preprocessing. Optionally, the pressure data is preprocessed, the pressure data is normalized, and the pressure values below the threshold T (for example, T can be 0.2) are cleared 0. Then, only the local area with larger pressure remains in the pressure data, which can be understood as the area where the limb directly contacts the pressure mat.

[0128] 8. Comparison of F G and F. If the similarity is high enough (for example, 0.9), F is output as the correct pose.

[0129] 9. Pose correction. If the similarity of F G and F is lower than the threshold (for example, 0.9), F G is divided into several blocks according to the spatial position, the similarity of each block is compared, and the block with lower similarity is selected, and the corresponding limb part is the limb with recognition error. For a given arbitrary local pose pose (a set of local joint node coordinates According to the knowledge of anthropometry, the mass distribution of each limb segment can be calculated, and the pressure distribution pattern F1' on the pressure mat can be calculated according to the posture of each limb. For the current pressure distribution F, find the set of local postures pose for which all F1' ≈ F (the similarity is higher than a threshold value), as the local candidate posture set {pose}. Find the optimal match by combining the limb inference information and the image-recognized skeleton node information. Transform the local candidate posture set {pose} to the image coordinate system through the transformation matrix M, to obtain {pose G}. Compare {pose G} with the image-based Select the pose G with the highest similarity (which can be calculated by the cosine distance method, etc.) as the target local posture.

[0130] 10. Replace the recognized incorrect nodes with the error-corrected node positions, as the final result.

[0131] In the embodiment of the present application, the measurement pressure information and the image information at the same time are heterogeneous information and cannot be directly fused. Optionally, the image information is processed to obtain predicted pressure information according to the principles of anthropometry and physics. The measurement pressure information and the predicted pressure information are further compared. If the similarity between the two is greater than a preset threshold, the target posture of the user can be obtained according to the image information. When the similarity between the measurement pressure information and the predicted pressure information is less than or equal to the preset threshold, it does not mean that the target posture obtained according to the image information is completely not close to the actual posture of the user, but only means that some local postures in the target posture obtained according to the image information are recognized incorrectly. Therefore, the measurement pressure information and the predicted pressure information are both divided into M parts according to the M parts of the target object, and then the information corresponding to the L incorrectly recognized local postures is found. Then, the L local postures are predicted through the pressure information, and a target posture closer to the actual posture of the user is obtained according to the M-L predicted pressure sub-information obtained from the image information (excluding the predicted pressure sub-information corresponding to the L incorrectly recognized local postures) and the L predicted pressure sub-information obtained from the pressure information, thereby improving the posture recognition accuracy of the user.

[0132] To describe the posture recognition method in the embodiment of the present application in more detail, the posture recognition system architecture described above will be used for an exemplary description, Figure 4C in combination with Figure 4C , which is an exemplary flowchart of the posture recognition method provided in the embodiment of the present application.

[0133] 1. Image recognition of the pressure mat. The position of the pressure mat (image space) is detected through the image. This step can use existing technologies such as target detection algorithms to detect the position of the pressure mat.​

[0134] 2. Pressure mat position calibration. The position of the pressure mat is calibrated to unify the pressure coordinate system and the image coordinate system.

[0135] Optionally, a preset pressure mat placement area is marked on the display device to guide the user to place the pressure mat at the preset position, facilitating the conversion of points on the pressure coordinate system to the image coordinate system.

[0136] 3. Time synchronization. The image data and the pressure mat data are time-synchronized. Since the sampling frequencies of the image data and the pressure mat data are different, in one implementation, the high-frequency data can be down-sampled to make the sampling frequencies consistent; in another implementation, the high-frequency data is acquired only when the low-frequency data is refreshed, and the high-frequency data at other times is directly discarded and does not participate in the calculation.

[0137] 4. Image recognition of skeletal joints. Based on the image information, an image recognition algorithm is used to recognize the coordinates of all the joints of the user n is the number of joints, and the value of n is different for different image recognition algorithms, such as 10, 14, 15, 30, etc.

[0138] 5. Pressure detection. The pressure data F is acquired through the pressure sensing device. In one implementation, the pressure data can be two-dimensional matrix information [F], marking the pressure values F of each position ij .

[0139] 6. Preprocessing. Optionally, the pressure data is preprocessed, and the pressure data is normalized, and the pressure values below the threshold T (for example, T can be 0.2) are cleared 0. Then only the local area with larger pressure is left in the pressure data, which can be understood as the area where the limb directly contacts the pressure mat.

[0140] 7. Limb inference based on the physiological model of limb mass, the pressure distribution model, and the pressure data. For a given arbitrary pose POSE (defined by a set of joint coordinates ), the mass distribution of each limb can be calculated according to the knowledge of anthropometry, and the pressure distribution pattern F' on the pressure mat can be calculated according to the pose of each limb. For the current pressure distribution F, find the set of poses POSE that satisfy F' ≈ F (the similarity is higher than the threshold), as the candidate pose set {POSE}.

[0141] 8. When the limbs overlap a lot and the pressure distribution is dense, the candidate pose set {POSE} is large, and the calculation overhead is too large, at this time, the output pose POSE t-1 of the previous moment can be combined to infer the current moment.

[0142] Optionally, the pose POSE t-1 is compared with Calculate the distance difference {D} of each joint point i} when selecting the candidate set {POSE}, only select those that meet:

[0143] |POSE P -POSE t-1 |<α·{D i}

[0144] Put the pose P into the candidate set and compare.

[0145] 9, transform the candidate pose set {POSE} to the image coordinate system through the transformation matrix M, get {POSE G}, compare {POSE G} with select the pose G with the highest similarity S (which can be calculated by cosine distance method) as the current pose.

[0146] In the embodiment of the application, the user's posture can be recognized in combination with image information and pressure information. For the part with limb overlap or limb occlusion, the prediction can be made on the basis of image information in combination with pressure information or on the basis of pressure information in combination with image information, but when there are many predicted target posture information in the target candidate posture set obtained on the basis of pressure information, the multiple predicted target posture information needs to be compared with the measured pressure information respectively, and the calculation amount is large. Therefore, in combination with the target posture output at the previous moment of the target object, the predicted target posture information in the target candidate posture set is screened, the predicted target posture information with too low similarity is eliminated, the number of predicted target posture information that needs to be compared is reduced, thereby reducing the calculation cost and improving the efficiency.

[0147] Please refer to Figure 5 , Figure 5 The application provides a posture recognition device schematic diagram provided by the embodiment of the application, the posture recognition device 50 can include: a first receiving unit 501, a first processing unit 502, a second receiving unit 503, a second processing unit 504, a third processing unit 505, a fourth processing unit 506, a fifth processing unit 507, a sixth processing unit 508, and the detailed description of each module is as follows.

[0148] The first receiving unit 501 is used for collecting the first image of the target object at the first moment through the camera;

[0149] The first processing unit 502 is used for determining the first joint point information corresponding to the target object at the first moment according to the first image, and the first joint point information includes the coordinate values of a plurality of preset joint points of the target object on the first coordinate plane;

[0150] The second receiving unit 503 is configured to collect pressure sensing data of the target object at the first time point by the pressure sensing mat.

[0151] The second processing unit 504 is configured to determine first measured pressure information of the target object at the first time point according to the pressure sensing data, the first measured pressure information including coordinate values of a plurality of measured feature points of the target object on a second coordinate plane and corresponding pressure values.

[0152] The third processing unit 505 is configured to identify a first target pose of the target object at the first time point based on the first joint information and the first measured pressure information.

[0153] In a possible implementation, the second processing unit 504 is specifically configured to: perform normalization processing on the pressure sensing data, and clear pressure sensing data lower than a first preset threshold in the normalized pressure sensing data to obtain the first measured pressure information.

[0154] In a possible implementation, the third processing unit 505 is specifically configured to: generate first predicted pressure information according to the first joint information, the first predicted pressure information including coordinate values of a plurality of predicted feature points matched with the first joint information on the first coordinate plane and corresponding pressure values; calculate a first similarity between the first predicted pressure information and the first measured pressure information; and if the first similarity is greater than a second preset threshold, generate the first target pose based on the first joint information.

[0155] In a possible implementation, the third processing unit 505 is specifically configured to: process the coordinate values of the plurality of measured feature points on the second coordinate plane in the first measured pressure information to obtain second measured pressure information; the second measured pressure information including corresponding coordinate values of the plurality of measured feature points mapped on the first coordinate plane and corresponding pressure values; and calculate the first similarity between the first predicted pressure information and the first measured pressure information based on the second measured pressure information and the first predicted pressure information.

[0156] In a possible implementation, the device further includes: a fourth processing unit 506, configured to: if the first similarity is less than or equal to the second preset threshold, divide the first measured pressure information into M first measured pressure sub-information, and divide the first predicted pressure information into M first predicted pressure sub-information, to generate M groups of to-be-compared pressure information; the M first measured pressure sub-information correspond to M parts of the target object, and the M first predicted pressure sub-information correspond to the M parts; each of the groups of to-be-compared pressure information includes the first measured pressure sub-information and the first predicted pressure sub-information corresponding to the same part of the M parts of the target object; the fourth processing unit is configured to: calculate a second similarity of the first measured pressure sub-information and the first predicted pressure sub-information in each of the groups of to-be-compared pressure information; the fourth processing unit is further configured to: determine L groups of to-be-compared pressure information whose second similarity is less than or equal to a third preset threshold; the fourth processing unit is further configured to: determine L second predicted pressure sub-information according to the first measured pressure sub-information in the L groups of to-be-compared pressure information respectively; and generate the first target attitude based on the L second predicted pressure sub-information and M-L first predicted pressure sub-information in M-L groups of to-be-compared pressure information; the M-L groups of to-be-compared pressure information are groups of to-be-compared pressure information other than the L groups of to-be-compared pressure information in the M groups of to-be-compared pressure information.

[0157] In a possible implementation, the fourth processing unit 506 is specifically configured to: determine local error posture pressure information; the local error posture pressure information includes the first measurement pressure sub-information of each of the L groups of to-be-compared pressure information; obtain N preset local posture information, and generate N second predicted pressure sub-information based on the N preset local posture information respectively; calculate third similarity of the N second predicted pressure sub-information and the local error posture pressure information respectively, to obtain a local candidate posture information set; the local candidate posture information set includes one or more second predicted pressure sub-information corresponding to the preset local posture information whose third similarity with the local error posture pressure information is greater than a fourth threshold; divide the first joint information into M sub-information matched with the M parts of the target object, and determine a first joint sub-information; the first joint sub-information is the joint information corresponding to the local error posture pressure information; calculate fourth similarity between each preset local posture information in each local candidate posture information set and the first joint sub-information respectively, to obtain a target local posture information; the target local posture information is the preset local posture information with the highest fourth similarity; obtain the L second predicted pressure sub-information according to the target local posture information.

[0158] In a possible implementation, the fourth processing unit 506 is specifically configured to: obtain S preset target posture information, and generate S second predicted pressure information based on the S preset target posture information respectively; calculate fifth similarity of the S second predicted pressure information and the first measurement pressure information respectively, and add the second predicted pressure information with the fifth similarity greater than a fifth preset threshold to a target candidate posture set; calculate sixth similarity between the predicted target posture information in each target candidate posture set and the first joint information respectively, to obtain the predicted target posture information with the highest sixth similarity; generate the first target posture based on the predicted target posture information with the highest sixth similarity.

[0159] In a possible implementation, when the target candidate posture set includes j predicted target posture information, the device further includes a fifth processing unit 507 configured to compare the j predicted target posture information in the target candidate posture set with second joint information corresponding to a second target posture output at a previous time of the first time, and remove the predicted target posture information with a similarity less than or equal to a seventh preset threshold from the target candidate posture set.

[0160] In a possible implementation, the device further includes a sixth processing unit 508 configured to calculate a seventh similarity between the first target pose and a target pose at a previous time; and if the seventh similarity is less than or equal to an eighth preset threshold, correct the first target pose according to the target pose at the previous time.

[0161] It should be noted that the functions of the pose recognition device 50 described in the embodiments of the present application can be seen from the above Figure 2 The related descriptions of steps S201-S205 in the method embodiments are not repeated here.

[0162] The embodiments of the present application provide a chip system, characterized in that the chip system includes at least one processor, a memory and an interface circuit, the memory, the interface circuit and the at least one processor are interconnected through a circuit, and the at least one memory stores instructions; when the instructions are executed by the processor, any one of the above-mentioned pose recognition methods is implemented.

[0163] The embodiments of the present application provide a computer storage medium, characterized in that the computer storage medium stores a computer program, and the computer program is executed by a processor to implement any one of the above-mentioned pose recognition methods.

[0164] The embodiments of the present application provide a computer program, characterized in that the computer program includes instructions, and when the computer program is executed by a computer, the computer executes any one of the above-mentioned pose recognition methods.

[0165] The present application provides a terminal device having the function of implementing any one of the above-mentioned pose recognition methods. The function can be realized by hardware, or realized by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above-mentioned functions.

[0166] The present application provides a smart device having the function of implementing any one of the above-mentioned pose recognition methods. The function can be realized by hardware, or realized by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above-mentioned functions.

[0167] The present application provides a terminal device including a processor, and the processor is configured to support the terminal device to execute the corresponding functions in any one of the above-mentioned pose recognition methods. The terminal device can further include a memory coupled with the processor, which saves necessary program instructions and data of the terminal device. The terminal device can further include a communication interface for communication between the terminal device and other devices or communication networks.

[0168] In the above described embodiments, the description of each embodiment is focused on a certain aspect, and the parts not described in detail in a certain embodiment can be referred to the relevant description of other embodiments.

[0169] It should be noted that, for the foregoing method embodiments, in order to describe the embodiments simply and concisely, the foregoing method embodiments are described as a series of action combinations, but those skilled in the art should know that the present application is not limited to the order of the actions described, because according to the present application, some steps can be performed in other order or at the same time. In addition, those skilled in the art should know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0170] In several embodiments provided by the present application, it should be understood that the disclosed apparatus can be implemented in other manners. For example, the embodiments of the apparatus described above are merely schematic; the division of the units is merely logical function division; an actual implementation can be another division manner, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections can be indirect couplings or communication connections through some interfaces, devices or units, and can be in electrical, mechanical or other forms.

[0171] The units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, that is, can be located in one place, or can be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiments.

[0172] In addition, each functional unit in the embodiments of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be implemented in the form of hardware, or in the form of software functional units.

[0173] The integrated unit described above, if realized in the form of a software function unit and sold or used as an independent product, can be stored in a computer-readable storage medium. Based on such understanding, the technical solutions of the present application essentially or say the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server or a network device, etc., and specifically can be a processor in the computer device) to execute all or part of the steps of the above-mentioned methods of various embodiments of the present application. Among them, the aforementioned storage medium can include: a U disk, a mobile hard disk, a magnetic disk, an optical disk, a read-only memory (Read-Only Memory, abbreviated as: ROM) or a random access memory (Random Access Memory, abbreviated as: RAM) and various program code storage media.

[0174] The above-described embodiments are only used to illustrate the technical solutions of the present application, not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.< / canvas> < / video> < / videoview> < / imgview> < / textview>

Claims

1. A method of gesture recognition, the method comprising: The method comprises: acquiring a first image of a target object at a first time point through a camera; determining first joint information corresponding to the target object at the first time point according to the first image, the first joint information comprising coordinate values of a plurality of preset joints of the target object on a first coordinate plane; acquiring pressure sensing data of the target object at the first time point through a pressure sensing pad; determining first measured pressure information of the target object at the first time point according to the pressure sensing data, the first measured pressure information comprising coordinate values of a plurality of measured feature points of the target object on a second coordinate plane and corresponding pressure values; identifying a first target pose of the target object at the first time point based on the first joint information and the first measured pressure information; calculating a seventh similarity between the first target pose and a target pose at a neighboring time point; if the seventh similarity is less than or equal to an eighth preset threshold, correcting the first target pose according to the target pose at the neighboring time point; the identifying the first target pose of the target object at the first time point based on the first joint information and the first measured pressure information comprises: generating first predicted pressure information according to the first joint information, the first predicted pressure information comprising coordinate values of a plurality of predicted feature points matched with the first joint information on the first coordinate plane and corresponding pressure values; calculating a first similarity between the first predicted pressure information and the first measured pressure information; if the first similarity is greater than a second preset threshold, generating the first target pose based on the first joint information.

2. The method of claim 1, wherein, the determining the first measured pressure information of the target object at the first time point according to the pressure sensing data comprises: normalizing the pressure sensing data, and clearing pressure sensing data lower than a first preset threshold in the normalized pressure sensing data to obtain the first measured pressure information.

3. The method of claim 1, wherein, the calculating the first similarity between the first predicted pressure information and the first measured pressure information comprises: processing coordinate values of the plurality of measured feature points in the first measured pressure information on the second coordinate plane to obtain second measured pressure information, the second measured pressure information comprising corresponding coordinate values and corresponding pressure values of the plurality of measured feature points mapped on the first coordinate plane; calculating the first similarity between the first predicted pressure information and the first measured pressure information based on the second measured pressure information and the first predicted pressure information.

4. The method of claim 2, wherein, after the calculating the first similarity between the first predicted pressure information and the first measured pressure information, the method further comprises: If the first similarity is less than or equal to the second preset threshold, the first measured pressure information is divided into M first measured pressure sub-information, and the first predicted pressure information is divided into M first predicted pressure sub-information, and M groups of to-be-compared pressure information are generated; wherein the M first measured pressure sub-information correspond to M parts of the target object, and the M first predicted pressure sub-information correspond to the M parts; each of the to-be-compared pressure information groups includes the first measured pressure sub-information and the first predicted pressure sub-information corresponding to the same part of the M parts of the target object; A second similarity of the first measured pressure sub-information and the first predicted pressure sub-information in each of the to-be-compared pressure information groups is calculated; L to-be-compared pressure information groups with a second similarity lower than or equal to a third preset threshold are determined; L second predicted pressure sub-information is determined according to the first measured pressure sub-information in the L to-be-compared pressure information groups respectively; The first target attitude is generated based on the L second predicted pressure sub-information and M-L first predicted pressure sub-information in M-L to-be-compared pressure information groups; the M-L to-be-compared pressure information groups are to-be-compared pressure information groups other than the L to-be-compared pressure information groups in the M to-be-compared pressure information groups.

5. The method of claim 4, wherein, The L second predicted pressure sub-information determined according to the first measured pressure sub-information in the L to-be-compared pressure information groups respectively includes: Local error attitude pressure information is determined; the local error attitude pressure information includes the first measured pressure sub-information of each of the L to-be-compared pressure information groups; N preset local attitude information is obtained, and N second predicted pressure sub-information is generated based on the N preset local attitude information respectively; A third similarity of the N second predicted pressure sub-information and the local error attitude pressure information is calculated respectively, and a local candidate attitude information set is obtained; the local candidate attitude information set includes preset local attitude information corresponding to one or more second predicted pressure sub-information in the N second predicted pressure sub-information with a third similarity greater than a fourth threshold value with the local error attitude pressure information; The first joint information is divided into M sub-information matched with the M parts of the target object, and a first joint sub-information is determined; the first joint sub-information is the joint information corresponding to the local error attitude pressure information; A fourth similarity between each of the preset local attitude information in each of the local candidate attitude information set and the first joint sub-information is calculated respectively, and a target local attitude information is obtained; the target local attitude information is the preset local attitude information with the highest fourth similarity; The L second predicted pressure sub-information is obtained according to the target local attitude information.

6. The method of claim 1 or 2, wherein, The first target attitude of the target object at the first time is identified based on the first joint information and the first measured pressure information. The method includes: S preset target posture information is acquired, and S second prediction pressure information is generated by processing the S preset target posture information respectively based on the S preset target posture information; Fifth similarity between the S second prediction pressure information and the first measurement pressure information is calculated respectively, and the second prediction pressure information with the fifth similarity greater than a fifth preset threshold is added to a target candidate posture set; Sixth similarity between the preset target posture information in each target candidate posture set and the first joint node information is calculated respectively, and the preset target posture information with the highest sixth similarity is obtained; The first target posture is generated based on the preset target posture information with the highest sixth similarity.

7. The method of claim 6, wherein, When the target candidate posture set includes j preset target posture information, before the sixth similarity between the preset target posture information in each target candidate posture set and the first joint node information is calculated respectively, the method further includes: The j preset target posture information in the target candidate posture set is compared with second joint node information corresponding to a second target posture output at a previous time of the first time, and the preset target posture information with a similarity less than or equal to a seventh preset threshold is removed from the target candidate posture set.

8. A device for gesture recognition, characterized in that The device includes: A first receiving unit is configured to acquire a first image of a target object at a first time through a camera; A first processing unit is configured to determine first joint node information corresponding to the target object at the first time according to the first image, the first joint node information including coordinate values of a plurality of preset joint nodes of the target object on a first coordinate plane; A second receiving unit is configured to acquire pressure sensing data of the target object at the first time through a pressure sensing mat; A second processing unit is configured to determine first measurement pressure information of the target object at the first time according to the pressure sensing data, the first measurement pressure information including coordinate values of a plurality of measurement feature points of the target object on a second coordinate plane and corresponding pressure values; A third processing unit is configured to identify a first target posture of the target object at the first time based on the first joint node information and the first measurement pressure information; A sixth processing unit is configured to calculate a seventh similarity between the first target posture and a target posture at an adjacent time; and if the seventh similarity is less than or equal to an eighth preset threshold, the first target posture is corrected according to the target posture at the adjacent time. The third processing unit is specifically configured to: generate first prediction pressure information according to the first joint node information, the first prediction pressure information including coordinate values of a plurality of prediction feature points matched with the first joint node information on the first coordinate plane and corresponding pressure values; calculate a first similarity between the first prediction pressure information and the first measurement pressure information; if the first similarity is greater than a second preset threshold, generate the first target posture based on the first joint node information.

9. The apparatus of claim 8, wherein, The second processing unit is specifically configured to: The pressure sensing data is normalized, and the pressure sensing data lower than a first preset threshold is cleared to obtain the first measured pressure information.

10. The apparatus of claim 9, wherein, The third processing unit is specifically configured to: process the coordinate values of the plurality of measurement feature points in the first measured pressure information on the second coordinate plane to obtain second measured pressure information; the second measured pressure information includes corresponding coordinate values and corresponding pressure values of the plurality of measurement feature points mapped on the first coordinate plane; based on the second measured pressure information and the first predicted pressure information, calculate the first similarity between the first predicted pressure information and the first measured pressure information.

11. The apparatus of claim 10, wherein, The device further includes: The fourth processing unit is configured to divide the first measured pressure information into M first measured pressure sub-information and divide the first predicted pressure information into M first predicted pressure sub-information to generate M groups of to-be-compared pressure information if the first similarity is less than or equal to the second preset threshold; wherein the M first measured pressure sub-information corresponds to M parts of the target object, and the M first predicted pressure sub-information corresponds to the M parts; each of the to-be-compared pressure information groups includes the first measured pressure sub-information and the first predicted pressure sub-information corresponding to the same part of the M parts of the target object; The fourth processing unit is configured to calculate the second similarity between the first measured pressure sub-information and the first predicted pressure sub-information in each of the to-be-compared pressure information groups. The fourth processing unit is further configured to determine L to-be-compared pressure information groups whose second similarity is less than or equal to a third preset threshold. The fourth processing unit is further configured to determine L second predicted pressure sub-information according to the first measured pressure sub-information in the L to-be-compared pressure information groups, respectively. The fourth processing unit is further configured to generate the first target attitude based on the L second predicted pressure sub-information and M-L first predicted pressure sub-information in M-L to-be-compared pressure information groups; the M-L to-be-compared pressure information groups are to-be-compared pressure information groups other than the L to-be-compared pressure information groups in the M to-be-compared pressure information groups.

12. The apparatus of claim 11, wherein, The fourth processing unit is specifically configured to: determine a local error attitude pressure information; the local error attitude pressure information includes the first measured pressure sub-information of each of the L to-be-compared pressure information groups; obtain N preset local attitude information, and process the N preset local attitude information to generate N second predicted pressure sub-information, respectively; calculate the third similarity between the N second predicted pressure sub-information and the local error attitude pressure information to obtain a local candidate attitude information set; the local candidate attitude information set includes preset local attitude information corresponding to one or more second predicted pressure sub-information in the N second predicted pressure sub-information whose third similarity with the local error attitude pressure information is greater than a fourth threshold. The first joint information is divided into M pieces of sub-information matched with the M parts of the target object, and first joint sub-information is determined; the first joint sub-information is joint information corresponding to the local error posture pressure information; A fourth similarity between each preset local posture information in each of the local candidate posture information sets and the first joint sub-information is calculated respectively, and target local posture information is obtained; the target local posture information is the preset local posture information with the highest fourth similarity; The L pieces of second predicted pressure sub-information are obtained according to the target local posture information.

13. The apparatus of claim 8 or 9, wherein, The third processing unit is specifically configured to: S pieces of preset target posture information are obtained, and S pieces of second predicted pressure information are generated based on the S pieces of preset target posture information respectively; A fifth similarity between the S pieces of second predicted pressure information and the first measured pressure information is calculated respectively, and the second predicted pressure information with a fifth similarity greater than a fifth preset threshold is added to a target candidate posture set; A sixth similarity between the preset target posture information in each of the target candidate posture sets and the first joint information is calculated respectively, and the preset target posture information with the highest sixth similarity is obtained; The first target posture is generated based on the preset target posture information with the highest sixth similarity.

14. The apparatus of claim 13, wherein, When the target candidate posture set includes j pieces of preset target posture information, the device further includes: A fifth processing unit configured to compare the j pieces of preset target posture information in the target candidate posture set with second joint information corresponding to a second target posture output at a previous time of the first time, and remove the preset target posture information with a similarity less than or equal to a seventh preset threshold from the target candidate posture set.

15. A chip system, characterized by The chip system includes at least one processor, a memory and an interface circuit, the memory, the interface circuit and the at least one processor are interconnected through a circuit, and the at least one memory stores instructions; when the instructions are executed by the processor, the method in any one of claims 1-7 is implemented.

16. A computer storage medium, comprising, The computer storage medium stores a computer program, and the computer program is executed by the processor to implement the method in any one of claims 1-7.

17. A computer program, characterized in that, The computer program includes instructions, and when the computer program is executed by a computer, the computer executes the method in any one of claims 1-7.

Images