Posture recognition method and device and computer readable storage medium

Abstract

The embodiment of the invention discloses a posture recognition method and device and a computer readable storage medium. The method comprises the following steps: acquiring an object attitude image of a target object, and extracting attitude features in the object attitude image; obtaining pressure distribution information generated by the target object based on the object attitude image, and performing feature extraction on the pressure distribution information to obtain pressure distribution features; determining a first weight corresponding to the attitude feature and a second weight corresponding to the pressure distribution feature; and performing attitude recognition processing on the target object based on the first weight, the second weight, the attitude feature and the pressure distribution feature to obtain an attitude recognition result of the target object. In this way, the posture of the user can be efficiently and accurately recognized by combining the visual features and the pressure distribution information corresponding to the target object, and then the posture recognition efficiency is improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a gesture recognition method, device, and computer-readable storage medium. Background Art

[0002] During extended periods of study or office work, people often develop poor posture, often failing to recognize and correct it in time, which can seriously impact their health. With the rapid development of computer technology, posture recognition methods can now be used to detect abnormal postures. For example, a camera can capture images of a user's sitting posture and then use artificial intelligence algorithms to perform posture recognition on the captured images.

[0003] During the research and practice of existing technologies, it was found that existing gesture recognition methods are extremely susceptible to external interference, and thus cannot effectively and accurately recognize user gestures, resulting in low gesture recognition efficiency. Summary of the Invention

[0004] The embodiments of the present application provide a gesture recognition method, device, and computer-readable storage medium, which can efficiently and accurately recognize a user's gesture, thereby improving gesture recognition efficiency.

[0005] The present invention provides a method for gesture recognition, including:

[0006] Acquire an object posture image of a target object, and extract posture features from the object posture image;

[0007] Acquiring pressure distribution information of the target object based on the object posture image, performing feature extraction on the pressure distribution information, and obtaining pressure distribution features of the pressure distribution information;

[0008] Determining a first weight corresponding to the posture feature and a second weight corresponding to the pressure distribution feature;

[0009] Performing posture recognition processing on the target object based on the first weight, the second weight, the posture feature, and the pressure distribution feature to obtain a posture recognition result of the target object.

[0010] Accordingly, an embodiment of the present application provides a gesture recognition device, comprising:

[0011] a first acquiring unit, configured to acquire an object posture image of a target object and extract posture features from the object posture image;

[0012] a second acquiring unit, configured to acquire pressure distribution information of the target object based on the object posture image, perform feature extraction on the pressure distribution information, and obtain pressure distribution features of the pressure distribution information;

[0013] a weight determination unit, configured to determine a first weight corresponding to the posture feature and a second weight corresponding to the pressure distribution feature;

[0014] The posture recognition unit is configured to perform posture recognition processing on the target object based on the first weight, the second weight, the posture feature, and the pressure distribution feature to obtain a posture recognition result of the target object.

[0015] In one embodiment, the gesture recognition unit includes:

[0016] a feature fusion subunit, configured to perform feature fusion processing on the posture feature and the pressure distribution feature according to the first weight and the second weight to obtain a target posture feature;

[0017] The gesture recognition subunit is configured to perform gesture recognition on the target object based on the target gesture feature to obtain a gesture recognition result.

[0018] In one embodiment, the gesture recognition subunit is configured to:

[0019] Predicting the matching degree between the target object and a plurality of preset candidate postures based on the target posture feature, and obtaining a posture matching probability corresponding to each candidate posture;

[0020] At least one target posture is determined from the candidate postures according to the posture matching probability, and the target posture is determined as the posture recognition result of the target object.

[0021] In one embodiment, the feature fusion subunit is configured to:

[0022] Performing weighted processing on the posture feature according to the first weight to obtain a weighted posture feature;

[0023] performing weighted processing on the pressure distribution feature according to the second weight to obtain a weighted pressure distribution feature;

[0024] The weighted posture feature and the weighted pressure distribution feature are subjected to feature splicing processing to obtain a target posture feature.

[0025] In one embodiment, the first acquiring unit is configured to:

[0026] Extracting key points from the object posture image to obtain key point features of the target object;

[0027] Performing facial recognition on the object posture image to obtain the face area of ​​the target object;

[0028] Extracting the head posture of the target object based on the area where the face is located to obtain the head posture features of the target object;

[0029] The key point features and the head posture features are fused to obtain posture features.

[0030] In one embodiment, the weight determination unit includes:

[0031] a state detection subunit, configured to detect first device state information of an image acquisition device that acquires the posture image of the object, and to detect second device state information of a pressure acquisition device that acquires the pressure distribution information;

[0032] The weight determination subunit is used to determine a first weight corresponding to the posture feature and a second weight corresponding to the pressure distribution feature based on the first device status information and the second device status information.

[0033] In one embodiment, the weight determination subunit includes:

[0034] a first scoring module, configured to score the state of the image acquisition device according to the first device state information to obtain a first state score of the image acquisition device;

[0035] a second scoring module, configured to score the status of the pressure acquisition device according to the second device status information to obtain a second status score of the pressure acquisition device;

[0036] A weight determination module is configured to determine a first weight corresponding to the posture feature based on the first state score, and to determine a second weight corresponding to the pressure distribution feature based on the second state score.

[0037] In one embodiment, the pressure collection device includes a plurality of preset pressure collection areas, corresponding pressure sensors are distributed in the pressure collection areas, the pressure distribution information includes pressure information collected by the pressure sensors distributed in each of the pressure collection areas, and the second scoring module is configured to:

[0038] Score the status of each pressure sensor in the pressure acquisition device according to the second device status information to obtain a second status score corresponding to each pressure sensor;

[0039] The weight determination module is used to:

[0040] determining, according to the second state score corresponding to each pressure sensor, a sub-weight of a sub-feature corresponding to each pressure information in the pressure distribution feature;

[0041] A second weight of the pressure distribution feature is determined based on the sub-weight corresponding to the pressure distribution feature.

[0042] In addition, an embodiment of the present application further provides a computer-readable storage medium, which stores a plurality of instructions, and the instructions are suitable for loading by a processor to execute the steps in any one of the gesture recognition methods provided in the embodiment of the present application.

[0043] In addition, an embodiment of the present application further provides a computer device, including a processor and a memory, wherein the memory stores an application program, and the processor is configured to run the application program in the memory to implement the gesture recognition method provided in the embodiment of the present application.

[0044] The present application also provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the steps of the gesture recognition method provided in the present application.

[0045] The embodiment of the present application obtains an object posture image of the target object, extracts posture features from the object posture image; obtains pressure distribution information generated by the target object based on the object posture image, performs feature extraction on the pressure distribution information, and obtains pressure distribution features of the pressure distribution information; determines a first weight corresponding to the posture feature and a second weight corresponding to the pressure distribution feature; performs posture recognition processing on the target object based on the first weight, the second weight, the posture feature, and the pressure distribution feature, and obtains a posture recognition result of the target object. In this way, by obtaining the object posture image and pressure distribution information of the target object to extract posture features and pressure distribution features, and determining a first weight corresponding to the posture feature and a second weight corresponding to the pressure distribution feature, posture recognition processing is performed on the target object based on the first weight, the second weight, the posture feature, and the pressure distribution feature. In this way, the user's posture can be efficiently and accurately recognized by combining the visual features and pressure distribution information corresponding to the target object, thereby improving the efficiency of posture recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0047] Figure 1This is a schematic diagram of an implementation scenario of a gesture recognition method provided in an embodiment of the present application;

[0048] Figure 2 This is a flow chart of a gesture recognition method provided in an embodiment of the present application;

[0049] Figure 3a This is a schematic diagram of key point extraction of a gesture recognition method provided by an embodiment of the present application;

[0050] Figure 3b Schematic diagram of a collection device for a gesture recognition method provided in an embodiment of the present application;

[0051] Figure 3c This is a schematic diagram of another acquisition device for a gesture recognition method provided by an embodiment of the present application;

[0052] Figure 3d This is another schematic diagram of a collection device for a gesture recognition method provided in an embodiment of the present application;

[0053] Figure 3e This is a schematic diagram of a pressure acquisition device for a posture recognition method provided in an embodiment of the present application;

[0054] Figure 3f This is a schematic diagram of a pressure sensor in a posture recognition method provided by an embodiment of the present application;

[0055] Figure 3g This is a schematic diagram of another pressure acquisition device for a posture recognition method provided by an embodiment of the present application;

[0056] Figure 4 This is a schematic diagram of a specific flow chart of a gesture recognition method provided in an embodiment of the present application;

[0057] Figure 5 is a structural diagram of a gesture recognition device provided in an embodiment of the present application;

[0058] Figure 6 It is a structural diagram of the computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0059] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without making creative efforts are within the scope of protection of this application.

[0060] The embodiments of the present application provide a gesture recognition method, device, and computer-readable storage medium. The gesture recognition device can be integrated into a computer device, which can be a server or a terminal.

[0061] Among them, the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, network acceleration services (Content Delivery Network, CDN), as well as basic cloud computing services such as big data and artificial intelligence platforms. Terminals may include but are not limited to mobile phones, computers, intelligent voice interaction devices, smart home appliances, vehicle-mounted terminals, aircraft, etc. Terminals and servers can be directly or indirectly connected through wired or wireless communication, and this application does not impose any restrictions on this.

[0062] See also Figure 1 , taking the gesture recognition device integrated into the computer equipment as an example, Figure 1 A schematic diagram of an implementation scenario of the posture recognition method provided in an embodiment of the present application, wherein the computer device can be a server or a terminal, and the computer device can obtain an object posture image of the target object and extract posture features from the object posture image; obtain pressure distribution information generated by the target object based on the object posture image, perform feature extraction on the pressure distribution information, and obtain pressure distribution features of the pressure distribution information; determine a first weight corresponding to the posture feature and a second weight corresponding to the pressure distribution feature; perform posture recognition processing on the target object based on the first weight, the second weight, the posture feature, and the pressure distribution feature to obtain a posture recognition result of the target object.

[0063] It should be noted that Figure 1 The schematic diagram of the implementation environment scenario of the gesture recognition method shown is merely an example. The implementation environment scenario of the gesture recognition method described in the embodiments of this application is intended to more clearly illustrate the technical solutions of the embodiments of this application and does not constitute a limitation of the technical solutions provided in the embodiments of this application. Persons skilled in the art will appreciate that with the evolution of data processing and the emergence of new business scenarios, the technical solutions provided in this application are equally applicable to similar technical problems.

[0064] The solutions provided in the embodiments of the present application are specifically described by the following embodiments. It should be noted that the description order of the following embodiments does not limit the preferred order of the embodiments.

[0065] This embodiment will be described from the perspective of a gesture recognition device. The gesture recognition device may be integrated into a computer device, which may be a server or a terminal. This application does not limit this.

[0066] See also Figure 2 , Figure 2 : is a flow chart of a gesture recognition method provided in an embodiment of the present application. The gesture recognition method includes:

[0067] In step 101, an object posture image of a target object is acquired, and posture features are extracted from the object posture image.

[0068] The target object may be an object for posture recognition, for example, a person, an animal, etc. The object posture image may be an image including the target object, in which the target object exhibits the posture to be recognized. The posture feature may be information representing the posture of the target object extracted based on the object posture image. Posture types may include sitting, standing, sleeping, and other types of postures. Specific postures may include normal and abnormal postures. Abnormal postures may include bending over, lying on a table, crossing legs, leaning left or right, slumping, lowering the head, and raising the head.

[0069] There are many ways to extract posture features from an object posture image. For example, key point extraction can be performed on the object posture image to obtain key point features of the target object, face recognition can be performed on the object posture image to obtain the face area of ​​the target object, head posture extraction can be performed on the target object based on the face area to obtain head posture features of the target object, and key point features and head posture features can be fused to obtain posture features.

[0070] Among them, the key point feature can be information representing the distribution of key points of the target object in the object posture image, the face area can be the area where the face of the target object is located in the object posture image, and the head posture feature can be information representing the posture of the head of the target object in the object posture image.

[0071] Among them, there are many ways to extract key points from the object posture image and obtain the key point features of the target object. For example, key point detection can be performed on the object posture image to obtain the key point information of the target object in the object posture image. The key point information may include information such as the category and position of the key points, so that the key point information can be converted into a feature vector to obtain the key point features of the target object.

[0072] For example, see Figure 3a , Figure 3aThis is a schematic diagram of a key point extraction method for posture recognition provided by an embodiment of the present application. A human key point estimation model can be used to detect key points of an object posture image. The human key point estimation model inputs the object posture image of the target object captured by the camera at the current moment or multiple moments, and can output but is not limited to Figure 3a The key point information of the 21 key points shown in is preprocessed into a one-dimensional feature vector to obtain the key point features of the target object.

[0073] There are many ways to extract the head posture of the target object based on the area where the face is located and obtain the head posture features of the target object. For example, the head posture estimation algorithm can be used to estimate the head posture of the image of the area where the face is located, so as to obtain the deflection angle of the target object's head in three-dimensional space. For example, in three-dimensional space, the rotation of an object can be represented by three Euler angles. The head posture estimation algorithm can be used to determine the direction of the head posture: Pitch (rotation around the X-axis), Yaw (rotation around the Y-axis), and Ro1l (rotation around the Z-axis). The scientific names for these angles are pitch, yaw, and roll, respectively. Generally speaking, this means looking up, shaking your head, and turning your head. Therefore, based on the deflection angles in the three directions output by the head posture estimation algorithm, thresholds (empirical values) for various abnormal postures can be set. When the deflection angle reaches a preset threshold for a certain abnormal posture, it can be considered that the current target object has an abnormal posture. For example, when Yaw exceeds a certain threshold, it can be considered that the face of the target object in the object posture image is in profile. When Roll exceeds a certain threshold, it can be considered that the target object in the object posture image is in a tilted head posture. When Pitch exceeds a certain threshold, it can be considered that the target object in the object posture image is in a head-down or head-up posture. In this way, abnormal head movements of the target object, such as head-down, head-up, head-tilted, and head-side, can be accurately analyzed based on head posture features.

[0074] After extracting the head pose of the target object based on the facial region and obtaining the head pose features of the target object, the key point features and the head pose features can be fused to obtain the pose features. There are various ways to fuse the key point features and the head pose features to obtain the pose features corresponding to the target object. For example, the key point features and the head pose features can be concatenated to obtain the pose features corresponding to the target object.

[0075] In step 102, pressure distribution information of the target object generated based on the object posture image is obtained, and features are extracted from the pressure distribution information to obtain pressure distribution features of the pressure distribution information.

[0076] Among them, the pressure distribution information can be information describing the distribution of pressure, for example, it can include multiple pressure information, the pressure information can be the pressure value collected by each pressure sensor, and the pressure distribution feature can be information characterizing the pressure distribution indicated by the pressure distribution information.

[0077] In order to more accurately and efficiently identify the posture of the target object, the posture recognition method provided in the embodiment of the present application obtains the object posture image and pressure distribution information of the target object at the same time, so as to combine the two modal information of visual information provided by the object posture image and pressure information provided by the pressure distribution information to achieve accurate recognition of the posture of the target object.

[0078] In one embodiment, the present application provides a gesture recognition system, which includes a pressure acquisition device provided with a pressure or stress distribution sensor and an image acquisition device including at least one camera, for example, please refer to Figure 3b , Figure 3b : This is a schematic diagram of an acquisition device for a posture recognition method provided in an embodiment of the present application. The posture recognition system includes a pressure acquisition device (seat) provided with a pressure or stress distribution sensor and an image acquisition device (camera) including at least one camera. The camera can capture the image information of the target object on the chair in whole or in part to obtain an image of the object's posture. The camera can be a special camera or a camera component on a mobile phone, smart computer, tablet computer, electronic blackboard or other device. Furthermore, the camera can be an ordinary optical lens or a wide-angle lens selected to increase the field of view. In addition, the camera can also refer to a multi-eye camera to collect richer image information, including image information at different horizontal angles and different heights. The camera can also be an infrared camera, which projects an infrared light source onto the object to be measured and uses the infrared camera to capture the reflected image, etc.

[0079] Optionally, the posture recognition system provided in the embodiment of the present application can perform posture recognition processing on the target object in a wired or wireless manner, and can also perform posture recognition processing on the target object in an offline state. Specifically, a microcontroller unit (MCU) can be installed in pressure acquisition devices such as seats and image acquisition devices such as cameras. The MCU can be configured with program functions required to implement the posture recognition method provided in the embodiment of the present application, so that data calculations related to the posture recognition method provided in the embodiment of the present application can be implemented based on the microcontroller installed in the pressure acquisition device and the image acquisition device, and the posture recognition result can be obtained. In addition, the posture recognition result can also be fed back through vibration and prompt sound.

[0080] Optionally, the posture recognition system provided in the embodiment of the present application may further include a data transmission module, which may be used to encrypt the data and posture recognition results during the posture recognition processing, and send the encrypted data and posture recognition results to the external terminal corresponding to the posture recognition method provided in the embodiment of the present application. In this way, data calculation or result statistics and other data processing may be performed based on the external terminal, thereby providing more powerful computing power for the data calculation and result analysis of posture recognition through the external terminal on the basis of ensuring the data security of posture recognition, thereby improving the accuracy of posture recognition.

[0081] In one embodiment, please refer to Figure 3c , Figure 3c This is a schematic diagram of another acquisition device for a gesture recognition method provided in an embodiment of the present application. The camera can be placed on a table near a seat, such as one with a built-in camera, or with a dedicated groove, buckle, hook, or bracket for a mobile phone, tablet computer, electronic blackboard, etc. Furthermore, for privacy reasons, the camera can have a lens cap or other covering, or have power or signal input controls to limit the camera's operation.

[0082] In a specific embodiment, please refer to Figure 3d , Figure 3d This is another schematic diagram of an acquisition device for a gesture recognition method provided in an embodiment of the present application, wherein the image acquisition device can be a lens connected to a smart tablet, the camera model can be GC08A3, the pixels can be 8 million, the optical size can be 1 / 4", the frame rate can be 30FPS, the pixel size can be 1.12μm, the color array can be RGB Bayer, and the camera can be placed on a bracket set on a table.

[0083] Optionally, the chair provided in the embodiment of the present application may include a support leg structure, an adjustable bracket, a seat cushion, a lumbar support and other components. Among them, the support leg structure may be a pulley type, and this design allows the chair to move freely. The adjustable bracket can connect the support leg structure with other structures to adjust the height, front-to-back distance and backrest angle of the seat cushion and lumbar support. The seat cushion can usually be a soft filling material such as foam or latex, and can be covered with a wrapping material such as mesh or leather on the outside, and can be placed on a hard support structure. Optionally, the chair can also be provided with a footrest for placing the user's feet. Optionally, the chair can also be provided with armrests for placing the user's hands. Optionally, the chair can also be provided with a headrest for placing the user's neck or head.

[0084] Optional, please refer to Figure 3e , Figure 3eThis is a schematic diagram of a pressure collection device for a posture recognition method provided in an embodiment of the present application, wherein the seat may be provided with multiple pressure collection areas, which may be areas in the pressure collection device that collect pressure information. Multiple pressure sensors are distributed within the pressure collection area. For example, a pressure collection area may be provided on the seat cushion, and a pressure sensor capable of detecting pressure distribution may be provided within the pressure collection area. The pressure distribution sensor may be a pressure sensor of an independent supporting structure or a flexible pressure array sensor. Optionally, a pressure collection area may also be provided on the seat back or footrest, so that a pressure sensor may be configured in this area to increase the accuracy of pressure detection.

[0085] In one embodiment, the present application provides a pressure sensor with an independent support structure, for example, please refer to Figure 3f , Figure 3f This is a schematic diagram of a pressure sensor for a posture recognition method provided in an embodiment of the present application, wherein the upper portion of the pressure sensor is an integrated structure that can transmit the pressure of the seat cushion to a detection point connected to the lower base. A unit for detecting pressure is provided below the hemispherical structure. A feasible embodiment is a folding cantilever structure, with full-bridge strain gauges placed on both sides of the folding arm. By detecting the voltage difference of the Wheatstone bridge, the corresponding pressure value is output. The full-bridge strain gauge has a good inhibitory effect on temperature drift. Optionally, in order to reduce the deployment cost of the pressure sensor, other modes such as half-bridge can be selected. The upper and lower parts of the independent support structure are limited by a limiting method so that the upper structure can only move up and down. The limiting method can adopt a sleeve structure, or a hollow bottom plate, and a method of connecting the lower part of the hole with the upper part of the support structure with bolts.

[0086] Accordingly, under the seat cushion, the independent support structures have a unique distribution to fully and accurately obtain the pressure distribution required for posture recognition by using a limited number of pressure values. For example, please refer to Figure 3g , Figure 3g This is another diagram of a pressure collection device for a posture recognition method provided by an embodiment of the present application. In the pressure collection area of ​​the seat cushion, 8 independent pressure sensors can be set up. The distribution of the pressure sensors is divided into a 2x3 structure in the middle area, and the distance of the rear section is shorter than that of the front section. A long support structure can be set up in each of the left and right areas. Specifically, a precise flexible pressure sensor is used to measure the pressure distribution under normal sitting posture. Figure 3gAs shown in the pressure distribution diagram in the middle right figure, it can be seen that the color of the area with larger pressure values ​​is darker. Among them, the pressure at the ischium is more concentrated than the pressure from the thigh to the knee joint, and the pressure change at the ischium is more obvious during movements such as bending, hunching, and leaning forward. Therefore, the density of pressure sensors set at the front ischium can be lower than that of pressure sensors at the back. The pressure sensors are added at the left and right ends because there will be a more obvious left and right pressure distribution when the user leans left or right, lies on the table, or crosses their legs. In this way, these eight pressure sensors can accurately identify abnormal postures of the target object.

[0087] Optionally, a pressure collection area can be provided at the seat back, and a pressure sensor can be added to the pressure collection area to detect the pressure applied by the back to the seat back. The implementation method can be similar to that of the seat cushion.

[0088] Optionally, a pressure collection area may be provided at the footrest of the seat, and a pressure sensor may be added to the pressure collection area to detect the pressure applied by the foot to the footrest of the seat. The implementation method may be similar to the above method.

[0089] In this way, the numerical value collected by the pressure sensor can be input into the signal processing module through a pre-configured signal processing circuit, so that the pressure distribution information can be obtained. Among them, the signal processing circuit may include filtering, DC isolation, amplification and other modules. In a specific embodiment, the pressure signal within the measurement range can be output as an analog signal in a reasonable range of 0 to 3.3V through the signal processing circuit to achieve a high dynamic range. The signal processing module may include an analog to digital converter (ADC) and a digital signal processing module. Finally, the signal processing module can output a multi-channel time domain digital signal, so that the pressure distribution information can be obtained.

[0090] In one embodiment, the posture recognition model provided in the embodiment of the present application can be used to perform posture recognition on the pressure distribution information of the target object. Specifically, the pressure distribution information generated by the target object at a certain moment can be input into the fully connected network in the posture recognition model for full connection processing. Then, the result of the full connection processing can be normalized by the activation function (softmax, etc.), so as to output the probability that the sitting posture of the target object at that moment belongs to the preset multiple candidate postures. For example, please continue to refer to Figure 3g At each moment, the pressure distribution information can include 8 independent pressure values At the same time, the pressure value corresponds to the ground truth y i. Optionally, the input pressure value can be the value obtained by subtracting the baseline drift from the original measured pressure value. For example, assuming that the reference temperature is T0 and the actual temperature is T, the pressure value can be correspondingly subtracted from the baseline drift caused by temperature compensation. Optionally, the input pressure value can also be the difference between the reference value. For example, the difference between the various pressure values ​​when the user sits in a normal posture in the seat and the actual measured pressure value can be calculated, and the difference vector can be used as the final pressure value for posture recognition.

[0091] In step 103, a first weight corresponding to the posture feature and a second weight corresponding to the pressure distribution feature are determined.

[0092] Among them, the first weight can be information for measuring the importance of posture features, or information for characterizing the importance of posture features to posture recognition results when posture recognition is performed in combination with posture features and pressure distribution features; the second weight can be information for measuring the importance of pressure distribution features, or information for characterizing the importance of pressure distribution features to posture recognition results when posture recognition is performed in combination with posture features and pressure distribution features.

[0093] Optionally, the specific values ​​of the first weight and the second weight can be set according to actual needs. For example, they can be values ​​such as 0.5 and 0.6. The specific values ​​can be determined according to actual conditions and are not limited in this embodiment of the present application.

[0094] Optionally, the first weight may be a numerical value indicating the weight corresponding to the posture feature, or a weight distribution consisting of multiple weight values ​​corresponding to the posture feature. For example, it may be a weight matrix consisting of the weight corresponding to each pixel in the object posture image, that is, the first weight may include the weight corresponding to each pixel in the object posture image. The second weight may be a numerical value indicating the weight corresponding to the pressure distribution feature, or a weight distribution consisting of multiple weight values ​​corresponding to the pressure distribution feature. For example, it may be a weight matrix consisting of the weight corresponding to each pressure value in the pressure distribution information, that is, the second weight may include the weight corresponding to each pressure value in the pressure distribution information.

[0095] Among them, there can be many ways to determine the first weight corresponding to the posture feature and the second weight corresponding to the pressure distribution feature. For example, the first device status information of the image acquisition device that acquires the posture image of the object can be detected, and the second device status information of the pressure acquisition device that acquires the pressure distribution information can be detected. Based on the first device status information and the second device status information, the first weight corresponding to the posture feature and the second weight corresponding to the pressure distribution feature are determined.

[0096] The image acquisition device may be a device that acquires an image of an object's posture, for example, please continue to refer to Figure 3bThe image acquisition device may be a camera, and the first device status information may be information describing the status of the image acquisition device, for example, it may include status information such as the shooting angle, shooting light, shooting clarity, and whether it is faulty. The pressure acquisition device may be a device that collects pressure information, for example, it may be a device equipped with a pressure sensor such as a seat. The second device status information may be information describing the status of the pressure acquisition device, for example, it may include status information such as whether the sensor of the pressure acquisition device is faulty, the sensor accuracy, and the measurement range.

[0097] In this way, the embodiment of the present application extracts posture features and pressure distribution features by acquiring the object posture image and pressure distribution information of the target object, thereby determining the first weight corresponding to the posture feature and the second weight corresponding to the pressure distribution feature based on the first device status information of the image acquisition device that acquires the object posture image and the second device status information of the pressure acquisition device that acquires the pressure distribution information, thereby performing posture recognition processing on the target object based on the first weight, the second weight, the posture feature and the pressure distribution feature. In this way, the user's posture can be efficiently and accurately identified in combination with the device status and the posture image and pressure distribution information of the target object, thereby improving the posture recognition efficiency.

[0098] Among them, there can be multiple ways to determine the first weight corresponding to the posture feature and the second weight corresponding to the pressure distribution feature based on the first device status information and the second device status information. For example, the image acquisition device can be scored according to the first device status information to obtain the first status score of the image acquisition device, and the pressure acquisition device can be scored according to the second device status information to obtain the second status score of the pressure acquisition device. The first weight corresponding to the posture feature is determined based on the first status score, and the second weight corresponding to the pressure distribution feature is determined based on the second status score.

[0099] Among them, the image acquisition device is scored according to the first device status information, and there can be multiple ways to obtain the first status score of the image acquisition device. For example, the corresponding scoring rules can be set in advance for the image acquisition device. For example, when the camera is unavailable, the first status score can be 0. When the camera is available, but the shooting angle of the camera is too different to capture the target object, the first status score can be 0. When the camera is available, but the shooting angle of the camera is different and only the head of the target object can be captured, the first status score can be 50 points, etc. The specific scoring rules can be set according to actual conditions, and the embodiments of the present application are not limited here.

[0100] Among them, there are many ways to score the status of the pressure acquisition device according to the second device status information, and to obtain the second status score of the pressure acquisition device. For example, the corresponding scoring rules can be set in advance for the pressure acquisition device. For example, when the pressure sensor is not available, the second status score can be 0; when the pressure sensor is available, but the accuracy of the pressure sensor is poor, the second status score can be 80; when the pressure sensor is available, but there is a gap in the measurement range of the sensor, the second status score can be 50 points, etc. The specific scoring rules can be set according to actual conditions, and the embodiments of the present application are not limited here.

[0101] Among them, there can be many ways to determine the first weight corresponding to the posture feature based on the first state score, and to determine the second weight corresponding to the pressure distribution feature based on the second state score. For example, when the first state score is 100 points, the first weight can be 1, when the first state score is 60 points, the first weight can be 0.6, when the first state score is 0 points, the first weight can be 0, when the second state score is 100 points, the second weight can be 1, when the second state score is 70 points, the second weight can be 0.7, when the second state score is 0 points, the second weight can be 0, and so on.

[0102] In a specific embodiment, when the image acquisition device is in a device failure state such as the camera being unavailable, the first weight can be 0, that is, posture recognition is not performed in combination with the object posture image; when the pressure acquisition device is in a device failure state such as the sensor being unavailable or the data being abnormal, the second weight can be 0, that is, posture recognition is not performed in combination with the pressure distribution information.

[0103] Among them, there are many ways to score the status of the pressure acquisition device according to the second device status information, and obtain the second status score of the pressure acquisition device. For example, the pressure acquisition device may include multiple preset pressure acquisition areas, and corresponding pressure sensors may be distributed on the pressure acquisition areas. The pressure distribution information may include pressure information collected by the pressure sensors distributed in each pressure acquisition area. In this way, the status of each pressure sensor in the pressure acquisition device can be scored according to the second device status information to obtain the second status score corresponding to each pressure sensor.

[0104] Correspondingly, the step of determining the second weight corresponding to the pressure distribution feature based on the second state score may include: determining the sub-weight of the sub-feature corresponding to each pressure information in the pressure distribution feature according to the second state score corresponding to each pressure sensor, and determining the second weight of the pressure distribution feature based on the sub-weight corresponding to the pressure distribution feature.

[0105] The second weight may include multiple sub-weights, wherein the sub-weights may be weights corresponding to pressure sensors in a pressure acquisition device. The sub-features may be features corresponding to each piece of pressure information in the pressure distribution feature, and each sub-feature corresponds to a sub-weight.

[0106] In this way, by scoring the status of the image acquisition device and the pressure acquisition device based on the first device status information and the second device status information, and thereby determining the weights corresponding to the posture features and the pressure distribution features based on the status scores, the specific values ​​of the weights can be adaptively changed according to the acquired device status or the quality of the image and pressure distribution data. Thus, the confidence of the posture features of the visual modality and the pressure distribution features of the pressure modality can be indicated based on the flexibly determined weights, thereby enabling more accurate and appropriate posture recognition of the target object based on the acquired posture features and pressure distribution features, thereby improving the accuracy and effectiveness of posture recognition. For example, when the vision module is run alone and the device status information of the image acquisition device is acquired, and the collected device status information detects that the camera of the image acquisition device is partially blocked or the image is blurred, the confidence of the image acquisition device is low, and the status score of the image acquisition device can be adjusted to a lower score. When the pressure module is run alone and the device status information of the pressure acquisition device is acquired, and the collected device status information detects that the pressure acquisition device is faulty or has poor device accuracy, the confidence of the pressure acquisition device is low, and the status score of the pressure acquisition device can be adjusted to a lower score.

[0107] Optionally, there are multiple ways to determine the first weight corresponding to the posture feature and the second weight corresponding to the pressure distribution feature based on the first device status information and the second device status information. For example, the first posture weight corresponding to each pixel in the object posture image can be determined according to the first device status information, the first pressure weight corresponding to each pressure value in the pressure distribution information can be determined according to the second device status information, the second posture weight corresponding to each pixel in the object posture image in the current posture can be determined according to a preset posture type, the second pressure weight corresponding to each pressure value in the pressure distribution information in the current posture can be determined according to the preset posture type, the first posture weight and the second posture weight are fused to obtain the first weight corresponding to the posture feature in each posture type, and the first pressure weight and the second pressure weight are fused to obtain the second weight corresponding to the pressure distribution feature in each posture type.

[0108] Among them, the first posture weight can be the weight corresponding to each pixel point in the object posture image determined according to the first device status information, the first pressure weight can be the weight corresponding to each pressure value in the pressure distribution information determined according to the second device status information, and the preset posture type can be the posture type that can be identified during the posture recognition process. For example, it can include postures such as bending over, lying on the table, crossing legs, leaning left and right, slumping, lowering the head, and raising the head. The specific posture types included can be set according to actual needs. The second posture weight can be the weight corresponding to each pixel in the object posture image determined under each posture type. For example, when the current posture type to be identified is the head-down or head-up type, the pixel points in the object posture image belonging to the head area of ​​the target object will be given a higher weight, and the weight corresponding to the pixel points in the object posture image that do not belong to the head area of ​​the target object will be lower than the weight corresponding to the pixel points in the object posture image belonging to the head area of ​​the target object. For example, when the current posture type to be identified is the crossed-legs type, the pixel points in the object posture image belonging to the lower body area of ​​the target object will be given a higher weight, and the weight corresponding to the pixel points in the object posture image belonging to the upper body area of ​​the target object will be lower than the weight corresponding to the pixel points in the object posture image belonging to the lower body area of ​​the target object, and so on. The second pressure weight can be the weight corresponding to each pressure value in the pressure distribution information determined under each posture type. For example, when the posture type to be identified currently is the left or right tilt type, the pressure values ​​belonging to the left and right areas of the target object in the pressure distribution information will be given a higher weight, and the weight of the pressure value that does not belong to the left or right area in the pressure distribution information will be lower than the weight of the pressure value belonging to the left and right areas of the target object in the pressure distribution information. For another example, when the posture type to be identified currently is the slumped type, the pressure values ​​belonging to the front and back areas of the target object in the pressure distribution information will be given a higher weight, and the weight of the pressure value that does not belong to the front and back areas in the pressure distribution information will be lower than the weight of the pressure value belonging to the front and back areas of the target object in the pressure distribution information, and so on.

[0109] Among them, there can be multiple ways to determine the second posture weight corresponding to each pixel point in the object posture image in the current posture according to the preset posture type. For example, at least one first area and a second area other than the first area can be determined in the object posture image according to the preset posture type, and the weight corresponding to the pixel point in the first area is determined based on the preset weight corresponding to the first area, and the weight corresponding to the pixel point in the second area is determined based on the preset weight corresponding to the second area, so as to obtain the second posture weight corresponding to each pixel point in the object posture image in the current posture.

[0110] The first region may be a region in the object posture image that has a greater impact on the recognition result of the current posture type, or may be a more important region under the current posture type. For example, when the current posture type to be recognized is a head-down or head-up type, the first region may be the head region of the target object in the object posture image; when the current posture type to be recognized is a leg-crossed type, the first region may be the lower body region of the target object in the object posture image. The preset weights herein may be weights pre-set for the first region and the second region, wherein the preset weight corresponding to the first region may be greater than the preset weight corresponding to the second region.

[0111] Among them, there can be multiple ways to determine the second pressure weight corresponding to each pressure value in the pressure distribution information in the current posture according to the preset posture type. For example, at least one third area and a fourth area other than the third area can be determined in the pressure distribution information according to the preset posture type. The weight corresponding to the pressure value in the third area is determined based on the preset weight corresponding to the third area, and the weight corresponding to the pressure value in the fourth area is determined based on the preset weight corresponding to the fourth area, so as to obtain the second pressure weight corresponding to each pressure value in the pressure distribution information in the current posture.

[0112] The third region may be a region in the pressure distribution information that has a greater impact on the recognition result of the current posture type, or may be a more important region under the current posture type. For example, when the posture type to be recognized is a left-right tilt type, the third region may be a region in the pressure distribution information indicating the pressure values ​​on the left and right sides of the target object. When the posture type to be recognized is a slumped type, the third region may be a region in the pressure distribution information indicating the pressure values ​​on the front and back sides of the target object. The preset weights herein may be weights pre-set for the third and fourth regions, wherein the preset weight corresponding to the third region may be greater than the preset weight corresponding to the fourth region.

[0113] Among them, the first posture weight and the second posture weight are fused to obtain the first weight corresponding to the posture feature under each posture type, and the first pressure weight and the second pressure weight are fused to obtain the second weight corresponding to the pressure distribution feature under each posture type. There are many ways to do this. For example, the first posture weight and the second posture weight can be weight-accumulated to obtain the first weight corresponding to the posture feature under each posture type, and the first pressure weight and the second pressure weight can be weight-accumulated to obtain the second weight corresponding to the pressure distribution feature under each posture type. The first posture weight and the second posture weight can also be multiplied to obtain the first weight corresponding to the posture feature under each posture type, and the first pressure weight and the second pressure weight can be multiplied to obtain the second weight corresponding to the pressure distribution feature under each posture type. Among them, the specific fusion processing method can be set according to actual conditions, and the embodiments of the present application are not limited here.

[0114] Optionally, in the process of determining the first weight corresponding to the posture feature and the second weight corresponding to the pressure distribution feature based on the first device status information and the second device status information, when the weight of one modal information is lower, the weight of the other modal information can be increased accordingly. For example, when the confidence of the object posture image is lower and the first weight of the determined posture feature is smaller, the second weight of the pressure distribution feature corresponding to the posture feature can be increased accordingly. When the confidence of the pressure distribution information is lower and the second weight of the determined pressure distribution feature is smaller, the first weight of the posture feature corresponding to the pressure distribution feature can be increased accordingly. In this way, the efficiency of posture recognition can be improved.

[0115] In one embodiment, the determination process of the first weight corresponding to the posture feature and the second weight corresponding to the pressure distribution feature can be updated in an iterative manner. For example, posture recognition processing can be performed based on the object posture image and pressure distribution information, so that when it is recognized that the object posture image has abnormal conditions such as occlusion and blur, it indicates that the state of the current image acquisition device is abnormal. At this time, the confidence of the object posture image obtained is low, so that the first device status information of the image acquisition device can be obtained, and the first weight corresponding to the object posture image is updated according to the first device status information. Accordingly, when it is recognized that the pressure distribution information has abnormal conditions such as data anomalies and missing data, it can be indicated that the state of the current pressure acquisition device is abnormal. At this time, the confidence of the pressure distribution information obtained is low, so that the second device status information of the pressure acquisition device can be obtained, and the second weight corresponding to the pressure distribution information is updated according to the second device status information, so that accurate and effective posture recognition processing of the target object can be achieved based on the device status, thereby effectively improving the efficiency of posture recognition.

[0116] In one embodiment, the first weight corresponding to the posture feature and the second weight corresponding to the pressure distribution feature may also be set directly according to actual needs.

[0117] In step 104 , a posture recognition process is performed on the target object based on the first weight, the second weight, the posture feature, and the pressure distribution feature to obtain a posture recognition result of the target object.

[0118] Among them, the posture recognition result can be the result obtained by posture recognition of the target object, and can be information describing the current posture of the target object. For example, it can be posture recognition results such as normal, abnormal, etc., and can also be posture recognition results such as normal, bending over, lying on the table, crossing legs, leaning left or right, slumping, lowering the head, and raising the head.

[0119] Among them, there are many ways to obtain the posture recognition result of the target object by performing posture recognition processing on the target object based on the first weight, the second weight, the posture feature and the pressure distribution feature. For example, the posture feature and the pressure distribution feature can be fused according to the first weight and the second weight to obtain the target posture feature, and the posture recognition of the target object can be performed based on the target posture feature to obtain the posture recognition result.

[0120] The target posture feature may be information that combines posture features and pressure distribution features.

[0121] Among them, there are many ways to obtain the target posture feature by performing feature fusion processing on the posture feature and the pressure distribution feature according to the first weight and the second weight. For example, the posture feature can be weighted according to the first weight to obtain the weighted posture feature, the pressure distribution feature can be weighted according to the second weight to obtain the weighted pressure distribution feature, and the weighted posture feature and the weighted pressure distribution feature can be feature spliced ​​to obtain the target posture feature.

[0122] The weighted posture feature may be a posture feature obtained by weighting the posture feature according to a first weight, and the weighted pressure distribution feature may be a pressure distribution feature obtained by weighting the pressure distribution feature according to a second weight.

[0123] There are many ways to perform feature splicing on the weighted posture feature and the weighted pressure distribution feature to obtain the target posture feature. For example, assuming that the weighted posture feature can be expressed as C1=q 1 (c1, c2, ..., cn), where q 1 Expressed as the first weight, (c1, c2, ..., cn) can be expressed as the posture feature, and the weighted pressure distribution feature can be expressed as C2 = q 2 (a1, a2, ..., an), where q2 Expressed as the second weight, (a1, a2, ..., an) can be expressed as the pressure distribution feature, then the target posture feature can be expressed as C = C1 + C2 = [q 1 (c1, c2, ..., cn), q 2 (a1, a2, ..., an)], etc.

[0124] After performing feature fusion processing on the posture features and the pressure distribution features according to the first weight and the second weight to obtain the target posture features, the posture of the target object can be recognized based on the target posture features to obtain the posture recognition result. There can be many ways to obtain the posture recognition result by performing posture recognition on the target object based on the target posture features. For example, the degree of matching between the target object and multiple preset candidate postures can be predicted based on the target posture features to obtain the posture matching probability corresponding to each candidate posture. According to the posture matching probability, at least one target posture is determined from the candidate postures, and the target posture is determined as the posture recognition result of the target object.

[0125] Among them, the posture matching probability can be the probability that the target object belongs to each candidate posture based on the target posture feature prediction. The candidate posture can be a plurality of possible postures set in advance, for example, it can include normal, bending over, lying on the table, crossing legs, leaning left and right, sitting slumped, lowering the head, raising the head and other postures. The target posture can be the posture that matches the target object among the candidate postures, that is, the posture of the target object predicted based on the target posture feature.

[0126] There are various ways to predict the degree of match between the target object and multiple preset candidate postures based on the target posture features, and to obtain the posture matching probability corresponding to each candidate posture. For example, a posture recognition model based on an artificial intelligence model can be used to predict the posture matching probability of the target object belonging to each candidate posture based on the target posture features. Optionally, the posture recognition model can be a neural network model for posture recognition, which can include a feature extraction layer and a posture recognition layer. The feature extraction layer can be a fully connected network or a convolutional neural network structure for extracting features from the object posture image and pressure distribution information. The posture recognition layer can include a neural network architecture (Transformer) based on a self-attention mechanism and a normalization network. The Transformer network is used to extract multimodal information from the target posture features, and the normalization network is used to classify the posture of the target object based on the target posture features. For example, the probability of the target object's current posture belonging to one or more of the candidate posture categories (normal, hunched over, lying on the table, crossed legs, leaning left and right, and slumped) can be output.

[0127] In one embodiment, please refer to Figure 4 , Figure 4 : This is a specific flow chart of a posture recognition method provided in an embodiment of the present application. The posture recognition method provided in an embodiment of the present application may include a data alignment module, a feature fusion module, and a classification module. The data alignment module is used to align the timestamps of pressure data (pressure distribution information) and visual data (object posture image) using methods such as time domain interpolation or resampling. In the feature fusion module, pressure distribution features and posture features are extracted from the pressure distribution information and the object posture image through a convolutional neural network, and the pressure distribution features and posture features are feature-weighted and fused according to a first weight and a second weight, thereby obtaining a target posture feature that integrates pressure distribution and visual information. The function of this classification module is to perform posture recognition based on the extracted target posture features. It can be implemented using a multi-layer fully connected network including a Transformer network. The activation function of the network output layer can use the activation function (softmax) used for multi-classification problems to achieve single-label multi-classification posture recognition results. For example, the posture recognition result of which of the six categories of normal, bending over, lying on the table, crossing legs, leaning left and right, and slumped can be output. The logistic regression activation function (sigmoid) can also be used to achieve multi-label multi-classification posture recognition results. For example, the posture recognition result of the target object's current sitting posture having three abnormal postures of bending over, crossing legs, and leaning left and right can be output.

[0128] Thus, the posture recognition method provided in the embodiments of the present application provides a solution for accurately identifying the posture of a target object using a seat (pressure acquisition device) based on a limited number of pressure sensors in conjunction with a camera (image acquisition device). By combining visual information and pressure distribution information, abnormal postures of the target object's upper and lower limbs can be accurately identified and classified simultaneously, thereby improving the efficiency of posture recognition. On this basis, by detecting the device status information of the pressure acquisition device and the image acquisition device, the weights of the posture features and the pressure distribution features are determined according to the device status, and posture recognition is performed based on the weights and the posture features and the pressure distribution features. When poor lighting, device failure, or other factors affect the normal operation of the camera or the camera is unavailable, the seat solution for posture recognition based on pressure distribution information becomes the dominant information for identifying the posture of the target object, thereby achieving accurate and effective recognition of the posture of the target object. When the seat has a fault such as an abnormality in the collected pressure value, for example, an abnormal pressure sensor, the camera solution for posture recognition based on the object posture image can become the dominant information for identifying and classifying the target object's posture, achieving efficient and accurate recognition of the user's posture, thereby further effectively improving the efficiency of posture recognition.

[0129] As can be seen from the above, the embodiment of the present application obtains an object posture image of the target object, extracts posture features from the object posture image; obtains pressure distribution information generated by the target object based on the object posture image, performs feature extraction on the pressure distribution information, and obtains pressure distribution features of the pressure distribution information; determines a first weight corresponding to the posture feature and a second weight corresponding to the pressure distribution feature; performs posture recognition processing on the target object based on the first weight, the second weight, the posture feature, and the pressure distribution feature to obtain a posture recognition result of the target object. In this way, by obtaining the object posture image and pressure distribution information of the target object to extract posture features and pressure distribution features, and determining the first weight corresponding to the posture feature and the second weight corresponding to the pressure distribution feature, the target object is subjected to posture recognition processing based on the first weight, the second weight, the posture feature, and the pressure distribution feature. In this way, the user's posture can be efficiently and accurately recognized by combining the visual features and pressure distribution information corresponding to the target object, thereby improving the efficiency of posture recognition.

[0130] In order to better implement the above method, an embodiment of the present invention further provides a gesture recognition device, which can be integrated into a computer device, and the computer device can be a terminal or a server.

[0131] For example, Figure 5 , which is a schematic diagram of the structure of a gesture recognition device provided in an embodiment of the present application, the gesture recognition device may include a first acquisition unit 201, a second acquisition unit 202, a weight determination unit 203, and a gesture recognition unit 204, as follows:

[0132] A first acquiring unit 201 is configured to acquire an object posture image of a target object and extract posture features from the object posture image;

[0133] The second acquisition unit 202 is configured to acquire pressure distribution information of the target object based on the object posture image, perform feature extraction on the pressure distribution information, and obtain pressure distribution features of the pressure distribution information;

[0134] A weight determination unit 203, configured to determine a first weight corresponding to the posture feature and a second weight corresponding to the pressure distribution feature;

[0135] The gesture recognition unit 204 is configured to perform gesture recognition processing on the target object based on the first weight, the second weight, the gesture feature, and the pressure distribution feature to obtain a gesture recognition result of the target object.

[0136] In one embodiment, the gesture recognition unit 204 includes:

[0137] A feature fusion subunit is used to perform feature fusion processing on the posture feature and the pressure distribution feature according to the first weight and the second weight to obtain the target posture feature;

[0138] The posture recognition subunit is used to perform posture recognition on the target object based on the target posture feature to obtain a posture recognition result.

[0139] In one embodiment, the gesture recognition subunit is configured to:

[0140] Predicting the matching degree between the target object and multiple preset candidate postures based on the target posture features, and obtaining the posture matching probability corresponding to each candidate posture;

[0141] At least one target posture is determined from the candidate postures according to the posture matching probability, and the target posture is determined as the posture recognition result of the target object.

[0142] In one embodiment, the feature fusion subunit is configured to:

[0143] Performing weighted processing on the posture feature according to the first weight to obtain a weighted posture feature;

[0144] performing weighted processing on the pressure distribution feature according to the second weight to obtain a weighted pressure distribution feature;

[0145] The weighted posture features and weighted pressure distribution features are spliced ​​together to obtain the target posture features.

[0146] In one embodiment, the first acquiring unit 201 is configured to:

[0147] Extract key points from the object posture image to obtain the key point features of the target object;

[0148] Perform facial recognition on the object posture image to obtain the target object's face area;

[0149] Extract the head posture of the target object based on the face area to obtain the head posture features of the target object;

[0150] The key point features and head posture features are fused to obtain posture features.

[0151] In one embodiment, the weight determination unit 203 includes:

[0152] a state detection subunit, configured to detect first device state information of an image acquisition device for acquiring an image of a posture of an object, and to detect second device state information of a pressure acquisition device for acquiring pressure distribution information;

[0153] The weight determination subunit is used to determine a first weight corresponding to the posture feature and a second weight corresponding to the pressure distribution feature based on the first device status information and the second device status information.

[0154] In one embodiment, the weight determination subunit includes:

[0155] A first scoring module is used to score the state of the image acquisition device according to the first device state information to obtain a first state score of the image acquisition device;

[0156] a second scoring module, configured to score the status of the pressure acquisition device according to the second device status information to obtain a second status score of the pressure acquisition device;

[0157] The weight determination module is used to determine a first weight corresponding to the posture feature based on the first state score, and to determine a second weight corresponding to the pressure distribution feature based on the second state score.

[0158] In one embodiment, the pressure collection device includes a plurality of preset pressure collection areas, each of which is provided with corresponding pressure sensors. The pressure distribution information includes pressure information collected by the pressure sensors distributed in each pressure collection area. The second scoring module is configured to:

[0159] Scoring the status of each pressure sensor in the pressure acquisition device according to the second device status information to obtain a second status score corresponding to each pressure sensor;

[0160] Weight determination module, used to:

[0161] Determining, according to the second state score corresponding to each pressure sensor, a sub-weight of a sub-feature corresponding to each pressure information in the pressure distribution feature;

[0162] A second weight of the pressure distribution feature is determined based on the sub-weight corresponding to the pressure distribution feature.

[0163] In specific implementation, the above units can be implemented as independent entities, or can be arbitrarily combined to be implemented as the same or several entities. The specific implementation of the above units can be found in the previous method embodiments and will not be repeated here.

[0164] As can be seen from the above, the embodiment of the present application obtains the object posture image of the target object through the first acquisition unit 201, and extracts the posture features from the object posture image; the second acquisition unit 202 obtains the pressure distribution information of the target object generated based on the object posture image, performs feature extraction on the pressure distribution information, and obtains the pressure distribution features of the pressure distribution information; the weight determination unit 203 determines the first weight corresponding to the posture feature and the second weight corresponding to the pressure distribution feature; the posture recognition unit 204 performs posture recognition processing on the target object based on the first weight, the second weight, the posture feature, and the pressure distribution feature, and obtains the posture recognition result of the target object. In this way, by obtaining the object posture image and pressure distribution information of the target object to extract the posture features and the pressure distribution features, and determining the first weight corresponding to the posture feature and the second weight corresponding to the pressure distribution feature, the posture recognition processing of the target object is performed based on the first weight, the second weight, the posture feature, and the pressure distribution feature. In this way, the user's posture can be efficiently and accurately recognized by combining the visual features and pressure distribution information corresponding to the target object, thereby improving the efficiency of posture recognition.

[0165] The present application also provides a computer device, such as Figure 6 , which shows a schematic diagram of the structure of a computer device involved in an embodiment of the present application. The computer device may be a terminal. Specifically:

[0166] The computer device may include one or more processing core processors 301, one or more computer readable storage media memories 302, a power supply 303, an input unit 304 and other components. Those skilled in the art will understand that Figure 6 The computer device structure shown in the figure does not constitute a limitation on the computer device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange components differently.

[0167] Processor 301 is the control center of the computer device. It connects all components of the computer device using various interfaces and circuits. It executes software programs and / or modules stored in memory 302 and accesses data stored in memory 302 to perform various computer functions and process data. Optionally, processor 301 may include one or more processing cores. Preferably, processor 301 integrates an application processor and a modem processor. The application processor primarily handles the operating system, user interface, and application programs, while the modem processor primarily handles wireless communications. It is understood that the modem processor may not be integrated into processor 301.

[0168] The memory 302 can be used to store software programs and modules. The processor 301 executes various functional applications and data processing by running the software programs and modules stored in the memory 302. The memory 302 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 302 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage device. Accordingly, the memory 302 may also include a memory controller to provide the processor 301 with access to the memory 302.

[0169] The computer device also includes a power supply 303 for supplying power to various components. Preferably, the power supply 303 can be logically connected to the processor 301 via a power management system, thereby enabling the power management system to manage charging, discharging, and power consumption. The power supply 303 can also include one or more DC or AC power supplies, a recharging system, a power failure detection circuit, a power converter or inverter, a power status indicator, and other arbitrary components.

[0170] The computer device may further include an input unit 304 , which may be configured to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0171] Although not shown, the computer device may further include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 301 in the computer device will load the executable files corresponding to one or more application processes into the memory 302 according to the following instructions, and the processor 301 will run the application stored in the memory 302 to implement various functions as follows:

[0172] Acquire an object posture image of the target object, extract posture features from the object posture image; acquire pressure distribution information of the target object generated based on the object posture image, perform feature extraction on the pressure distribution information, and obtain pressure distribution features of the pressure distribution information; determine a first weight corresponding to the posture feature and a second weight corresponding to the pressure distribution feature; perform posture recognition processing on the target object based on the first weight, the second weight, the posture feature, and the pressure distribution feature, and obtain a posture recognition result of the target object.

[0173] The specific implementation of each of the above operations can be found in the previous embodiments and will not be described in detail here. It should be noted that the computer device provided in the embodiment of the present application and the gesture recognition method in the above embodiment are of the same concept, and the specific implementation process is detailed in the above method embodiment and will not be described in detail here.

[0174] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be accomplished by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0175] To this end, an embodiment of the present application provides a computer-readable storage medium storing a plurality of instructions that can be loaded by a processor to execute the steps of any of the gesture recognition methods provided in the embodiments of the present application. For example, the instructions can execute the following steps:

[0176] Acquire an object posture image of the target object, extract posture features from the object posture image; acquire pressure distribution information of the target object generated based on the object posture image, perform feature extraction on the pressure distribution information, and obtain pressure distribution features of the pressure distribution information; determine a first weight corresponding to the posture feature and a second weight corresponding to the pressure distribution feature; perform posture recognition processing on the target object based on the first weight, the second weight, the posture feature, and the pressure distribution feature, and obtain a posture recognition result of the target object.

[0177] The computer-readable storage medium may include a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0178] Since the instructions stored in the computer-readable storage medium can execute the steps in any one of the gesture recognition methods provided in the embodiments of the present application, the beneficial effects that can be achieved by any one of the gesture recognition methods provided in the embodiments of the present application can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0179] According to one aspect of the present application, a computer program product or computer program is provided, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in the various optional implementations provided in the above embodiments.

[0180] The above is a detailed introduction to a posture recognition method, device and computer-readable storage medium provided in the embodiments of the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core ideas. At the same time, for those skilled in the art, based on the ideas of the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present application.

Claims

1. A gesture recognition method, characterized in that: include: Acquire an object posture image of a target object, and extract posture features from the object posture image; Acquiring pressure distribution information of the target object based on the object posture image, performing feature extraction on the pressure distribution information, and obtaining pressure distribution features of the pressure distribution information; Determining a first weight corresponding to the posture feature and a second weight corresponding to the pressure distribution feature; Performing posture recognition processing on the target object based on the first weight, the second weight, the posture feature, and the pressure distribution feature to obtain a posture recognition result of the target object.

2. The gesture recognition method according to claim 1, wherein: The performing posture recognition processing on the target object based on the first weight, the second weight, the posture feature, and the pressure distribution feature to obtain a posture recognition result of the target object includes: Performing feature fusion processing on the posture feature and the pressure distribution feature according to the first weight and the second weight to obtain a target posture feature; Performing posture recognition on the target object based on the target posture feature to obtain a posture recognition result.

3. The gesture recognition method according to claim 2, wherein: The performing posture recognition on the target object based on the target posture feature to obtain a posture recognition result includes: Predicting the matching degree between the target object and a plurality of preset candidate postures based on the target posture feature, and obtaining a posture matching probability corresponding to each candidate posture; At least one target posture is determined from the candidate postures according to the posture matching probability, and the target posture is determined as the posture recognition result of the target object.

4. The gesture recognition method according to claim 2, wherein: The performing feature fusion processing on the posture feature and the pressure distribution feature according to the first weight and the second weight to obtain a target posture feature includes: performing weighted processing on the posture feature according to the first weight to obtain a weighted posture feature; performing weighted processing on the pressure distribution feature according to the second weight to obtain a weighted pressure distribution feature; The weighted posture feature and the weighted pressure distribution feature are subjected to feature splicing processing to obtain a target posture feature.

5. The gesture recognition method according to claim 1, wherein: Extracting posture features from the object posture image includes: Extracting key points from the object posture image to obtain key point features of the target object; Performing facial recognition on the object posture image to obtain the face area of ​​the target object; Extracting the head posture of the target object based on the area where the face is located to obtain the head posture features of the target object; The key point features and the head posture features are fused to obtain posture features.

6. The gesture recognition method according to any one of claims 1 to 5, characterized in that: The determining of the first weight corresponding to the posture feature and the second weight corresponding to the pressure distribution feature includes: detecting first device status information of an image acquisition device that acquires the posture image of the object, and detecting second device status information of a pressure acquisition device that acquires the pressure distribution information; Based on the first device state information and the second device state information, a first weight corresponding to the posture feature and a second weight corresponding to the pressure distribution feature are determined.

7. The gesture recognition method according to claim 6, wherein: The determining, based on the first device state information and the second device state information, a first weight corresponding to the posture feature and a second weight corresponding to the pressure distribution feature includes: Score the state of the image acquisition device according to the first device state information to obtain a first state score of the image acquisition device; Score the status of the pressure acquisition device according to the second device status information to obtain a second status score of the pressure acquisition device; A first weight corresponding to the posture feature is determined based on the first state score, and a second weight corresponding to the pressure distribution feature is determined based on the second state score.

8. The gesture recognition method according to claim 7, wherein: The pressure collection device includes a plurality of preset pressure collection areas, corresponding pressure sensors are distributed on the pressure collection areas, the pressure distribution information includes pressure information collected by the pressure sensors distributed in each of the pressure collection areas, and the status of the pressure collection device is scored according to the second device status information to obtain a second status score of the pressure collection device, including: Score the status of each pressure sensor in the pressure acquisition device according to the second device status information to obtain a second status score corresponding to each pressure sensor; The determining a second weight corresponding to the pressure distribution feature based on the second state score includes: determining, according to the second state score corresponding to each pressure sensor, a sub-weight of a sub-feature corresponding to each pressure information in the pressure distribution feature; A second weight of the pressure distribution feature is determined based on the sub-weight corresponding to the pressure distribution feature.

9. A gesture recognition device, characterized in that: include: a first acquiring unit, configured to acquire an object posture image of a target object and extract posture features from the object posture image; a second acquiring unit, configured to acquire pressure distribution information of the target object based on the object posture image, perform feature extraction on the pressure distribution information, and obtain pressure distribution features of the pressure distribution information; a weight determination unit, configured to determine a first weight corresponding to the posture feature and a second weight corresponding to the pressure distribution feature; The posture recognition unit is configured to perform posture recognition processing on the target object based on the first weight, the second weight, the posture feature, and the pressure distribution feature to obtain a posture recognition result of the target object.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor to execute the steps in the gesture recognition method according to any one of claims 1 to 8.

11. A computer device, characterized in that: The method comprises a memory, a processor, and a computer program stored in the memory and capable of running on the processor, wherein the processor implements the gesture recognition method according to any one of claims 1 to 8 when executing the computer program.

Images