Sensing data processing method, device, equipment and medium based on pressure insole

Abstract

The present application provides a sensing data processing method, device, equipment and medium based on pressure insoles, belonging to the field of data processing technology. The method is applied to pressure insoles, and the pressure insoles include at least one sensing unit, each sensing unit includes a hemispherical force-conducting column and a corresponding group of capacitor nodes, and the hemispherical force-conducting column is used to distribute the multi-dimensional pressure of the sole to a group of capacitor nodes, so that at least one sensing unit outputs at least one group of sensing capacitance signals, and the method includes: according to the pressure calibration data of at least one sensing unit, converting at least one group of sensing capacitance signals into at least one group of plantar distributed pressure values, and inputting them into a pre-trained multi-dimensional pressure prediction model in the form of a two-dimensional vector, so as to predict the three-dimensional pressure vector of the sole corresponding to the multi-dimensional pressure of the sole based on at least one group of plantar distributed pressure values. The present application can achieve high-precision estimation of the complete three-dimensional ground reaction force.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a sensing data processing method, device, equipment and medium based on pressure insoles. Background Art

[0002] In recent years, gait feature extraction technology based on plantar tactile information has gradually attracted attention. As a non-invasive and easy-to-wear collection device, plantar pressure insoles can measure the pressure distribution information of various areas of the sole in real time through array-arranged sensors during individual walking.

[0003] However, although the array-type plantar pressure insole can measure the vertical pressure on the sensor point, it cannot estimate the three-dimensional ground reaction force on the sole. The interpolation algorithm, which can solve the problem of low spatial resolution caused by the large spatial distance between sensor points, is currently a common method for estimating the vertical ground reaction force of the array-type plantar pressure insole. However, the interpolation algorithm can only fit the vertical ground reaction force and cannot estimate the complete three-dimensional ground reaction force.

[0004] Therefore, how to develop a three-dimensional ground reaction force estimation algorithm for array-type plantar pressure insoles is a technical problem that needs to be solved urgently. Summary of the invention

[0005] The purpose of this application is to provide a sensing data processing method, device, equipment and storage medium based on pressure insoles to solve the above problems.

[0006] To achieve the above objectives, in a first aspect, the present application proposes a sensing data processing method based on a pressure insole, which is applied to a pressure insole, wherein the pressure insole includes at least one sensing unit, each of which includes a hemispherical force-conducting column and a corresponding group of capacitor nodes, and the hemispherical force-conducting column is used to distribute the multi-dimensional pressure of the sole to the group of capacitor nodes, so that the at least one sensing unit outputs at least one group of sensing capacitor signals, and the sensing data processing method includes:

[0007] According to the pressure calibration data of the at least one sensing unit, converting the at least one group of sensing capacitance signals into at least one group of plantar distributed pressure values, each group of plantar distributed pressure values ​​including a group of sensing pressure values ​​corresponding to the group of capacitance nodes;

[0008] The at least one set of plantar distribution pressure values ​​are input into a pre-trained multi-dimensional pressure prediction model in the form of a two-dimensional vector, so as to predict a three-dimensional plantar pressure vector corresponding to the multi-dimensional plantar pressure based on the at least one set of plantar distribution pressure values.

[0009] In some embodiments, before inputting the at least one set of plantar distribution pressure values ​​into a pre-trained multi-dimensional pressure prediction model in the form of a two-dimensional vector, the method includes:

[0010] Acquire a test data set of multi-dimensional pressure on the sole of the pressure insole and a pressure sensing data set obtained by synchronous measurement by the at least one sensing unit;

[0011] The multidimensional pressure prediction model is obtained based on the test data set of the multidimensional plantar pressure and the pressure sensing data set obtained by synchronous measurement of the at least one sensing unit and based on the training of the neural network model.

[0012] In some embodiments, the multidimensional pressure prediction model is obtained based on the test data set based on the multidimensional plantar pressure and the pressure sensing data set obtained by synchronous measurement of the at least one sensing unit, and based on the training of the neural network model, including:

[0013] Inputting the pressure sensing data set into an initial neural network model in the form of a two-dimensional vector to train the initial neural network model to output a predicted value of a three-dimensional plantar pressure vector corresponding to the multi-dimensional plantar pressure;

[0014] Calculating the plantar three-dimensional pressure prediction loss based on the plantar multi-dimensional pressure test data set and the predicted value of the plantar three-dimensional pressure vector;

[0015] Based on the predicted three-dimensional plantar pressure loss, the initial neural network model is feedback updated until the predicted three-dimensional plantar pressure loss reaches a predetermined condition, thereby training to obtain the multi-dimensional pressure prediction model.

[0016] In some embodiments, the calculating of the predicted plantar three-dimensional pressure loss based on the plantar multi-dimensional pressure test data set and the predicted value of the plantar three-dimensional pressure vector further includes:

[0017] Calculating the mean square error loss between the test data set of the multi-dimensional plantar pressure and the predicted value of the plantar three-dimensional pressure vector;

[0018] Calculating the F-norm of the weight matrix of each layer in the initial neural network model, and determining the regularization term loss based on the F-norm of the weight matrix of each layer and a preset strength parameter;

[0019] The plantar three-dimensional pressure prediction loss is calculated based on the mean square error loss and the regularization term loss.

[0020] In some embodiments, the pressure insole further comprises an inertial sensing unit, and the method further comprises:

[0021] Based on the posture measurement data set of the inertial sensor unit, determine a group of posture measurement data in the posture measurement data set that is within a predetermined range at the same time as the multi-dimensional pressure on the sole of the foot;

[0022] The step of inputting the at least one set of plantar distributed pressure values ​​in the form of a two-dimensional vector into a pre-trained multi-dimensional pressure prediction model, so as to predict a three-dimensional plantar pressure vector corresponding to the multi-dimensional plantar pressure based on the at least one set of plantar distributed pressure values, comprises:

[0023] The at least one set of plantar distributed pressure values ​​and the set of posture measurement data are input into a pre-trained multi-dimensional pressure prediction model in the form of a two-dimensional vector, so as to predict a three-dimensional plantar pressure vector corresponding to the multi-dimensional plantar pressure based on the at least one set of plantar distributed pressure values ​​and the set of posture measurement data.

[0024] In some embodiments, before inputting the at least one set of plantar distribution pressure values ​​and the set of posture measurement data into a pre-trained multi-dimensional pressure prediction model in the form of a two-dimensional vector, the method includes:

[0025] Acquire a standard data set of multi-dimensional plantar pressure of the pressure insole, a sensing sample data set obtained by synchronous measurement of the at least one sensing unit, and a posture sample data set obtained by measurement of the inertial sensing unit;

[0026] Based on the standard data set of multi-dimensional plantar pressure, the sensing sample data set obtained by synchronous measurement of the at least one sensing unit, and the posture sample data set obtained by measurement of the inertial sensing unit, the multi-dimensional pressure prediction model is obtained based on neural network model training.

[0027] In a second aspect, the present application further proposes a sensing data processing device based on a pressure insole, the sensing data processing device being applied to a pressure insole, the pressure insole comprising at least one sensing unit, each of the sensing units comprising a hemispherical force-conducting column and a corresponding group of capacitor nodes, the hemispherical force-conducting column being used to distribute the multi-dimensional pressure of the sole to the group of capacitor nodes, so that the at least one sensing unit outputs at least one group of sensing capacitor signals, the sensing data processing device comprising:

[0028] a sensing pressure acquisition unit, configured to convert the at least one group of sensing capacitance signals into at least one group of foot plantar distribution pressure values ​​according to the pressure calibration data of the at least one sensing unit, each group of foot plantar distribution pressure values ​​including a group of sensing pressure values ​​corresponding to the group of capacitance nodes;

[0029] The three-dimensional pressure prediction unit is used to input the at least one group of plantar distribution pressure values ​​into a pre-trained multi-dimensional pressure prediction model in the form of a two-dimensional vector, so as to predict the three-dimensional plantar pressure vector corresponding to the multi-dimensional plantar pressure based on the at least one group of plantar distribution pressure values.

[0030] In a third aspect, the present application also proposes an electronic device, comprising: one or more processors; and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors execute the pressure insole-based sensing data processing method as described above.

[0031] In a fourth aspect, the present application further proposes a computer storage medium, wherein the storage medium stores executable instructions, and when the instructions are executed by a processor, the processor executes the sensing data processing method based on the pressure insole as described above.

[0032] Compared with the prior art, the beneficial effects of this application include:

[0033] On the one hand, in the pressure insole used in this application, each sensing unit includes a hemispherical force-conducting column and a corresponding set of capacitor nodes. The hemispherical force-conducting column can distribute the multi-dimensional pressure of the sole to a corresponding set of capacitor nodes. These capacitor nodes can capture more detailed pressure distribution information, thereby improving the spatial resolution of the sensing unit. On the other hand, by inputting the distributed pressure value of the sole into the pre-trained multi-dimensional pressure prediction model, the three-dimensional pressure vector of the sole corresponding to the multi-dimensional pressure of the sole can be predicted. This includes not only the pressure component in the vertical direction, but also the pressure component in the horizontal direction, thereby achieving a high-precision estimation of the complete three-dimensional ground reaction force. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope of the present application.

[0035] Figure 1 This is a schematic diagram of the structure of a pressure insole in an embodiment of the present application;

[0036] Figure 2 This is a schematic diagram of the structure of a sensor unit in one embodiment of the present application;

[0037] Figure 3 This is a schematic diagram of the structure of a capacitor node in an embodiment of the present application;

[0038] Figure 4 This is a schematic diagram of the force of a hemispherical force-guiding column in a first force direction in one embodiment of the present application;

[0039] Figure 5 This is a flow chart of a sensing data processing method based on a pressure insole in an embodiment of the present application;

[0040] Figure 6 It is a partial flow chart of a sensing data processing method based on a pressure insole in an embodiment of the present application;

[0041] Figure 7 It is a partial flow chart of a sensing data processing method based on a pressure insole in an embodiment of the present application;

[0042] Figure 8 This is a schematic diagram of a neural network model structure of a sensing data processing method based on a pressure insole in an embodiment of the present application;

[0043] Fig. 9 It is a partial flow chart of a sensing data processing method based on a pressure insole in an embodiment of the present application;

[0044] Fig.10 It is a partial flow chart of a sensing data processing method based on a pressure insole in an embodiment of the present application;

[0045] Fig.11 This is a functional module diagram of a sensing data processing device based on a pressure insole in an embodiment of the present application;

[0046] Fig.12 Schematic diagram of the structure of an electronic device involved in the sensing data processing method based on pressure insoles in an embodiment of the present application.

[0047] Explanation of the accompanying drawings: 100, pressure insole; 1, sensing unit; 11, hemispherical force-guiding column; 12, transversely arranged electrodes; 13, longitudinally arranged electrodes; 14, capacitor node; 200, sensing data processing device; 21, sensing pressure acquisition unit; 22, three-dimensional pressure prediction unit; 900 - electronic device. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations.

[0049] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application, but merely represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present application.

[0050] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0051] In the description of this application, it should be noted that the terms "center", "upper", "lower", "vertical", "horizontal", "inner", "outer", etc. indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, or the positions or positional relationships in which the product of the application is usually placed when in use. They are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific position, be constructed and operated in a specific position, and therefore cannot be understood as a limitation on this application. In addition, the terms "first", "second", "third", etc. are only used to distinguish the description, and cannot be understood as indicating or implying relative importance.

[0052] In addition, the terms "horizontal", "vertical", "overhanging" and the like do not mean that the components are required to be absolutely horizontal or overhanging, but can be slightly tilted. For example, "horizontal" only means that its direction is more horizontal than "vertical", and does not mean that the structure must be completely horizontal, but can be slightly tilted.

[0053] In the description of this application, it should also be noted that, unless otherwise clearly specified and limited, the terms "set", "install", "connect", and "connect" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a direct connection, or an indirect connection through an intermediate medium, or it can be the internal communication of two elements. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0054] As mentioned above, although the array-type plantar pressure insole can measure the vertical pressure on the sensor point, it cannot estimate the three-dimensional ground reaction force on the sole. The interpolation algorithm that can solve the problem of low spatial resolution caused by the large spatial distance between sensor points is currently a common means to estimate the vertical ground reaction force of the array-type plantar pressure insole. However, the interpolation algorithm can only fit the vertical ground reaction force and cannot estimate the complete three-dimensional ground reaction force. Therefore, how to develop a three-dimensional ground reaction force estimation algorithm for array-type plantar pressure insoles is a technical problem that needs to be solved urgently. To this end, the present application proposes a sensing data processing method, device, equipment and medium based on pressure insoles, which can achieve high-precision estimation of the complete three-dimensional ground reaction force.

[0055] The present application embodiment provides a sensing data processing method based on the pressure insole 100, which is applied to Figures 1 to 3 The pressure insole 100 shown in the figure comprises at least one sensor unit 1, each of which comprises a hemispherical force-conducting column 11 and a corresponding group of capacitor nodes 14, wherein the hemispherical force-conducting column 11 is used to distribute the multi-dimensional pressure of the sole to the group of capacitor nodes 14, so that the at least one sensor unit 1 outputs at least one group of sensing capacitance signals. Each group of capacitor nodes comprises at least four capacitor nodes 14, which are formed by crossing at least two rows of transversely arranged electrodes 12 and at least two columns of longitudinally arranged electrodes 13 in the double-layer flexible electrode layer.

[0056] It should be understood that since the contact between the sole of the foot and the ground is not completely flat, but has a certain curvature and angle, the pressure on the sole of the foot during movement is multi-dimensional, including normal force and tangential force. The normal force is the pressure perpendicular to the sole of the foot, while the tangential force is the pressure parallel to the sole of the foot, which is usually generated when the foot moves or twists. The hemispherical force-guiding column 11, due to the symmetry of its protruding hemispherical structure, can effectively distribute the normal force and tangential force in the multi-dimensional pressure of the sole of the foot to a corresponding set of capacitor nodes 14 to generate a set of sensing capacitor signals.

[0057] Specifically, Figure 4 As shown, when the normal force and tangential force are applied to the sole of the foot at the same time, the side of the hemispherical force-guiding column 11 will be deformed by the tangential force and deviate in the direction of the tangential force. This deviation will cause the capacitor nodes 14 on the side of the force-guiding column to be unevenly stressed, and the capacitor nodes 14 in the offset direction will bear more pressure. This uneven stress will generate different sensing capacitance signals between the capacitor nodes 14, thereby reflecting the size and direction of the tangential force. Therefore, the sensing unit 1 can achieve comprehensive monitoring of multi-dimensional pressure on the sole of the foot.

[0058] like Figure 5As shown, the sensing data processing method based on the pressure insole 100 provided in the embodiment of the present application includes:

[0059] Step S10, converting the at least one group of sensing capacitance signals into at least one group of plantar distribution pressure values ​​according to the pressure calibration data of the at least one sensing unit 1, each group of plantar distribution pressure values ​​including a group of sensing pressure values ​​corresponding to the group of capacitance nodes 14.

[0060] The sensing capacitance signal in this embodiment refers to the capacitance value change generated by each capacitance node 14 in the sensing unit 1 when multi-dimensional pressure is applied to the sole of the foot. The capacitance value change of each capacitance node 14 reflects the pressure exerted on the capacitance node 14. The pressure calibration data refers to the functional relationship between the capacitance value and the pressure obtained by pre-calibrating each capacitance node 14 in each sensing unit 1 under specific conditions. For example, the capacitance value of the capacitance node 14 can be measured under known pressure conditions to establish a mapping relationship between the capacitance value and the pressure.

[0061] In this embodiment, the capacitance value of each capacitor node 14 can be converted into a corresponding sensed pressure value by using the functional relationship in the pressure calibration data. Further, the sensed pressure values ​​of all capacitor nodes 14 in the same sensing unit 1 are combined into a group of plantar distribution pressure values. Each group of plantar distribution pressure values ​​reflects the plantar pressure distribution of the area covered by the sensing unit 1.

[0062] Step S20, inputting the at least one set of plantar distributed pressure values ​​into a pre-trained multi-dimensional pressure prediction model in the form of a two-dimensional vector, so as to predict a three-dimensional plantar pressure vector corresponding to the multi-dimensional plantar pressure based on the at least one set of plantar distributed pressure values.

[0063] The multi-dimensional pressure prediction model in this embodiment is a pre-trained neural network model, and its purpose is to predict the three-dimensional pressure vector of the sole of the foot based on the input sole distribution pressure value.

[0064] The plantar distribution pressure values ​​converted in step S10 are organized into a two-dimensional vector, and each row element in each two-dimensional vector represents a group of plantar distribution pressure values ​​corresponding to a sensor unit 1. Assuming that each sensor unit 1 has N capacitor nodes 14, the number of elements in each row of each two-dimensional vector is N. Assuming that the number of sensor units is M, the plantar distribution pressure values ​​can be organized into an M*N two-dimensional vector.

[0065] Furthermore, the two-dimensional vector is input into a pre-trained multi-dimensional pressure prediction model, wherein the multi-dimensional pressure prediction model outputs a corresponding three-dimensional pressure vector of the sole of the foot according to the input two-dimensional vector. Each three-dimensional pressure vector includes pressure components of the sole of the foot in three directions (e.g., x-axis, y-axis, z-axis).

[0066] In this embodiment, on the one hand, in the pressure insole 100 used in the present application, each sensing unit 1 includes a hemispherical force-conducting column 11 and a corresponding group of capacitor nodes 14. The hemispherical force-conducting column 11 can distribute the multi-dimensional pressure of the sole to a corresponding group of capacitor nodes 14. These capacitor nodes 14 can capture more refined pressure distribution information, thereby improving the spatial resolution of the sensing unit 1. On the other hand, by inputting the distributed pressure value of the sole into the pre-trained multi-dimensional pressure prediction model, the three-dimensional pressure vector of the sole corresponding to the multi-dimensional pressure of the sole can be predicted. This includes not only the pressure component in the vertical direction, but also the pressure component in the horizontal direction, thereby achieving a high-precision estimation of the complete three-dimensional ground reaction force.

[0067] In one embodiment, Figure 6 As shown, the step S20 includes:

[0068] Step A10, obtaining a test data set of multi-dimensional sole pressure of the pressure insole 100 and a pressure sensing data set synchronously measured by the at least one sensing unit 1.

[0069] The test data set of the multi-dimensional pressure on the sole of the pressure insole 100 in this embodiment refers to the calibration pressure applied when the pressure insole 100 is calibrated in three dimensions by a high-precision three-dimensional pressure calibration device (such as a force measuring treadmill), that is, the true value of the pressure components on the sole in three directions (for example, the x-axis, the y-axis, and the z-axis).

[0070] The pressure sensing data set refers to a set of pressure sensing data synchronously measured by each sensing unit 1 in the pressure insole 100 when the pressure insole 100 is subjected to three-dimensional pressure calibration by a high-precision three-dimensional pressure calibration device.

[0071] It should be noted that the test data set and the pressure sensing data set are synchronized in time, that is, at the same time point, the data of the two are matched.

[0072] Step A20, based on the test data set of multi-dimensional plantar pressure and the pressure sensing data set obtained by synchronous measurement of the at least one sensing unit 1, the multi-dimensional pressure prediction model is obtained based on neural network model training.

[0073] In this embodiment, a multi-dimensional pressure prediction model can be obtained by training a neural network model to learn the complex mapping relationship between the pressure sensing data set and the test data set.

[0074] In some embodiments, Figure 7 As shown, the step A20 includes:

[0075] Step B10, inputting the pressure sensing data set into the initial neural network model in the form of a two-dimensional vector to train the initial neural network model to output a predicted value of the three-dimensional plantar pressure vector corresponding to the multi-dimensional plantar pressure.

[0076] The initial neural network model in this embodiment is a network structure constructed according to the complex mapping relationship between the pressure sensing data set and the test data set. The initial neural network model may include: an input layer, at least two hidden layers and an output layer, wherein the number of nodes in the input layer and the number of nodes in the first hidden layer of the at least two hidden layers correspond to the number of the at least one sensing unit 1, the input layer is used to input the two-dimensional vector of the at least one group of plantar distribution pressure values, the at least two hidden layers are used to calculate the feature vector of the at least one group of plantar distribution pressure values, and the output layer is used to output the three-dimensional plantar pressure vector.

[0077] For example, a pressure insole 100 has 32 sensor units 1, and each sensor unit 1 has 4 capacitor nodes 14. Figure 8 The input layer of the initial neural network model shown has 32 input nodes for inputting the plantar pressure distribution value in the form of a 32*4 two-dimensional vector. Each row element in the two-dimensional vector is represented as ,in The hidden layer includes a first hidden layer and a second hidden layer, which are composed of 32 nodes and 64 nodes respectively, and are used to obtain the characteristic vector of the plantar distribution pressure value. The output layer has an output node for outputting a three-dimensional plantar pressure vector, which is composed of pressure components in three directions (for example, x-axis, y-axis, and z-axis).

[0078] It is further important to understand that each layer in the initial neural network in this embodiment performs a linear transformation (the product of the weight matrix and the input) and a nonlinear transformation (activation function). For example, in the first hidden layer, and , where x is the input foot plantar distribution pressure value, is the first layer weight matrix, is the bias term, is the first layer linear output, is the output result of the first layer after the ReLU activation function. In the second hidden layer, and ,in, is the second layer weight matrix, is the bias term, The second layer linear output, is the output result of the second layer after the ReLU activation function. In the output layer, and , where w3 is the weight matrix of the output layer, b3 is the bias term, The linear output result of the output layer is: ,Since this is a regression problem, the output is continuous, so there is no need to use an activation function to constrain the output to a certain range.

[0079] Step B20, calculating the predicted loss of the three-dimensional plantar pressure based on the test data set of the multi-dimensional plantar pressure and the predicted value of the three-dimensional plantar pressure vector.

[0080] In this embodiment, the difference between the predicted value and the true value can be calculated by a preset loss function, such as mean square error (MSE) or root mean square error (RMSE).

[0081] In some embodiments, Fig. 9 As shown, the step B20 includes:

[0082] Step B21, calculating the mean square error loss between the test data set of the multi-dimensional plantar pressure and the predicted value of the three-dimensional plantar pressure vector.

[0083] This embodiment uses mean square error (MSE) as the loss function, and calculates the mean square error loss between the true value in the test data set and the predicted value of the plantar three-dimensional pressure vector.

[0084] Step B22, calculate the F norm of the weight matrix of each layer in the initial neural network model, and determine the regularization term loss based on the F norm of the weight matrix of each layer and the preset strength parameter.

[0085] In order to avoid overfitting during model training, that is, the model performs well on the pressure sensing dataset used for training, but performs poorly on new data that has not been seen. This embodiment uses L2 regularization (also known as weight decay) to reduce model complexity, and at the same time, by adjusting the preset strength parameter λ of the regularization term, the model's fitting ability and generalization ability are balanced.

[0086] It is important to understand that if the preset strength parameter λ is set too large, the model may become too simple, resulting in underfitting (poor performance on the pressure sensing dataset used for training). On the contrary, if the preset strength parameter λ is set too small, the model may become too complex, resulting in overfitting. Therefore, adjusting the preset strength parameter λ to an appropriate value is key.

[0087] The regularization term loss in this embodiment can be obtained by To calculate, where λ is the preset strength parameter, L is the number of layers of the neural network, It is The F-norm of the layer weight matrix. The F-norm (Frobenius norm) is a matrix norm that represents the square root of the sum of the squares of the matrix elements.

[0088] Step B23, calculating the plantar three-dimensional pressure prediction loss based on the mean square error loss and the regularization term loss.

[0089] The calculation process of the predicted loss of the three-dimensional pressure on the sole of the foot in this embodiment is as follows:

[0090]

[0091] in, is the mean square error loss, is the regularization loss.

[0092] Step B30, based on the predicted three-dimensional pressure loss of the plantar foot, the initial neural network model is feedback updated until the predicted three-dimensional pressure loss of the plantar foot reaches a predetermined condition, thereby training the multi-dimensional pressure prediction model.

[0093] In this embodiment, the gradient of the plantar three-dimensional pressure prediction loss relative to the model parameters (weights and biases) can be calculated by the back propagation algorithm. According to the calculated gradient, the model parameters are updated using an optimization algorithm (such as gradient descent method, Adam, etc.). Then, the above model training process is performed based on the updated neural network model until the predetermined conditions are met, and the training process is stopped. The obtained neural network model is the multidimensional pressure prediction model.

[0094] The predetermined condition referred to in this embodiment may refer to the loss of the plantar three-dimensional pressure prediction reaching or being lower than a preset loss threshold. It may also refer to the number of training times reaching a preset maximum number of iterations. It may also refer to the rate of change of the loss being lower than a preset change threshold in a number of consecutive iterations, indicating that the model has converged.

[0095] In this embodiment, on the one hand, by using a test data set generated by a high-precision three-dimensional pressure calibration device, combined with a pressure sensing data set obtained by synchronous measurement of the sensing unit 1 in the pressure insole 100, a multi-dimensional pressure prediction model can be accurately trained. On the other hand, the use of a neural network model, especially a deep neural network containing multiple hidden layers, can capture the deep-level features of the pressure sensing data and improve the prediction ability of the model. On the third hand, by adding a regularization term in the calculation process of the plantar three-dimensional pressure prediction loss, the complexity of the model can be effectively controlled, overfitting can be prevented, and the generalization ability of the model can be improved.

[0096] In one embodiment, the pressure insole 100 may also include an inertial sensing unit, which continuously measures the posture (such as acceleration, angular velocity, etc.) of the object being measured to form a posture measurement data set. The sensing data processing method based on the pressure insole 100 in the embodiment of the present application also includes:

[0097] Step C10: Based on the posture measurement data set of the inertial sensor unit, determine a group of posture measurement data in the posture measurement data set that is within a predetermined range at the same time as the multi-dimensional pressure on the sole of the foot.

[0098] The predetermined range referred to in this embodiment can be within a preset time range before and after the same moment when the pressure insole senses the multi-dimensional pressure on the sole, or within a preset time range before the same moment, to ensure that the amount of a set of posture measurement data obtained matches the number of capacitor nodes 14. If the amount of posture measurement data is less than the number of capacitor nodes 14 within the selected predetermined range, the amount of data can be made consistent with the number of capacitor nodes 14 by padding with zeros, so as to ensure the stability of the multi-dimensional pressure prediction model and the consistency of the input data.

[0099] Further, the step S20 may include:

[0100] Step C20, input the at least one set of plantar distribution pressure values ​​and the set of posture measurement data into a pre-trained multi-dimensional pressure prediction model in the form of a two-dimensional vector, so as to predict a three-dimensional plantar pressure vector corresponding to the multi-dimensional plantar pressure based on the at least one set of plantar distribution pressure values ​​and the set of posture measurement data.

[0101] In this embodiment, the plantar distribution pressure value converted in step S10 and the set of posture measurement data determined in step C10 are organized into a two-dimensional vector. Assuming that each sensor unit 1 has N capacitor nodes 14, and the number of sensor units is M, the plantar distribution pressure value and the corresponding set of posture measurement data can be organized into a (M+1)*N two-dimensional vector, that is, the set of posture measurement data can be added as a newly added row element to the M*N two-dimensional vector of the plantar distribution pressure value.

[0102] Furthermore, the (M+1)*N two-dimensional vector is input into the pre-trained multi-dimensional pressure prediction model, wherein the multi-dimensional pressure prediction model outputs the corresponding three-dimensional pressure vector of the sole according to the input two-dimensional vector. Each three-dimensional pressure vector includes pressure components of the sole in three directions (e.g., x-axis, y-axis, z-axis).

[0103] In this embodiment, the posture measurement data (such as acceleration, angular velocity, etc.) provides the movement state of the foot in space, while the plantar distribution pressure value reflects the force of the foot on the ground. Combining these two types of data, the multi-dimensional pressure prediction model can have a more comprehensive understanding of the state of the foot, thereby improving the accuracy of the prediction. In addition, during walking or running, the posture of the foot will continue to change. The posture measurement data can help the multi-dimensional pressure prediction model dynamically adjust the prediction results to make it more consistent with the actual force of the foot.

[0104] Based on the above embodiments, Fig.10 As shown, before step S20, the following steps may also be included:

[0105] Step A30, obtaining a standard data set of multi-dimensional plantar pressure of the pressure insole, a sensing sample data set obtained by synchronous measurement of the at least one sensing unit, and a posture sample data set obtained by measurement of the inertial sensing unit.

[0106] The standard data set of multi-dimensional plantar pressure of the pressure insole 100 in this embodiment refers to the calibration pressure applied when the pressure insole 100 is calibrated in three dimensions using a high-precision three-dimensional pressure calibration device (such as a force measuring treadmill), that is, the actual values ​​of the pressure components of the plantar in three directions (for example, the x-axis, the y-axis, and the z-axis), which serves as the target label for the training model.

[0107] The sensing sample data set refers to a collection of pressure sensing data synchronously measured by each sensing unit 1 in the pressure insole 100 when the pressure insole 100 is subjected to three-dimensional pressure calibration by a high-precision three-dimensional pressure calibration device. It is used as one of the input features of the model to provide the specific distribution of plantar pressure.

[0108] The posture sample data set refers to a collection of posture measurement data synchronously measured by the inertial sensor unit in the pressure insole 100 when the pressure insole 100 is calibrated in three dimensions by a high-precision three-dimensional pressure calibration device. It is used as one of the input features of the model to provide posture information of the foot and help the model predict the multi-dimensional pressure of the plantar more accurately.

[0109] It should be noted that the standard data set, the sensing sample data set and the posture sample data set are synchronized in time, that is, at the same time point, the data of the three are matched.

[0110] Step A40, based on the standard data set of multi-dimensional plantar pressure, the sensing sample data set obtained by synchronous measurement of the at least one sensing unit, and the posture sample data set obtained by the inertial sensing unit, the multi-dimensional pressure prediction model is obtained based on neural network model training.

[0111] The specific training process of the multidimensional pressure prediction model in this embodiment is different from step A20 in that a posture sample data set is added as the input of the multidimensional pressure prediction model to make the model more comprehensive and accurate, and the rest of the training process is similar to the previous embodiment. Therefore, the contents in this embodiment that are the same or similar to the steps in the above embodiment can be referred to the above description and will not be repeated later.

[0112] In this embodiment, the posture sample data set can be used as an influencing factor input into the multidimensional pressure prediction model, so that the multidimensional pressure prediction model can predict the sensing sample data set in combination with the movement posture of the foot, thereby improving the accuracy and robustness of the prediction.

[0113] In one embodiment, Fig.11 As shown, the embodiment of the present application further proposes a sensing data processing device 200 based on the pressure insole 100, the sensing data processing device 200 is applied to the pressure insole 100, the pressure insole 100 includes at least one sensing unit 1, each of the sensing units 1 includes a hemispherical force-conducting column 11 and a corresponding group of capacitor nodes 14, the hemispherical force-conducting column 11 is used to distribute the multi-dimensional pressure of the sole to the group of capacitor nodes 14, so that the at least one sensing unit 1 outputs at least one group of sensing capacitance signals, the sensing data processing device 200 includes:

[0114] A sensing pressure acquisition unit 21, configured to convert the at least one set of sensing capacitance signals into at least one set of plantar distribution pressure values ​​according to the pressure calibration data of the at least one sensing unit 1, each set of plantar distribution pressure values ​​including a set of sensing pressure values ​​corresponding to the set of capacitance nodes 14;

[0115] The three-dimensional pressure prediction unit 22 is used to input the at least one set of plantar distribution pressure values ​​into a pre-trained multi-dimensional pressure prediction model in the form of a two-dimensional vector, so as to predict the three-dimensional plantar pressure vector corresponding to the multi-dimensional plantar pressure based on the at least one set of plantar distribution pressure values.

[0116] Among them, the specific embodiments executed by each module in the sensing data processing device 200 based on the pressure insole 100 of the present application are basically the same as the embodiments of the sensing data processing method based on the pressure insole 100 mentioned above, and will not be repeated here.

[0117] In one embodiment, a computer storage medium is provided, on which executable instructions are stored. When the instructions are executed by a processor, the processor executes the steps in the above-mentioned method embodiments.

[0118] In one embodiment, an electronic device 900 is provided, comprising one or more processors; a memory, wherein one or more programs are stored in the memory, wherein when the one or more programs are executed by the one or more processors, the one or more processors execute the steps in the above-mentioned method embodiments.

[0119] In one embodiment, Fig.12 As shown, it shows a schematic diagram of the structure of an electronic device 900 involved in the sensing data processing method based on the pressure insole. The electronic device 900 includes a central processing unit (CPU) 901, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 902 or the program loaded from the storage part 908 to the random access memory (RAM) 903. In the RAM 903, various programs and data required for the operation of the electronic device 900 are also stored. The CPU 901, the ROM 902, and the RAM 903 are connected to each other through a bus 904. The input / output (I / O) interface 905 is also connected to the bus 904.

[0120] The following components are connected to the I / O interface 905: an input section 906 including a keyboard, a mouse, etc.; an output section 907 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 908 including a hard disk, etc.; and a communication section 909 including a network interface card such as a LAN card, a modem, etc. The communication section 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to the I / O interface 905 as needed. A removable medium 911, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 910 as needed, so that a computer program read therefrom is installed into the storage section 908 as needed.

[0121] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, including a computer-readable medium carrying instructions, in such an embodiment, the instructions can be downloaded and installed from a network through a communication part 909, and / or installed from a removable medium 911. When the instructions are executed by a central processing unit (CPU) 901, the various method steps described in the present application are executed.

[0122] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

[0123] In addition, those skilled in the art will appreciate that, although some embodiments herein include certain features included in other embodiments but not other features, the combination of features of different embodiments is meant to be within the scope of the present application and form different embodiments. For example, in the above claims, any one of the claimed embodiments may be used in any combination. The information disclosed in this background technology section is intended only to deepen the understanding of the overall background technology of the present application and should not be regarded as an admission or in any form of implication that the information constitutes prior art known to those skilled in the art.

Claims

1. A sensing data processing method based on pressure insoles, characterized in that: The method is applied to a pressure insole, which includes a plurality of sensing units formed by a double-layer flexible electrode layer and a plurality of hemispherical force-conducting columns located on the double-layer flexible electrode layer, each of the sensing units includes a hemispherical force-conducting column and a corresponding group of capacitor nodes, the hemispherical force-conducting column is used to distribute the multi-dimensional pressure of the sole to the group of capacitor nodes, so that the plurality of sensing units output a plurality of groups of sensing capacitor signals, each group of capacitor nodes includes at least four capacitor nodes, which are formed by the intersection of at least two rows of electrodes arranged horizontally and at least two columns of electrodes arranged vertically in the double-layer flexible electrode layer, and each group of capacitor nodes outputs a corresponding group of sensing capacitor signals; The sensing data processing method comprises: According to the pressure calibration data of the multiple sensing units, the multiple groups of sensing capacitance signals are converted into multiple groups of plantar distributed pressure values, each group of plantar distributed pressure values ​​includes a group of sensing pressure values ​​corresponding to each group of capacitance nodes, wherein the capacitance value of each capacitance node is converted into a corresponding sensing pressure value by using a functional relationship in the pressure calibration data, and the sensing pressure values ​​of all capacitance nodes in the same sensing unit are combined into a group of plantar distributed pressure values; The multiple groups of plantar distributed pressure values ​​are input in the form of two-dimensional vectors into the input layer nodes corresponding to the number of the multiple sensor units in the pre-trained multi-dimensional pressure prediction model, and each row element in the two-dimensional vector represents each group of plantar distributed pressure values, so as to predict the three-dimensional plantar pressure vector corresponding to the multi-dimensional plantar pressure based on the multiple groups of plantar distributed pressure values.

2. The sensing data processing method based on the pressure insole according to claim 1 is characterized in that: Before inputting the plurality of groups of plantar distribution pressure values ​​into a pre-trained multi-dimensional pressure prediction model in the form of two-dimensional vectors, the method includes: Acquire a test data set of multi-dimensional pressure on the sole of the pressure insole and a pressure sensing data set obtained by synchronous measurement of the multiple sensing units; Based on the test data set of the multi-dimensional plantar pressure and the pressure sensing data set obtained by synchronous measurement of the multiple sensing units, the multi-dimensional pressure prediction model is obtained based on neural network model training.

3. The sensing data processing method based on the pressure insole according to claim 2 is characterized in that: The multidimensional pressure prediction model is obtained based on the test data set of the multidimensional plantar pressure and the pressure sensing data set obtained by synchronous measurement of the multiple sensing units, and based on the training of the neural network model, including: Inputting the pressure sensing data set into an initial neural network model in the form of a two-dimensional vector to train the initial neural network model to output a predicted value of a three-dimensional plantar pressure vector corresponding to the multi-dimensional plantar pressure; Calculating the plantar three-dimensional pressure prediction loss based on the plantar multi-dimensional pressure test data set and the predicted value of the plantar three-dimensional pressure vector; Based on the predicted three-dimensional plantar pressure loss, the initial neural network model is feedback updated until the predicted three-dimensional plantar pressure loss reaches a predetermined condition, thereby training to obtain the multi-dimensional pressure prediction model.

4. The sensing data processing method based on the pressure insole according to claim 3 is characterized in that: The calculating of the plantar three-dimensional pressure prediction loss based on the plantar multi-dimensional pressure test data set and the predicted value of the plantar three-dimensional pressure vector also includes: Calculating the mean square error loss between the test data set of the multi-dimensional plantar pressure and the predicted value of the plantar three-dimensional pressure vector; Calculating the F-norm of the weight matrix of each layer in the initial neural network model, and determining the regularization term loss based on the F-norm of the weight matrix of each layer and a preset strength parameter; The plantar three-dimensional pressure prediction loss is calculated based on the mean square error loss and the regularization term loss.

5. The sensing data processing method based on pressure insoles according to claim 3 is characterized in that: The multidimensional pressure prediction model includes: an input layer, at least two hidden layers and an output layer, wherein the number of nodes in the input layer and the number of nodes in the first hidden layer of the at least two hidden layers correspond to the number of the multiple sensing units, the input layer is used to input the two-dimensional vectors of the multiple groups of plantar distribution pressure values, the at least two hidden layers are used to calculate the characteristic vectors of the multiple groups of plantar distribution pressure values, and the output layer is used to output the three-dimensional plantar pressure vector.

6. The sensing data processing method based on the pressure insole according to claim 1, characterized in that: The pressure insole further comprises an inertial sensing unit, and the method further comprises: Based on the posture measurement data set of the inertial sensor unit, determine a group of posture measurement data in the posture measurement data set that is within a predetermined range at the same time as the multi-dimensional pressure on the sole of the foot; The step of inputting the plurality of sets of plantar distribution pressure values ​​in the form of two-dimensional vectors into a pre-trained multi-dimensional pressure prediction model, so as to predict the plantar three-dimensional pressure vectors corresponding to the plantar multi-dimensional pressures based on the plurality of sets of plantar distribution pressure values, comprises: The multiple sets of plantar distribution pressure values ​​and the set of posture measurement data are input into a pre-trained multi-dimensional pressure prediction model in the form of two-dimensional vectors, so as to predict the plantar three-dimensional pressure vector corresponding to the plantar multi-dimensional pressure based on the multiple sets of plantar distribution pressure values ​​and the set of posture measurement data.

7. The sensing data processing method based on the pressure insole according to claim 6 is characterized in that: Before inputting the multiple groups of plantar distribution pressure values ​​and the group of posture measurement data into a pre-trained multi-dimensional pressure prediction model in the form of two-dimensional vectors, the method includes: Acquire a standard data set of multi-dimensional plantar pressure of the pressure insole, a sensing sample data set obtained by synchronous measurement of the multiple sensing units, and a posture sample data set obtained by measurement of the inertial sensing unit; Based on the standard data set of multi-dimensional plantar pressure, the sensing sample data set obtained by synchronous measurement of the multiple sensing units, and the posture sample data set obtained by measurement of the inertial sensing unit, the multi-dimensional pressure prediction model is obtained based on neural network model training.

8. A sensing data processing device based on a pressure insole, characterized in that: The sensing data processing device is applied to a pressure insole, which includes a plurality of sensing units formed by a double-layer flexible electrode layer and a plurality of hemispherical force-conducting columns located on the double-layer flexible electrode layer, each of the sensing units includes a hemispherical force-conducting column and a corresponding group of capacitor nodes, the hemispherical force-conducting column is used to distribute the multi-dimensional pressure of the sole to the group of capacitor nodes, so that the plurality of sensing units output a plurality of groups of sensing capacitor signals, each group of capacitor nodes includes at least four capacitor nodes, which are formed by the intersection of at least two rows of electrodes arranged horizontally and at least two columns of electrodes arranged vertically in the double-layer flexible electrode layer; The sensing data processing device comprises: a sensing pressure acquisition unit, configured to convert the plurality of sensing capacitance signals into a plurality of foot plantar distribution pressure values ​​according to the pressure calibration data of the plurality of sensing units, each group of foot plantar distribution pressure values ​​comprising a group of sensing pressure values ​​corresponding to the group of capacitance nodes, wherein the capacitance value of each capacitance node is converted into a corresponding sensing pressure value by using a functional relationship in the pressure calibration data, and the sensing pressure values ​​of all capacitance nodes in the same sensing unit are combined into a group of foot plantar distribution pressure values; A three-dimensional pressure prediction unit is used to input the multiple groups of plantar distribution pressure values ​​in the form of two-dimensional vectors into the input layer nodes corresponding to the number of the multiple sensing units in the pre-trained multidimensional pressure prediction model, wherein each row element in the two-dimensional vector represents each group of plantar distribution pressure values ​​respectively, so as to predict the three-dimensional plantar pressure vector corresponding to the multi-dimensional plantar pressure based on the multiple groups of plantar distribution pressure values.

9. An electronic device, characterized in that: include: one or more processors; a memory for storing one or more programs, Wherein, when the one or more programs are executed by the one or more processors, the one or more processors execute the sensing data processing method based on the pressure insole as described in any one of claims 1 to 7.

10. A computer storage medium, characterized in that: The storage medium stores executable instructions, and when the instructions are executed by the processor, the processor executes the sensing data processing method based on the pressure insole according to any one of claims 1 to 7.

Images