In-shoe condition monitoring system based on sensor technology

Abstract

The invention relates to the technical field of sensor data analysis, in particular to an in-shoe condition monitoring system based on a sensor technology, which comprises an acquisition processing unit, a model analysis unit and an early warning feedback unit. A model analysis unit analyzes sensor data through a multi-branch deep neural network architecture, extracts pressure distribution space features, humidity and temperature time sequence features and motion space-time features respectively, adopts an attention mechanism to perform weighted fusion, and performs training through historical data; the cross entropy loss function and the self-adaptive learning rate are used for adjusting strategy optimization parameters, accurate classification results and quantitative evaluation are output, the early warning feedback unit provides different early warning modes according to evaluation results, the conditions in the shoes can be comprehensively monitored in real time, and the accuracy of monitoring the conditions in the shoes is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of sensor data analysis, and in particular to a shoe condition monitoring system based on sensor technology. Background Art

[0002] Sensor data analysis is an important technology. In athletes' training and competition scenarios, monitoring and management of foot status is a key link in improving athletic performance and preventing injuries. However, there is currently a lack of effective integrated solutions. During high-intensity exercise, the environment inside the shoe is a complex dynamic system. Factors such as pressure distribution, humidity, temperature, and movement posture are interrelated and will affect the foot.

[0003] Abnormal pressure distribution in the shoe, abnormal humidity and temperature changes, and deviations in sports posture are not isolated problems, but act together on the athlete's feet. Uneven pressure distribution may change the force on the foot, thereby affecting the sports posture, and abnormal sports posture may further aggravate the irrationality of pressure distribution. At the same time, the adverse in-shoe environment created by abnormal humidity and temperature will make the foot skin tissue fragile, reduce its tolerance to pressure and adverse posture, and increase the risk of injury. These factors combined will not only reduce the athlete's comfort and interfere with their sports performance, but also make it difficult for athletes to make timely adjustments during training and competitions due to the inability to fully and timely grasp the status of the foot, affecting normal training and competition. In order to solve this technical problem, we provide a shoe condition monitoring system based on sensor technology. Summary of the invention

[0004] The purpose of the present invention is to provide a shoe condition monitoring system based on sensor technology to solve the problems raised in the above background technology.

[0005] To achieve the above purpose, a shoe condition monitoring system based on sensor technology is provided, including a collection and processing unit, a model analysis unit and an early warning feedback unit;

[0006] The acquisition and processing unit acquires the in-shoe condition data through a multimodal sensing sequence, and processes the in-shoe condition data using a linear regression algorithm;

[0007] The model analysis unit analyzes the pressure sensor data, humidity and temperature sensor data, and motion posture data in the shoe through a multi-branch deep neural network architecture, and extracts the pressure distribution spatial features, time series features, and motion spatiotemporal features respectively, and uses the attention mechanism to adaptively weighted fuse the extracted features to obtain a fused feature vector, and then uses the historical in-shoe condition data as a training set, the fused feature vector as the input of the model, and the classification label of the in-shoe condition as the target output, and uses the cross entropy loss function to measure the difference between the model prediction and the true label, and finally uses the gradient descent algorithm to adjust the deep neural network parameters according to the difference value, and inputs the real-time collected in-shoe condition data into the deep neural network for analysis, and outputs the classification result and quantitative evaluation of the current in-shoe condition;

[0008] The early warning feedback unit is used to receive the quantitative evaluation results and provide health early warning information to the user according to the output results of the deep neural network.

[0009] As a further improvement of the technical solution, the multimodal sensing sequence in the acquisition and processing unit includes a pressure sensor, a temperature and humidity sensor, and an inertial measurement module, as follows:

[0010] Pressure sensors, humidity sensors, temperature sensors and inertial measurement modules are arranged in the shoe according to a preset layout, and data from each sensor is synchronously collected at a fixed frequency to form shoe condition data;

[0011] For the pressure sensor data, a relationship model between pressure and sole deformation is established through linear regression. For the humidity and temperature sensor data, linear regression is used to analyze the correlation between humidity and temperature and the changing trend over time.

[0012] As a further improvement of the technical solution, the model analysis unit includes a network architecture module, and the multi-branch deep neural network in the network architecture module processes the pressure sensor data and extracts the spatial characteristics of the pressure distribution, which is specifically as follows:

[0013] Collecting pressure sensor data, dividing it into regions according to the forefoot, midfoot and heel, and for each region, converting the pressure data into a two-dimensional pressure data matrix according to a preset resolution, wherein each element in the matrix represents the pressure value at the corresponding position;

[0014] Construct a spatial convolutional neural network with three convolutional layers. The first layer uses a 3*3 convolution kernel, the second layer uses a 5*5 convolution kernel, and the third layer uses a 7*7 convolution kernel. Initialize the weight parameters and bias parameters of the convolution kernel.

[0015] The constructed pressure data matrix is ​​input into the first convolution layer of the network. The 3*3 convolution kernel slides on the matrix with a preset step size to perform convolution operation. For the local area covered by each convolution kernel, its weighted sum is calculated and a bias is added to obtain the convolution result corresponding to the area. The convolution result is activated using a preset activation function to obtain the first convolution feature map.

[0016] The first convolution feature map is passed as input to the second convolution layer, the 5*5 convolution kernel repeats the above operation to obtain the second convolution feature map, the second convolution feature map is used as input to enter the third convolution layer, the 7*7 convolution kernel performs convolution operation to obtain the third convolution feature map;

[0017] The feature maps obtained by convolution kernels of different scales are weightedly fused to obtain the spatial characteristics of pressure distribution.

[0018] As a further improvement of the technical solution, the multi-branch deep neural network in the network architecture module uses a long short-term memory network to extract the time series characteristics of humidity and temperature sensor data, as follows:

[0019] Arrange the data collected by the humidity and temperature sensors in chronological order, normalize them, and map the humidity and temperature data to [0,1] respectively;

[0020] Determine the structural parameters of the LSTM network, including the number of hidden layers, the number of neurons in each hidden layer, and the dimensions of input and output, and initialize the weight matrix and bias vector of the LSTM network;

[0021] The normalized humidity and temperature time series data are input into the long short-term memory network in sequence according to the time step. For each time step, the input gate, forget gate, output gate and the hidden state of the current time step are calculated. The above calculation process is repeated until the entire time series data is processed.

[0022] The hidden state of the last time step is used as the time series feature vector of the extracted humidity and temperature sensor data.

[0023] As a further improvement of the technical solution, the multi-branch deep neural network in the network architecture module uses a three-dimensional convolutional neural network combined with an optical flow algorithm to extract motion spatiotemporal features, as follows:

[0024] Collect image sequences of motion posture data, apply optical flow algorithm to adjacent frame images to calculate optical flow field, and obtain motion information;

[0025] Set the structure of the three-dimensional convolutional neural network, that is, determine the number of layers of the network, the number, size and step size of the convolution kernels of each layer, initialize the weight parameters and bias parameters of the three-dimensional convolutional neural network, and provide a starting point for network training;

[0026] The optical flow field data and the original image sequence data are used as the input of the 3D convolutional neural network, where for each 3D convolutional layer:

[0027] The convolution kernel slides on the three-dimensional data to perform convolution operations, calculates the weighted sum of the area covered by each convolution kernel and the corresponding data, and adds a bias to obtain the convolution result. The nonlinear activation function is then applied to activate the convolution result to obtain the feature map. The feature map output by the last convolution layer of the network is used as the motion spatiotemporal features of the extracted motion posture.

[0028] As a further improvement of the technical solution, the model analysis unit includes a feature fusion module, and the feature fusion module uses an attention mechanism to perform adaptive weighted fusion on the extracted features, as follows:

[0029] Extract the feature vectors and corresponding feature dimensions of the pressure sensor data, humidity and temperature sensor data, and motion posture data from the network architecture module, concatenate these feature vectors into a total feature vector in sequence, and encode the classification label of the shoe condition;

[0030] A fully connected layer and a bias vector are introduced, and the score vector is obtained by calculation through the formula. The score vector is normalized to obtain the attention weight vector. The total feature vector and the attention weight vector are weighted summed to obtain the final fused feature vector.

[0031] As a further improvement of the technical solution, the feature fusion module adds a regularization term when using the cross entropy loss function to measure the difference between the model prediction and the true label, as follows:

[0032] Set the model prediction output to The true label is y. First, the basic part of the cross entropy loss function is calculated, namely: Where C is the number of categories for the shoe condition classification;

[0033] Assume the model parameter is θ, the regularization coefficient is λ, and calculate the L2 regularization term Add the cross entropy loss function to the regularization term to get the final loss function, that is: L = L CE +L reg .

[0034] As a further improvement of the technical solution, the model analysis unit includes a parameter adjustment module. When the parameter adjustment module uses the gradient descent algorithm to adjust the parameters of the deep neural network, an adaptive learning rate adjustment strategy is adopted, which is as follows:

[0035] Set the initial learning rate, decay rate, and adjust the step size period, and update the learning rate according to the set period during training;

[0036] At each update, the model parameters are updated according to the gradient descent algorithm based on the calculated current learning rate and the gradient of the loss function with respect to the model parameters.

[0037] Compared with the prior art, the present invention has the following beneficial effects:

[0038] In the shoe condition monitoring system based on sensor technology, the model analysis unit plays an important role. Its multi-branch deep neural network architecture can process pressure, humidity, temperature and motion posture data respectively, accurately extract the spatial characteristics of pressure distribution, time series characteristics and motion spatiotemporal characteristics, and provide rich information for a comprehensive understanding of the shoe condition. The attention mechanism can adaptively weight and fuse features, highlight the influence of key features, and enable the model to grasp the shoe status more accurately. The cross entropy loss function is combined with the regularization term to balance the prediction accuracy and generalization ability, prevent overfitting, and improve the model stability. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 It is an overall block diagram of the present invention.

[0040] The meaning of each number in the figure is:

[0041] 1. Acquisition and processing unit; 2. Model analysis unit; 21. Network architecture module; 22. Feature fusion module; 23. Parameter adjustment module; 3. Early warning feedback unit. DETAILED DESCRIPTION

[0042] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0043] The present invention provides a shoe condition monitoring system based on sensor technology, please refer to Figure 1 As shown, it includes a set processing unit 1, a model analysis unit 2 and a warning feedback unit 3;

[0044] The acquisition and processing unit 1 acquires the in-shoe condition data through a multimodal sensing sequence and processes the in-shoe condition data using a linear regression algorithm, as follows:

[0045] Place the humidity sensor in the gap between the insole and the sole, and place the temperature sensor next to the humidity sensor. This can effectively sense the humidity changes in the shoe and avoid measurement deviations due to improper positioning, so as to ensure that the temperature environment in the shoe can be accurately measured. At the same time, it should also be considered that it cannot directly contact the skin of the sole of the foot to avoid excessive interference from human body temperature and affect the accurate measurement of the ambient temperature in the shoe.

[0046] The inertial measurement module can be fixed at the middle and rear part of the shoe near the ankle. This position is relatively stable and can better sense the changes in the movement posture of the entire shoe in space. Pressure sensors are evenly distributed in key stress areas such as the front, middle and rear of the shoe sole to accurately capture pressure changes at different positions, thereby fully reflecting the stress conditions of the sole during walking and exercise.

[0047] During data collection, each sensor collects data synchronously at a set frequency of 100 Hz under the control of the microcontroller, ensuring that pressure, humidity, temperature and motion posture data are obtained at the same time, forming a set of shoe condition data samples for subsequent comprehensive analysis and processing.

[0048] First, the collected pressure sensor data is filtered and the pressure values ​​of multiple consecutive sampling points are averaged to smooth out the noise data caused by the sensor's own fluctuations or slight external interference. Then the filtered data is normalized to facilitate the subsequent calculation and analysis when establishing the relationship model.

[0049] Prepare experimental samples, let the testers wear shoes equipped with sensors to perform various activities, such as standing, walking, running, etc., and record the pressure sensor data in different states and the actual deformation degree of the sole. Take the pressure sensor data as the independent variable and the deformation degree of the sole as the dependent variable, and use the linear regression algorithm to train the model to obtain the relationship model y=kx+b between pressure and deformation degree of the sole; where y represents the deformation degree of the sole, x represents the pressure sensor data, and k and b are coefficients determined through training. In this way, in subsequent practical applications, when the pressure sensor data is obtained, the model can be used to quickly estimate the deformation of the sole, thereby evaluating the force uniformity of the shoe and the support effect on the foot.

[0050] For the humidity and temperature sensor data, filtering processing is also performed first to remove the noise data caused by environmental electromagnetic interference and other factors. Then the processed humidity and temperature data are normalized respectively. Then, the data is collected continuously for 1 hour, and the curve is drawn with time as the horizontal axis and humidity and temperature values ​​as the vertical axis.

[0051] Linear regression is used to analyze the relationship between humidity and temperature, and the correlation coefficient r between the two is calculated. If r = 1, it indicates that humidity and temperature are positively correlated, that is, humidity tends to increase when temperature rises. If r = -1, it is negatively correlated, so as to fully understand the changes in the microenvironment inside the shoe over time, so as to promptly detect whether the shoe is too humid or the temperature is too high, which may affect the health of the feet.

[0052] The model analysis unit 2 includes a network architecture module 21 , a feature fusion module 22 and a parameter adjustment module 23 .

[0053] The multi-branch deep neural network in the network architecture module 21 processes the pressure sensor data and extracts the spatial characteristics of the pressure distribution, as follows:

[0054] First, the network architecture module 21 needs to determine the weighting coefficients for fusing feature maps of different layers. The initial values ​​can be set empirically. The weighting coefficient of the first layer feature map is ω1=0.3, the weighting coefficient of the second layer feature map is ω2=0.3, and the weighting coefficient of the third layer feature map is ω3=0.4. These weighting coefficients can be adjusted and optimized according to the performance of the model in the subsequent model training process to achieve the best fusion effect.

[0055] Since convolution operations in different layers may result in different sizes of feature maps, they need to be adjusted to the same size before fusion. If the sizes of the feature maps of the second and third layers are smaller than those of the first layer, upsampling can be used to enlarge them to the same size as the first layer. After convolution and pooling operations, the size of the feature maps of the second layer becomes half of the original size, and the size of the feature maps of the third layer becomes one quarter of the original size. In this case, the feature maps of the second layer are upsampled by 2 times bilinear interpolation, and the feature maps of the third layer are upsampled by 4 times bilinear interpolation.

[0056] The three layers of feature maps after resizing are weighted fused. Suppose the first layer feature map is F1, the second layer feature map is F2, and the third layer feature map is F3. The fused pressure distribution space feature map F is calculated as follows: F=ω1F1+ω2F2+ω3F3; for each pixel position (i, j) in the feature map, the above formula is used to calculate, and the pixel values ​​of the three feature maps at the corresponding positions are multiplied by the corresponding weighting coefficients and then added to obtain the pixel value of the fused feature map F at the position (i, j).

[0057] The fused feature map is normalized to map the pixel values ​​in the feature map to between [0, 1] to improve the stability and efficiency of subsequent model processing. Each pixel value in the feature map is divided by the maximum pixel value in the feature map, that is: The normalized fusion feature map F obtained in this way normalThat is the final pressure distribution spatial feature, which integrates the pressure distribution spatial information extracted by convolution kernels of different scales, and can more comprehensively and accurately reflect the distribution of pressure in the shoe, providing a strong basis for subsequent analysis and evaluation of the shoe condition.

[0058] Among them, the multi-branch deep neural network in the network architecture module 21 uses a long short-term memory network to extract the time series characteristics of the humidity and temperature sensor data, as follows:

[0059] The network architecture module 21 obtains the time series data of the humidity and temperature sensors through the acquisition processing unit 1, and arranges them in chronological order to have a uniform time interval. The long short-term memory network processes the data in time steps and requires ordered and equally spaced data to facilitate the network to accurately capture the changes in data over time and improve the accuracy and stability of feature extraction.

[0060] Assume that the LSTM network has two hidden layers, each with 64 neurons. The input dimension of the LSTM network is determined according to the number of features of the humidity and temperature data, which is 2 here. Since we are only extracting features, we can set it according to the length of the feature vector, that is, 32, to enhance the network's ability to learn and express data features.

[0061] Initialize the weight matrix and bias vector of the long short-term memory network to provide initial parameters for network training, so that the network can learn and adjust from different starting points, increasing the possibility of finding the global optimal solution.

[0062] The normalized humidity and temperature time series data are input into the long short-term memory network in time steps. For each time step t, its input gate i is calculated. t :i t =σ(W i ·[h t-1 , x t ]+b i ), where W i is the weight matrix of the input gate, h t-1 is the hidden state of the previous time step, x t is the input data of the current time step, i.e., a vector consisting of temperature and humidity, b i is the bias vector of the input gate, and σ is the activation function, which controls the degree to which the current input information enters the cell state, accurately filters information related to the current moment, highlights the impact of important data, and reduces redundant information interference.

[0063] Calculate the forget gate f t :f t =σ(W f ·[h t-1 , x t ]+bf );W f and b f It is the weight matrix and bias vector of the forget gate, which is used to decide whether to retain or discard the cell state information at the previous moment, effectively retain the key information in the long-term dependency relationship, maintain information continuity, and avoid the loss of key information.

[0064] Compute cell state update candidates Where W C and b C are the relevant weight matrices and bias vectors, and tanh is the hyperbolic tangent activation function, which is used to generate the update content of the current input that may affect the cell state, provide flexible cell state update candidates, enrich the source of cell state updates, and enhance information integration capabilities.

[0065] Update cell status in Represents element-by-element multiplication, which is used to combine the results of the forget gate and the input gate to update the cell state, reasonably integrate new and old information, and make the cell state effectively reflect the long-term dependence and current changes in the data sequence.

[0066] Calculate the output gate o t :o t =σ(W o ·[h t-1 , x t ]+b o );W o and b o It is the weight matrix and bias vector of the output gate, which is used to control the ratio of cell state information output to the hidden state, selectively output useful information for the current task, focus on key features, and improve the pertinence of feature expression.

[0067] Calculate the hidden state at the current time step It is used to generate comprehensive information of the current time step to represent the fused cell state and output gate information, and obtain the hidden state containing time series information for subsequent calculation or feature extraction.

[0068] Repeat the above calculation process until the entire time series data is processed, and the cell state C of the long short-term memory network is t , the network can store the long-term dependency of past humidity and temperature data, and the hidden state h t The current input and memory information are comprehensively represented at each time step. The unique structure of the long short-term memory network realizes long-term memory, which can process long sequence data without losing key dependencies and accurately capture complex patterns and long-term trends in time series.

[0069] After processing the entire time series data, the hidden state h of the last time step can be TAs the time series feature vector of the extracted humidity and temperature sensor data, where T is the total length of the time series, the expression of the time series feature vector is where h pool It is the time series feature vector, and the hidden state contains the time series features. The feature representation is obtained simply and effectively, and the humidity and temperature time series features that can be used for subsequent analysis are obtained.

[0070] This feature vector will be used as the time series feature representation of the humidity and temperature sensor data, and will be used for subsequent fusion with the features extracted from the pressure sensor data and motion posture data, as well as the classification and analysis of the overall in-shoe condition. Multi-sensor data fusion can comprehensively evaluate the in-shoe condition, improve the evaluation accuracy by integrating multi-source information, and achieve more accurate in-shoe condition monitoring and classification.

[0071] Among them, the multi-branch deep neural network in the network architecture module 21 uses a three-dimensional convolutional neural network combined with an optical flow algorithm to extract motion spatiotemporal features, as follows:

[0072] Collect video sequences or image sequences of motion posture data to ensure that the images have a uniform size for easy subsequent processing and analysis. Unified parameters can ensure the stable operation of the algorithm, reduce the computational complexity caused by differences in data specifications, and improve feature extraction efficiency and accuracy.

[0073] Apply the optical flow algorithm to adjacent frame images to calculate the optical flow field, as follows:

[0074] Let I(x, y, t) represent the grayscale value of the pixel at coordinate (x, y) at time t. The optical flow algorithm is based on the constant brightness assumption I(x, y, t) = I(x+u, y+v, t+1); where u and v are the optical flow components in the x and y directions, respectively. The motion information is estimated by calculating the displacement of pixels between adjacent frames. The optical flow can intuitively reflect the motion of objects between adjacent frames, provide detailed local motion information, and supplement key motion data for subsequent feature extraction.

[0075] Design a three-dimensional convolutional neural network structure and determine the number of layers L of the network and the number of convolution kernels K in each layer l , Size C l ×C l ×T l , where C l Indicates the size of the space dimension, T l Indicates the size of the time dimension and the step size S l ,According to the spatiotemporal characteristics of motion posture data, a suitable network architecture is designed, and convolution kernels of different scales can extract multi-scale spatiotemporal features and enhance the network's learning ability of motion spatiotemporal features.

[0076] Initialize the weight parameters of the 3D convolutional neural network And bias parameters, l represents the number of layers, k represents the kth convolution kernel, which provides a starting point for network training. Network training requires initial parameters. Random initialization can prevent the network from falling into the local optimum, increase the probability of finding the global optimal solution, and enable the network to learn features better.

[0077] The optical flow field data O(x, y, t) and the original image sequence data I(x, y, t) are used as the input of the 3D convolutional neural network. For each 3D convolutional layer l:

[0078] The convolution kernel slides on the three-dimensional data to perform convolution operations, calculate the weighted sum of the area covered by each convolution kernel and the corresponding data, and add the bias. Suppose the input data is X l (x, y, t), the kth convolution kernel Output feature map The calculation formula is:

[0079]

[0080] Through the convolution operation, local spatiotemporal features are extracted, and the characteristic patterns in the data are automatically learned to obtain a representative local spatiotemporal feature map. The convolution result is activated by a nonlinear activation function to enhance the nonlinear expression ability of the network. The activated feature map is but Increase the network's ability to process complex data relationships, enable the network to learn more abstract features, and improve the richness and accuracy of feature expression.

[0081] After multiple layers of convolution operations, as the number of network layers increases, the feature map gradually integrates more advanced spatiotemporal feature information, and has a more comprehensive representation of the spatiotemporal changes of the entire motion posture sequence. Suppose the feature map output by the last convolutional layer of the network is F(x, y, t), which integrates the feature extraction and abstraction results of the previous layers. The network hierarchical structure can gradually abstract features, understand motion postures from local to global, and obtain feature representations containing rich spatiotemporal information for subsequent analysis or classification tasks.

[0082] The feature map F(x, y, t) output by the last convolutional layer of the network is used as the extracted spatiotemporal motion features of the motion posture. The feature map of the last layer of the network contains high-level abstract features, which simply and directly obtains feature representation, and obtains spatiotemporal motion features that can be used for subsequent fusion with other sensor data features and analysis of the conditions inside the shoe. It is used for subsequent fusion with features extracted from pressure sensor data and humidity and temperature sensor data, as well as classification and analysis of the overall conditions inside the shoe.

[0083] The feature fusion module 22 uses the attention mechanism to perform adaptive weighted fusion on the extracted features, as follows:

[0084] Suppose the feature vectors of pressure sensor data, humidity and temperature sensor data, and motion posture data extracted from the network architecture module 21 are F p 、F h and F m , whose dimensions are d p ,d h ,d m , these feature vectors are concatenated in order to form a total feature vector F = [F p , F h , F m ], the dimension is d=d p +d h +d m , encode the classification label of the shoe condition. Assume there are C categories. Use one-hot encoding to represent the label y as a C-dimensional vector, where only the position corresponding to the true category is 1 and the rest are 0. The classification label is converted into a numerical form that the model can handle, which is convenient for calculating the correlation between features and labels, so that the label information can be effectively integrated into the attention mechanism calculation.

[0085] The feature fusion module 22 introduces a trainable fully connected layer W s and the bias vector b s , by calculating S = F × W s +b s Get a C-dimensional score vector S, where each dimension S i It represents the correlation score between the corresponding feature and the i-th classification label. By learning the relationship between features and labels through the fully connected layer, it can automatically capture complex associations and accurately quantify the importance of features to different classifications.

[0086] Apply the softmax function to normalize the score vector S to obtain the attention weight vector A, that is, This makes each element of the weight vector A between 0 and 1, and each weight A i It represents the relative importance of the corresponding feature under the i-th classification label. Normalization ensures reasonable distribution of weights, highlights relatively important features and facilitates weighted fusion, providing a scientific weight basis for feature fusion.

[0087] The total feature vector F is weighted summed with the attention weight vector A to obtain the final fusion feature vector F fusion , the specific calculation is: Features are fused based on weights to strengthen the influence of key features and improve the overall representation ability of features, which is beneficial for classification.

[0088] During the training process of the entire shoe condition monitoring model, the fused feature vector F fusionInput to the subsequent classification layer, and use the back propagation algorithm to update the fully connected layer W according to the loss between the predicted result and the true label y s The weight and bias b s Through continuous training iterations, the attention mechanism can more accurately learn the correlation between features and classification labels, thereby generating more reasonable attention weights and improving the accuracy and robustness of the entire model in classifying shoe conditions.

[0089] When the feature fusion module 22 uses the cross entropy loss function to measure the difference between the model prediction and the true label, a regularization term is added, as follows:

[0090] Assume the model prediction output is The probability distribution vector of each category obtained after the softmax function processing has a dimension of C, where C is the number of categories of the shoe condition classification. The true label is y. First, the basic part of the cross entropy loss function is calculated. Cross entropy can measure the difference between the predicted probability distribution and the true label distribution. It is a commonly used loss metric in classification tasks. It intuitively reflects the accuracy of model predictions, prompts the model to learn the correct classification decision boundary, provides a core optimization goal for model training, and guides the model parameters to be updated in the direction of reducing prediction errors.

[0091] Select the L2 regularization term, and set the model parameters to θ, where θ includes the weights and biases of each layer of the neural network, the regularization coefficient is λ, λ>0, and the regularization term is used to prevent the model from overfitting. L2 regularization constrains the parameters so that the model parameters will not be too large, thereby avoiding the model from overfitting the training data, improving the generalization ability of the model, and enabling the model to perform well on unseen data, thereby enhancing the stability and adaptability of the model and reducing the risk of performance degradation due to overfitting.

[0092] Calculate the L2 regularization term The sum here is over all the parameters θ of the model j It constrains the sum of squares of model parameters, limits the range of parameter values, uniformly penalizes all parameters, avoids some parameters being too large and dominating the model, promotes the model to learn a smoother decision function, makes the model more robust, reduces sensitivity to noise data, and improves prediction accuracy under different data distributions.

[0093] Add the cross entropy loss function to the regularization term to get the final loss function When optimizing the model, it is necessary to consider both the prediction accuracy and the generalization ability of the model. Combining the two in one loss function can balance the two goals during the training process. By adjusting the regularization coefficient, the model can flexibly control the trade-off between fitting the training data and generalizing to new data, and obtain a loss function that comprehensively considers the prediction error and model complexity. It can train an in-shoe condition monitoring model that is both accurate and has good generalization performance, thereby improving its reliability and effectiveness in practical applications.

[0094] During the model training process, based on the final loss function L, the back-propagation algorithm is used to calculate the gradient of the model parameters θ, and the parameters are updated according to the gradient descent algorithm to minimize the loss function L, thereby continuously optimizing the performance of the model so that it can more accurately predict and have stronger generalization capabilities in the task of shoe condition classification.

[0095] When the parameter adjustment module 23 uses the gradient descent algorithm to adjust the parameters of the deep neural network, an adaptive learning rate adjustment strategy is adopted, as follows:

[0096] First, the initial learning rate η0 is set, which is the step size of the parameter update at the beginning of model training. At the same time, the decay rate γ of the learning rate adjustment and the adjustment step period T are determined, which means that the learning rate is adjusted every T training batches to provide the initial parameter update amplitude for model training, and determine the basic rules and rhythm of subsequent learning rate adjustments. A larger learning rate is given at the beginning of training to quickly approach the optimal solution area, and then it is gradually reduced to avoid skipping the optimal solution, making the model training process smoother and more efficient, and balancing the convergence speed and accuracy.

[0097] During the training process, the learning rate is updated according to the set period. Let the current training round or batch index be t, then the current learning rate in It is expressed as a rounding-down operation. As the training progresses, the learning rate is gradually reduced according to the preset attenuation rule to adapt to the different requirements of the model for parameter update amplitudes at different training stages. The learning rate is automatically adjusted to reduce the workload and complexity of manual parameter adjustment. At the same time, it can make the model parameter update more refined in the later stage of training, improve the stability and convergence of model training, and avoid parameter update oscillation or failure to converge to the optimal solution due to excessive learning rate.

[0098] At each parameter update, the gradient of the current learning rate and loss function with respect to the model parameters is calculated. θ L, according to the gradient descent algorithm formula θ t+1 =θ t -η t ▽ θL is used to update the model parameters, and the adaptively adjusted learning rate is used to control the step size of the parameter update, so that the parameters are gradually optimized in the direction of reducing the loss function. The combination of appropriate learning rate and gradient information can effectively adjust the model parameters at different training stages, accelerate the model to converge to a better solution, and enable the model to flexibly adjust parameters according to the actual situation during training, improve the performance of the model in the task of shoe condition monitoring, learn the effective features and patterns in the data more quickly, and improve the accuracy and generalization ability of classification.

[0099] By adopting this adaptive learning rate adjustment strategy, the model can better balance the convergence speed and accuracy during the training process, avoiding the problems of low training efficiency or poor model performance caused by a fixed learning rate, thereby effectively improving the training effect and application performance of the entire in-shoe condition monitoring model, and outputting the classification results and quantitative evaluation of the current in-shoe condition.

[0100] The early warning feedback unit 3 is used to receive the quantitative evaluation results and provide health early warning information to the user according to the output results of the deep neural network.

[0101] In the present invention, the data inside the shoe is collected by the acquisition and processing unit 1 using a multimodal sensor for pressure, temperature, humidity and inertial measurement, and linear regression processing is performed. The model analysis unit 2 analyzes the sensor data through a multi-branch deep neural network architecture, and extracts the spatial characteristics of pressure distribution, the time series characteristics of humidity and temperature, and the spatiotemporal characteristics of motion respectively. The attention mechanism is weighted fusion, and historical data is used for training. The cross entropy loss function and the adaptive learning rate adjustment strategy are used to optimize the parameters, and accurate classification results and quantitative evaluations are output. The early warning feedback unit 3 provides different early warning methods according to the evaluation results. The present invention can comprehensively and real-time monitor the conditions inside the shoe and improve the accuracy of monitoring the conditions inside the shoe.

[0102] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments. The above embodiments and descriptions are only preferred examples of the present invention and are not intended to limit the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A shoe condition monitoring system based on sensor technology, characterized in that: It comprises a collection and processing unit (1), a model analysis unit (2) and an early warning feedback unit (3); The acquisition and processing unit (1) acquires the in-shoe condition data through a multi-modal sensing sequence, and processes the in-shoe condition data using a linear regression algorithm; The model analysis unit (2) analyzes the pressure sensor data, humidity and temperature sensor data and motion posture data in the shoe through a multi-branch deep neural network architecture, and extracts pressure distribution spatial features, time series features and motion spatiotemporal features respectively, and uses an attention mechanism to adaptively weight and fuse the extracted features to obtain a fused feature vector, and then uses historical in-shoe condition data as a training set, uses the fused feature vector as a model input, and uses the classification label of the in-shoe condition as a target output, and uses a cross entropy loss function to measure the difference between the model prediction and the true label. Finally, the gradient descent algorithm is used to adjust the deep neural network parameters according to the difference value, and the real-time collected in-shoe condition data is input into the deep neural network for analysis, and the classification result and quantitative evaluation of the current in-shoe condition are output; The early warning feedback unit (3) is used to receive the quantitative evaluation results and provide health early warning information to the user according to the output results of the deep neural network.

2. The in-shoe condition monitoring system based on sensor technology according to claim 1, characterized in that: The multimodal sensing sequence in the acquisition and processing unit (1) includes a pressure sensor, a temperature and humidity sensor, and an inertial measurement module, as follows: Pressure sensors, humidity sensors, temperature sensors and inertial measurement modules are arranged in the shoe according to a preset layout, and data from each sensor is synchronously collected at a fixed frequency to form shoe condition data; For the pressure sensor data, a relationship model between pressure and sole deformation is established through linear regression. For the humidity and temperature sensor data, linear regression is used to analyze the correlation between humidity and temperature and the changing trend over time.

3. The in-shoe condition monitoring system based on sensor technology according to claim 2 is characterized in that: The model analysis unit (2) comprises a network architecture module (21), and the multi-branch deep neural network in the network architecture module (21) processes the pressure sensor data and extracts the pressure distribution spatial characteristics in the following manner: Collecting pressure sensor data, dividing it into regions according to the forefoot, midfoot and heel, and for each region, converting the pressure data into a two-dimensional pressure data matrix according to a preset resolution, wherein each element in the matrix represents the pressure value at the corresponding position; Construct a spatial convolutional neural network with three convolutional layers. The first layer uses a 3*3 convolution kernel, the second layer uses a 5*5 convolution kernel, and the third layer uses a 7*7 convolution kernel. Initialize the weight parameters and bias parameters of the convolution kernel. The constructed pressure data matrix is ​​input into the first convolution layer of the network. The 3*3 convolution kernel slides on the matrix with a preset step size to perform convolution operation. For the local area covered by each convolution kernel, its weighted sum is calculated and a bias is added to obtain the convolution result corresponding to the area. The convolution result is activated using a preset activation function to obtain the first convolution feature map. The first convolution feature map is passed as input to the second convolution layer, the 5*5 convolution kernel repeats the above operation to obtain the second convolution feature map, the second convolution feature map is used as input to enter the third convolution layer, the 7*7 convolution kernel performs convolution operation to obtain the third convolution feature map; The feature maps obtained by convolution kernels of different scales are weightedly fused to obtain the spatial characteristics of pressure distribution.

4. The in-shoe condition monitoring system based on sensor technology according to claim 3 is characterized in that: The multi-branch deep neural network in the network architecture module (21) uses a long short-term memory network to extract the time series features of the humidity and temperature sensor data, as follows: Arrange the data collected by the humidity and temperature sensors in chronological order, normalize them, and map the humidity and temperature data to [0,1] respectively; Determine the structural parameters of the LSTM network, including the number of hidden layers, the number of neurons in each hidden layer, and the dimensions of input and output, and initialize the weight matrix and bias vector of the LSTM network; The normalized humidity and temperature time series data are input into the long short-term memory network in sequence according to the time step. For each time step, the input gate, forget gate, output gate and the hidden state of the current time step are calculated. The above calculation process is repeated until the entire time series data is processed. The hidden state of the last time step is used as the time series feature vector of the extracted humidity and temperature sensor data.

5. The in-shoe condition monitoring system based on sensor technology according to claim 4 is characterized in that: The multi-branch deep neural network in the network architecture module (21) uses a three-dimensional convolutional neural network combined with an optical flow algorithm to extract motion spatiotemporal features, as follows: Collect image sequences of motion posture data, apply optical flow algorithm to adjacent frame images to calculate optical flow field, and obtain motion information; Set the structure of the three-dimensional convolutional neural network, that is, determine the number of layers of the network, the number, size and step size of the convolution kernels of each layer, initialize the weight parameters and bias parameters of the three-dimensional convolutional neural network, and provide a starting point for network training; The optical flow field data and the original image sequence data are used as the input of the 3D convolutional neural network, where for each 3D convolutional layer: The convolution kernel slides on the three-dimensional data to perform convolution operations, calculates the weighted sum of the area covered by each convolution kernel and the corresponding data, and adds a bias to obtain the convolution result. The nonlinear activation function is then applied to activate the convolution result to obtain the feature map. The feature map output by the last convolution layer of the network is used as the motion spatiotemporal features of the extracted motion posture.

6. The in-shoe condition monitoring system based on sensor technology according to claim 5 is characterized in that: The model analysis unit (2) includes a feature fusion module (22), and the feature fusion module (22) uses an attention mechanism to perform adaptive weighted fusion on the extracted features, specifically as follows: Extracting feature vectors and corresponding feature dimensions of pressure sensor data, humidity and temperature sensor data, and motion posture data from the network architecture module (21), sequentially concatenating these feature vectors into a total feature vector, and encoding the classification label of the condition inside the shoe; A fully connected layer and a bias vector are introduced, and the score vector is obtained by calculation through the formula. The score vector is normalized to obtain the attention weight vector. The total feature vector and the attention weight vector are weighted summed to obtain the final fused feature vector.

7. The in-shoe condition monitoring system based on sensor technology according to claim 6 is characterized in that: The feature fusion module (22) adds a regularization term when using the cross entropy loss function to measure the difference between the model prediction and the true label, as follows: The fused feature vector is used as the input of the model, and the model prediction output is set to The true label is y. First, the basic part of the cross entropy loss function is calculated, that is, Where C is the number of categories for the shoe condition classification; Assume the model parameter is θ, the regularization coefficient is λ, and calculate the L2 regularization term Add the cross entropy loss function to the regularization term to get the final loss function, that is: L = L CE +L reg .

8. The in-shoe condition monitoring system based on sensor technology according to claim 7 is characterized in that: The model analysis unit (2) comprises a parameter adjustment module (23). When the parameter adjustment module (23) uses a gradient descent algorithm to adjust the parameters of the deep neural network, it adopts an adaptive learning rate adjustment strategy, which is specifically as follows: Set the initial learning rate, decay rate, and adjust the step size period, and update the learning rate according to the set period during training; At each update, the model parameters are updated according to the gradient descent algorithm based on the calculated current learning rate and the gradient of the loss function with respect to the model parameters.

Images