Foot deformation prediction method, foot deformation prediction system, and foot health management system comprising same

Abstract

A foot deformation prediction system according to the technical idea of the present disclosure comprises: a communication unit for receiving a foot image of a user from a user terminal; an image conversion unit for generating an input mesh image corresponding to the foot image; and an artificial intelligence module configured to generate an input graph on the basis of the input mesh image and generate deformation prediction information corresponding to the foot image on the basis of coordinate values of nodes included in the input graph, wherein the deformation prediction information includes pressure information received by each of the nodes and displacement information indicating the degree to which the coordinate values of each of the nodes are changed by the pressure information received by each of the nodes.

Description

Method for predicting foot deformation, system for predicting foot deformation, and foot health management system including the same

[0001] The technical idea of ​​the present disclosure relates to foot deformation prediction, and more specifically, to a method and system for predicting foot deformation using a deep learning module, and a foot health management system including the same. This application claims priority from Korean Patent Application No. 10-2024-0015358, filed on January 31, 2024. The entire contents of this priority application are incorporated herein by reference.

[0002] The process of customizing shoes to fit the shape of a person's foot has traditionally been performed offline, by hand. Foot analysis typically involves taking a foot print of each individual's foot, limiting its ability to be performed online.

[0003] To overcome the limitations of offline methods, attempts have been made to capture foot images with a camera and transmit them online to determine foot shape. However, this method suffers from the problem of not fully reflecting the information necessary for custom shoemaking. Specifically, it fails to account for changes in foot shape and the pressure distribution on the sole that occur when a person stands or walks.

[0004] Therefore, there is a need for technology that can reduce the significant time and cost required for foot analysis while sufficiently reflecting the information required for custom shoe production.

[0005] Background technology related to this is Korean Patent Publication No. 10-2020-0130970.

[0006] According to an embodiment, a method and system for predicting foot deformation are provided, which predicts personalized foot deformation based on a foot image and obtains a foot image after deformation.

[0007] In addition, a foot health management system is provided that generates various information for managing the user's foot based on foot deformation prediction information.

[0008] However, the problems to be solved by the present invention are not limited to those mentioned above, and other problems to be solved that are not mentioned can be clearly understood by a person having ordinary knowledge of the present invention from the description below.

[0009] A method for predicting foot deformation performed by an artificial intelligence module according to a first aspect comprises: a step in which an encoder in the artificial intelligence module generates an input graph from an image of the foot before deformation and provides the graph to a pre-trained deep learning model in the artificial intelligence module; a step in which the pre-trained deep learning model outputs an output graph in which the deformation prediction of the foot is reflected in the input graph; and a step in which a decoder in the artificial intelligence module generates and outputs a post-deformation image in which the deformation prediction of the foot is reflected from the output graph; wherein the deep learning model is pre-trained to output an output graph of a post-deformation image in which the deformation prediction of the foot is reflected for at least one node constituting the foot when an input graph of the pre-deformation image is input.

[0010] A foot deformation prediction system according to a second aspect includes a communication unit that receives a user's foot image from a user terminal, a mesh image generation unit that generates an input mesh image corresponding to the foot image, an artificial intelligence module configured to generate an input graph based on the input mesh image, and generate deformation prediction information corresponding to the foot image based on coordinate values ​​of nodes included in the input graph using a graph neural network, wherein the deformation prediction information includes pressure information received by each node and displacement information indicating a degree to which the coordinate values ​​of each node change due to the pressure information received by each node.

[0011] A foot health management system according to a third aspect includes a foot deformation prediction system which receives a foot image of a user from a user terminal, and generates foot deformation prediction information including pressure information for each node and displacement information regarding the degree to which the coordinate values ​​of each node are changed by the pressure information based on coordinate values ​​of nodes included in an input graph corresponding to the foot image using a graph neural network, and a foot deformation information utilization system which provides foot deformation prevention information including an exercise program, treatment method, or insole drawing information corresponding to the user's foot to the user terminal based on the foot deformation prediction information.

[0012] According to an embodiment of the present disclosure, a foot health management system can be provided that generates personalized foot deformation prediction information based on a foot image and generates various information for managing a user's foot based on the foot deformation prediction information.

[0013] FIG. 1 is a drawing illustrating a foot health management system according to an exemplary embodiment of the present disclosure.

[0014] FIG. 2 is a diagram illustrating a foot deformation prediction system according to an exemplary embodiment of the present disclosure.

[0015] FIG. 3 is a drawing illustrating a method for predicting foot deformation according to an exemplary embodiment of the present disclosure.

[0016] FIG. 4 is a flowchart illustrating a method for predicting foot deformation according to an exemplary embodiment of the present disclosure.

[0017] FIG. 5 is a block diagram illustrating an insole manufacturing system according to an exemplary embodiment of the present disclosure.

[0018] FIG. 6 is a flowchart illustrating a method for manufacturing an insole according to an exemplary embodiment of the present disclosure.

[0019] FIG. 7 is a drawing illustrating an exercise program providing system according to an exemplary embodiment of the present disclosure.

[0020] FIG. 8 is a drawing illustrating a medical information providing system according to an exemplary embodiment of the present disclosure.

[0021] The advantages and features of the present invention, and the methods for achieving them, will become clearer with reference to the embodiments described below together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below and may be implemented in various different forms. These embodiments are provided solely to ensure that the disclosure of the present invention is complete and to fully inform those skilled in the art of the scope of the invention, and the present invention is defined solely by the scope of the claims.

[0022] The terms used in this invention have been selected from widely used, current terms, taking into account the functions of the invention. However, these terms may vary depending on the intentions of those skilled in the art, precedents, the emergence of new technologies, etc. Furthermore, in certain cases, terms may be arbitrarily selected by the applicant, in which case their meanings will be described in detail in the relevant description of the invention. Therefore, the terms used in this invention should not be defined simply as names, but rather based on their inherent meanings and the overall content of the invention.

[0023] When a part of a specification is said to 'include' a component, this does not mean that it excludes other components, but rather that it may include other components, unless otherwise stated.

[0024] Also, the term 'part' used in the specification means a software or hardware component such as an FPGA or ASIC, and the 'part' performs certain functions. However, the 'part' is not limited to software or hardware. The 'part' may be configured to reside on an addressable storage medium or may be configured to play one or more processors. Thus, as an example, the 'part' includes components such as software components, object-oriented software components, class components, and task components, as well as processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. The functionality provided within the components and 'parts' may be combined into a smaller number of components and 'parts' or further separated into additional components and 'parts'.

[0025] Hereinafter, various embodiments of the present invention are described with reference to the attached drawings.

[0026] FIG. 1 is a drawing illustrating a foot health management system according to an exemplary embodiment of the present disclosure.

[0027] Referring to FIG. 1, a foot health management system (1) may include a user terminal (100), a network (200), a foot deformation prediction system (300), and a foot deformation information utilization system (400).

[0028] The user terminal (100) can capture an image of the user's foot. Furthermore, the user terminal (100) can capture a video of the user's foot and a video of the user's walking. The user terminal (100) can provide the foot image, the foot video, or the walking video to the foot deformation prediction system (300) via the network (200).

[0029] The foot deformation prediction system (300) can predict the deformation of the user's foot based on a foot image, a foot video, or a walking video.

[0030] Specifically, the foot deformation prediction system (300) can generate a 3D model of a user's foot based on a foot image or a foot video, and can generate an input graph for inputting to an artificial intelligence model based on the 3D model. In an exemplary embodiment of the present disclosure, a GNN (Graph Neural Network) model may be used as the artificial intelligence model, but is not limited thereto. The graph may include a plurality of nodes and a plurality of edges. The nodes of the graph may be generated based on points of a point cloud or mesh image corresponding to the 3D model. Edges between nodes may be generated based on whether points are adjacent to each other. That is, the plurality of nodes and the plurality of edges in the input graph may represent the structure of the foot. Each node may be expressed as a coordinate value and a node type. The coordinate value may indicate the location of the corresponding node on the foot, and the node type may indicate which part of the foot the corresponding node is. For example, the node type may indicate whether the corresponding node corresponds to the heel of the foot, a specific toe, or the sole of the foot.

[0031] The foot deformation prediction system (300) can generate an output graph including foot deformation prediction information using an artificial intelligence model (e.g., a GNN model). The foot deformation prediction information can include pressure information and displacement information for each of a plurality of nodes.

[0032] Pressure information may represent the pressure received by each node in the structure of the foot. The pressure information may include at least one of the pressure received by the node from the ground, the pressure received by the node from an adjacent node, the pressure received by the node from the user's walking pattern, and the pressure received by the node from the user's weight.

[0033] Displacement information can indicate how much the coordinate values ​​of each node change when each node is continuously subjected to that pressure.

[0034] The foot deformation prediction system (300) may store various learning models according to the time at which a node receives pressure, and may output a plurality of output graphs including a plurality of pieces of displacement information according to the time at which the node receives pressure. For example, the foot deformation prediction system (300) may generate a first output graph including first displacement information indicating the degree to which the coordinate value of the node changes when the time at which the node receives pressure is a first time, and may generate a second output graph including second displacement information indicating the degree to which the coordinate value of the node changes when the time at which the node receives pressure is a second time. That is, the foot deformation prediction system (300) may provide the degree to which the foot is deformed over time through the output graph. The GNN may be learned in advance based on an input graph including nodes having coordinate values ​​and type values, and output graphs including nodes having pressure values ​​and displacement values.

[0035] The foot deformation information utilization system (400) can provide various information for managing the user's foot using foot deformation prediction information.

[0036] For example, the foot deformation information utilization system (400) may include an exercise program provision system (410), a medical information provision system (420), and an insole manufacturing system (430). The foot deformation information utilization system (400) may provide foot deformation prevention information including an exercise program for preventing foot deformation, a treatment method, or insole drawing information corresponding to the user's foot to a user terminal based on foot deformation prediction information.

[0037] The exercise program provision system (410) can provide an exercise program suitable for the user's foot based on foot deformation prediction information. Specifically, the exercise program provision system (410) can identify nodes where the pressure is greater than a first reference value and the displacement due to the pressure, i.e., the change in coordinate value, is greater than a second reference value. The exercise program provision system (410) can store massage methods, walking methods, management methods, etc. for each part of the foot.

[0038] The exercise program providing system (410) can provide the user terminal (100) with an exercise program for managing the foot portion corresponding to the identified node. For convenience of explanation, the exercise program providing system (410) has been described as identifying a node in which the pressure is greater than or equal to a first reference value and the displacement due to the pressure, i.e., the change in the coordinate value, is greater than or equal to a second reference value. However, the embodiment is not limited thereto. That is, the exercise program providing system (410) can divide the node into various stages according to the pressure and displacement, and provide different exercise programs for each stage.

[0039] The medical information provision system (420) can provide medical information suitable for the user's foot based on foot deformation prediction information. Specifically, the medical information provision system (420) can identify nodes where the pressure is greater than a first reference value and the displacement due to the pressure, i.e., the change in coordinate value, is greater than a second reference value. The medical information provision system (420) can store treatment methods for preventing deformation of each part of the foot.

[0040] The medical information provision system (420) can provide medical information for treating the foot portion corresponding to the identified node to the user terminal (100). For convenience of explanation, the medical information provision system (420) has been described as identifying a node in which the pressure is greater than or equal to a first reference value and the displacement due to the pressure, i.e., the change in the coordinate value, is greater than or equal to a second reference value. However, the embodiment is not limited thereto. That is, the medical information provision system (420) can divide the node into various stages according to the pressure and displacement, and provide different medical methods to the user terminal (100) for each stage.

[0041] The insole manufacturing system (430) can provide medical information suitable for the user's foot based on foot deformation prediction information. Specifically, the insole manufacturing system (430) can identify nodes where the pressure is greater than or equal to a first reference pressure value and where the displacement due to the pressure, i.e., the change in coordinate value, is greater than or equal to the first reference displacement value. The insole manufacturing system (430) can store an insole pattern for distributing pressure across each foot part.

[0042] The insole manufacturing system (430) can generate an insole drawing based on an insole pattern for distributing pressure on a foot portion corresponding to an identified node. That is, the insole drawing includes a first insole pattern having a shape of a user's sole, and may partially include a second insole pattern corresponding to a foot portion corresponding to an identified node. The first insole pattern may be referred to as a main structure, and the second insole pattern may be referred to as an independent structure. For convenience of explanation, the nodes are identified using a first reference pressure value and a first reference displacement value, but the nodes may also be identified using various reference pressure values ​​and reference displacement values. Accordingly, independent structures having locally varying densities and elasticities may be arranged in the insole drawing.

[0043] The first insole pattern may be formed by filling a repeating, regular pattern into the shape of the user's sole. The repeating, regular pattern may include a hexagonal honeycomb pattern, a pattern of repeating polygons such as triangles, or a star-shaped pattern. Alternatively, the first insole pattern may be formed by filling an irregular pattern that does not have a regular shape. Thereafter, the insole manufacturing system (430) may form a second insole pattern by reflecting foot deformation prediction information into the shape of the sole formed by the first insole pattern. For example, a portion of the foot that experiences a large displacement due to pressure may be formed with a second insole pattern that has a higher density and lower elasticity than the first insole pattern. A portion of the foot that receives a high pressure compared to the surrounding area may be formed with a second insole pattern that has a lower density and higher elasticity than the first insole pattern. That is, in order to disperse the pressure, the second insole pattern may have a lower density than the first insole pattern. Alternatively, the second insole pattern may have higher elasticity than the first insole pattern. The above-described first and second insole pattern generation methods are merely examples, and various other methods can be used to generate the insole patterns by reflecting displacement and pressure information from the foot deformation prediction information. Furthermore, referring to FIG. 5 described below, the generation of the above-described insole patterns can be performed by the pattern generation unit (433).

[0044] The insole manufacturing system (430) can acquire the shape of the user's sole using a mesh image (510) before inputting it into the deep learning model (332) to generate an insole drawing, and can generate a first insole pattern based on the acquired shape of the sole. Since the first insole pattern is generated based only on the shape of the user's sole, the shape of the sole deformed when pressure is applied or the insole pattern information for effective pressure distribution when the user is walking may not be reflected. Therefore, the insole manufacturing system (430) can additionally reflect the pressure information and displacement information of the output graph output through the deep learning model (332) to the first insole pattern, so as to generate a second insole pattern so that more effective pressure distribution can be achieved when pressure is applied to the user's sole.

[0045] Alternatively, the insole manufacturing system (430) can extract the user's sole shape, pressure information, and displacement information from the output graph output from the deep learning model (332), and generate the first insole pattern and the second insole pattern based on the extracted information. That is, the first insole pattern can be generated based on the sole shape in a state where pressure is applied. In this case, the insole pattern can be generated based on the expected foot shape when the insole is worn, compared to the case where the first insole pattern is generated from the foot shape of the mesh image (510) before being input to the deep learning model (332). Therefore, there is an effect of being able to manufacture the insole by more accurately reflecting the foot shape in a state where pressure is applied to the foot.

[0046] The insole manufacturing system (430) can determine the mesh image to be used when generating the first insole pattern based on the amount of change in the shape of the foot before and after deformation. More specifically, the insole manufacturing system (430) can use at least one image of the mesh image (510) before being input to the deep learning model (332) or the mesh image (550) after being output, to generate the first insole pattern, based on the amount of change in the shape of the foot. Depending on the user, there are cases where the degree of deformation is relatively small even when pressure is applied to the foot. In this case, the mesh image (510) generated based on an actually photographed image can more accurately reflect the shape of the actual user's foot than the mesh image (550) predicted by the deep learning model (332). This is because, if the degree of deformation due to pressure is small, considering the error occurring in the prediction process of the deep learning module (332), even if the shape of the foot deformed by pressure is not reflected, reducing the prediction error of the deep learning module (332) can reflect a more accurate foot shape.

[0047] Accordingly, the insole manufacturing system (430) can determine the first insole pattern based on the mesh image (550) of the foot predicted through the deep learning model (332) when the amount of change in the shape of the foot is greater than a preset reference value. Conversely, when the amount of change in the shape of the foot is less than a preset reference value, the first insole pattern can be determined based on the mesh image (510) before being input to the deep learning model (332).

[0048] According to exemplary embodiments of the present disclosure, a GNN model can be used to predict foot deformation by considering various structural characteristics of the foot, thereby providing personalized information to the user. Specifically, by predicting foot deformation based on structural characteristics of the foot, personalized exercise and rehabilitation programs can be provided, and effective treatment methods can be provided to patients. Furthermore, by providing customized insoles by considering pressure and displacement, personalized insoles can be provided to the user.

[0049] FIG. 2 is a diagram illustrating a foot deformation prediction system according to an exemplary embodiment of the present disclosure. FIG. 3 is a diagram illustrating a foot deformation prediction method according to an exemplary embodiment of the present disclosure.

[0050] Referring to FIG. 2, the foot deformation prediction system (300) may include a mesh image generation unit (310), a training unit (320), an artificial intelligence module (330), and a communication unit (340).

[0051] Referring to FIGS. 1 and 2, the communication unit (340) can receive a foot image from a user terminal (100) and output foot deformation prediction information.

[0052] Referring to FIGS. 2 and 3, the mesh image generation unit (310) can generate a mesh image (510) based on a foot image. The mesh image (510) can refer to a three-dimensional model for a foot image composed of points, lines, and planes. Specifically, the mesh image generation unit (310) can extract point cloud information based on the foot image. The point cloud information can include points that constitute the foot image. The mesh image generation unit (310) can generate a 3D CAD image based on the point cloud information, and generate the mesh image (510) based on the 3D CAD image.

[0053] The training unit (320) can train the artificial intelligence module (330) based on input graphs for various foot images. Specifically, the training unit (320) can provide the artificial intelligence module (330) with an input graph including nodes having coordinate values ​​and node types, and train the artificial intelligence module (330) so that an output graph including nodes having pressure information and displacement information is output from the artificial intelligence module (330).

[0054] Although not shown in the drawing, the training unit (320) can also train the artificial intelligence module (330) based on various user information in addition to the foot image. The user information may include, but is not limited to, the user's weight, gender, age, medical condition, body type, and walking habits.

[0055] The artificial intelligence module (330) may include an encoder (331), a deep learning model (332), and a decoder (333).

[0056] Referring to FIGS. 2 and 3, the encoder (331) may generate an input graph (520) based on a mesh image (510). The input graph (520) may include nodes (521) and edges (522). The nodes (521) may have corresponding coordinate values ​​and node types. The edges may indicate relationships between nodes, for example, whether nodes are adjacent to each other.

[0057] Referring to FIGS. 2 and 3, the deep learning model (332) can output an output graph (540) corresponding to the input graph (520) using learned parameters. The output graph (540) can include nodes (541) and edges (542). The nodes (541) can have pressure information and displacement information. The deep learning model (332) can be a graph neural network (GNN), but the embodiment is not limited thereto. For example, the deep learning model (332) can include one of a convolutional neural network (CNN), a recurrent neural network (RNN), a graph convolutional neural network (GCN), a long short-term memory (LSTM), a gated recurrent unit (GRU), a transformer, and a temporal conventional network (TCN).

[0058] In one embodiment, the artificial intelligence model may include a time series learning unit including one or more GNN layers, and a multi-class calculation unit including dense layers that predict multi-class probability values ​​for each time step. That is, the artificial intelligence model may learn coordinate values ​​and type values ​​appearing in the above-described input graph using the GNN layers to derive characteristics of time series data. Thereafter, the artificial intelligence model may predict multi-class probability values ​​for each time step, and may identify a time step with a high probability among them and output an output graph (540) including information on pressure information and displacement information.

[0059] The decoder (333) can convert the output graph (540) into a corresponding mesh image or 3D image (550). Specifically, the decoder (333) can generate a mesh image or 3D image (550) using color, size, and shape so that the pressure value and displacement value for each part of the foot are expressed based on the pressure information and displacement information of the node (541). That is, the decoder (330) can provide foot deformation prediction information including pressure information and displacement information.

[0060] The foot deformation prediction system (300) according to an exemplary embodiment of the present disclosure can predict foot deformation by considering the structure of the foot. Accordingly, it may be possible to provide personalized, optimized foot management information to the user terminal (100) using foot deformation prediction information.

[0061] FIG. 4 is a flowchart illustrating a method for predicting foot deformation according to an exemplary embodiment of the present disclosure. FIG. 4 may be described with reference to FIG. 1.

[0062] Referring to FIG. 4, the foot deformation prediction system (300) can obtain a foot image from the user terminal (100) (S401). Furthermore, the foot deformation prediction system (300) can further obtain a foot video and a user's walking video from the user terminal (100).

[0063] The foot deformation prediction system (300) can generate an input graph corresponding to the foot image (S402). Specifically, the foot deformation prediction system (300) can generate a mesh image corresponding to the foot image and generate an input graph including nodes and edges based on the mesh image. The nodes can have coordinate values ​​and node types indicating the location of each node in the foot image.

[0064] The foot deformation prediction system (300) can generate an output graph containing foot deformation prediction information by inputting an input graph to a pre-trained artificial intelligence model (S403). Specifically, the foot deformation prediction system (300) can generate an output graph in which each node contains pressure information and displacement information.

[0065] Pressure information may indicate the pressure each node experiences in the foot structure. The pressure information may include at least one of the following: pressure experienced by the node from the ground, pressure experienced by the node from adjacent nodes, pressure experienced by the node due to the user's walking pattern, and pressure experienced by the node due to the user's weight. Displacement information may indicate the degree to which the coordinate values ​​of each node change when the node continuously experiences the corresponding pressure.

[0066] The foot deformation prediction system (300) can store various learning models according to the time at which the node is subjected to pressure, and can also output multiple output graphs including multiple pieces of displacement information according to the time at which the node is subjected to pressure.

[0067] The foot deformation prediction system (300) can generate an output mesh image corresponding to the output graph (S404). Specifically, the foot deformation prediction system (300) can generate an output mesh image so that pressure information and displacement information are expressed using different colors, sizes, and shapes. In other words, the foot deformation prediction system (300) can generate an output mesh image that includes foot deformation prediction information.

[0068] Let us examine the foot deformation prediction method performed by the artificial intelligence module (330) by referring to the flow chart of Fig. 4 and the configuration diagram of Fig. 2 together.

[0069] According to an embodiment, a method for predicting foot deformation is such that an encoder (331) within an artificial intelligence module (330) generates an input graph from an image of the foot before deformation and provides the graph to a pre-trained deep learning model (332) within the artificial intelligence module (330) (S401, S402).

[0070] And, the pre-trained deep learning model (332) outputs an output graph in which the deformation prediction of the foot is reflected in the input graph. Here, the deep learning model (332) may be pre-trained to output an output graph of a post-deformation image in which the deformation prediction of the foot is reflected for at least one node constituting the foot when an input graph of an image before deformation is input (S403).

[0071] Then, the decoder (333) in the artificial intelligence module (330) generates and outputs a post-deformation image reflecting the deformation prediction of the foot from the output graph (S404).

[0072] Meanwhile, the deep learning model (332) may be trained using training data in which an input graph of mesh data corresponding to an image of the foot before deformation and an output graph of mesh data corresponding to an image of the foot after deformation are paired.

[0073] Additionally, the deformation prediction of the foot reflected in the input graph by the deep learning model (332) may include information on the pressure received by at least one node constituting the foot and information on the displacement of the node due to the pressure.

[0074] In addition, the deep learning model (332) may be learned by including at least one body information corresponding to the foot, such as weight, gender, age, disease, body type, and walking habits, and may further receive body information corresponding to the image of the foot before deformation when predicting the deformation of the foot. In this case, in step S403, an input graph and body information may be input together to the deep learning model (332), and an output graph reflecting the corresponding body information may be generated by the deep learning model (332).

[0075] FIG. 5 is a block diagram illustrating an insole manufacturing system according to an exemplary embodiment of the present disclosure.

[0076] Referring to FIG. 5, the insole manufacturing system (430) can manufacture a personalized insole based on foot deformation prediction information.

[0077] Specifically, the insole manufacturing system (430) may include a drawing generation system (431) and an insole production system (432).

[0078] The drawing generation system (431) may include a pattern generation unit (433) and an automatic ordering unit (434).

[0079] The pattern generation unit (433) can generate an insole pattern based on foot deformation prediction information. Specifically, the pattern generation unit (433) can generate an insole pattern using a first insole pattern having a shape corresponding to the shape of the sole, and a second insole pattern corresponding to a portion of a node where the pressure is greater than or equal to a first reference pressure value and the displacement is greater than or equal to the first reference displacement value. In other words, the insole pattern can be formed differently based on the pressure and displacement of each node.

[0080] The density of the first insole pattern may be higher than that of the second insole pattern. Alternatively, the elasticity of the first insole pattern may be lower than that of the second insole pattern. In other words, by using an insole pattern with a relatively low density or relatively high elasticity in areas of the foot where the pressure received is relatively high and the expected displacement is relatively large, the pressure can be distributed.

[0081] The automatic ordering unit (434) can order the production of an insole drawing modeled by the pattern generation unit (433) when a user makes a payment for the insole drawing. In one embodiment, when the insole drawing is completed in the pattern generation unit (433) or an order is completed in the automatic ordering unit (434), an insole production system (432) that produces insoles according to the insole drawing produced by the pattern generation unit (433) may be further included in the insole production system (430).

[0082] The insole production system (432) may include a milling machine (435), a 3D printer (436), or a laser cutter (437) that produces insoles in real time according to the shape of the insole drawing.

[0083] Fig. 6 is a flowchart illustrating a method for manufacturing an insole according to an exemplary embodiment of the present disclosure. Fig. 6 may be described with reference to Fig. 1.

[0084] Referring to FIG. 6, the insole manufacturing system (430) can receive foot deformation prediction information from the foot deformation prediction system (300) (S601). That is, the insole manufacturing system (430) can receive an output graph including nodes having pressure information and displacement information, or a 3D image corresponding to the output graph.

[0085] In some embodiments, the node may receive foot deformation prediction information that includes displacement information based on the time during which the node is subjected to the corresponding pressure. For example, the node's displacement information may be received when subjected to the corresponding pressure for 5 years. Alternatively, the node's displacement information may be received when subjected to the corresponding pressure for 10 years. In other words, by receiving displacement information based on the time during which the node is subjected to the corresponding pressure, an optimized insole pattern can be provided based on the lifespan of the shoe using the insole.

[0086] The insole manufacturing system (430) can generate an insole having multiple insole patterns based on pressure information and displacement information, i.e., foot deformation prediction information (S602). Specifically, the insole manufacturing system (430) can generate an insole including multiple insole patterns by matching each node to one of multiple groups based on the pressure information and displacement information, and using the insole pattern corresponding to each group.

[0087] For example, the insole manufacturing system (430) may match a node to a first group if the pressure is greater than or equal to a first pressure reference value and the displacement is greater than or equal to a first displacement reference value, match the node to a second group if the pressure is greater than or equal to a second pressure reference value and the displacement is greater than or equal to a second displacement reference value, and match the node to a third group if the pressure is greater than or equal to a third pressure reference value and the displacement is greater than or equal to a third displacement reference value. The first pressure reference value may be less than the second pressure reference value, and the second pressure reference value may be less than the third pressure reference value. The first displacement reference value may be less than the second displacement reference value, and the second displacement reference value may be less than the third displacement reference value. However, the embodiment is not limited thereto, and the insole manufacturing system (430) may also match nodes to a plurality of groups based on either the pressure value or the displacement value.

[0088] The insole manufacturing system (430) may use a first insole pattern for nodes corresponding to a first group, a second insole pattern for nodes corresponding to a second group, and a third insole pattern for nodes corresponding to a third group. The density of the first insole pattern may be higher than the density of the second insole pattern. The density of the second insole pattern may be higher than the density of the third insole pattern. The elasticity of the first insole pattern may be lower than the elasticity of the second insole pattern. The elasticity of the second insole pattern may be higher than the elasticity of the third insole pattern.

[0089] According to an exemplary embodiment of the present disclosure, an insole is created using different patterns depending on the foot structure, so that an insole that is optimized and personalized for the user can be provided.

[0090] Furthermore, in an exemplary embodiment of the present disclosure, an insole pattern is generated based on displacement information according to time under pressure, so that an insole pattern optimized for the lifespan of a shoe using the insole can be provided.

[0091] FIG. 7 is a drawing illustrating an exercise program providing system according to an exemplary embodiment of the present disclosure.

[0092] Referring to FIG. 7, the exercise program provision system (410) may include an optimal exercise combination unit (411) and a memory (412). In FIG. 7, the foot deformation prediction information may further include a node type. Furthermore, the foot deformation prediction information may further include the user's walking pattern.

[0093] The memory (412) can store various exercise programs corresponding to pressure values ​​and displacement values ​​for each part of the foot, i.e., each node type. For example, various exercise programs corresponding to pressure values ​​and displacement values ​​can be stored for each node type, i.e., each heel, sole, and toe.

[0094] The optimal exercise combination unit (411) may select exercise programs corresponding to foot deformation prediction information from among various exercise programs stored in the memory (412), and provide information on the selected exercise programs to the user terminal (100). Specifically, the optimal exercise combination unit (411) may select an exercise program for a part of the foot corresponding to a node in which the pressure is greater than or equal to a first reference pressure value and the displacement is greater than or equal to the first reference displacement value, and provide the selected exercise program to the user terminal (100). However, the embodiment is not limited thereto, and the optimal exercise combination unit (411) may also match a node to one of a plurality of groups based on various reference values ​​for pressure and displacement, and provide an exercise program corresponding to each group.

[0095] FIG. 8 is a drawing illustrating a medical information providing system according to an exemplary embodiment of the present disclosure.

[0096] Referring to FIG. 8, the medical information provision system (420) may include a treatment method selection unit (421) and a memory (422). In FIG. 8, the foot deformation prediction information may further include a node type. Furthermore, the foot deformation prediction information may further include the user's walking pattern.

[0097] The memory (422) can store various treatment methods corresponding to pressure and displacement values ​​for each part of the foot, i.e., each node type. For example, various treatment methods corresponding to pressure and displacement values ​​for each node type, i.e., heel, sole, and toes, can be stored.

[0098] The treatment method selection unit (421) may select a treatment method corresponding to the foot deformation prediction information among various treatment methods stored in the memory (422) and provide information on the selected treatment method to the user terminal (100). Specifically, the treatment method selection unit (421) may select a treatment method for a part of the foot corresponding to a node in which the pressure is greater than or equal to a first reference pressure value and the displacement is greater than or equal to the first reference displacement value, and provide the selected treatment method to the user terminal (100). However, the embodiment is not limited thereto, and the treatment method selection unit (421) may match a node to one of a plurality of groups based on various reference values ​​for pressure and displacement, and provide a treatment method corresponding to each group.

[0099] The combination of each step of each flowchart attached to the present invention may be performed by computer program instructions. These computer program instructions may be installed in a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing equipment, so that the instructions executed by the processor of the computer or other programmable data processing equipment create a means for performing the functions described in each step of the flowchart. These computer program instructions may also be stored in a computer-usable or computer-readable recording medium that can direct a computer or other programmable data processing equipment to implement the functions in a specific manner, so that the instructions stored in the computer-usable or computer-readable recording medium can also produce a manufactured article that includes an instruction means for performing the functions described in each step of the flowchart. Since the computer program instructions can also be installed on a computer or other programmable data processing device, a series of operational steps can be performed on the computer or other programmable data processing device to create a computer-executable process, and the instructions that cause the computer or other programmable data processing device to perform can also provide steps for performing the functions described in each step of the flowchart.

[0100] Additionally, each step may represent a module, segment, or portion of code that includes one or more executable instructions for performing a specific logical function(s). It should also be noted that in some alternative embodiments, the functions described in the steps may occur out of order. For example, two steps depicted in succession may actually be performed substantially concurrently, or the steps may sometimes be performed in reverse order, depending on the corresponding function.

[0101] The above description is merely an illustrative illustration of the technical idea of ​​the present invention, and those skilled in the art will appreciate that various modifications and variations can be made without departing from the essential quality of the present invention. Therefore, the embodiments disclosed in the present invention are intended to illustrate, rather than limit, the technical idea of ​​the present invention, and the scope of the technical idea of ​​the present invention is not limited by these embodiments. The scope of protection of the present invention should be interpreted by the following claims, and all technical ideas within a scope equivalent thereto should be interpreted as being included in the scope of the rights of the present invention.

Claims

1. A method for predicting foot deformation performed by an artificial intelligence module, A step of generating an input graph from an image of the foot before deformation by an encoder within the artificial intelligence module and providing the graph to a pre-trained deep learning model within the artificial intelligence module; A step in which the above-mentioned pre-trained deep learning model outputs an output graph in which the deformation prediction of the foot is reflected in the input graph; and A step of generating and outputting a post-deformation image in which the deformation prediction of the foot is reflected from the output graph by a decoder within the artificial intelligence module; The above deep learning model is trained to output an output graph of an image after deformation in which a deformation prediction of the foot is reflected for at least one node constituting the foot when an input graph of the image before deformation is input. Method for predicting foot deformity.

2. In paragraph 1, The above deep learning model is trained using training data in which an input graph of mesh data corresponding to an image of the foot before deformation and an output graph of mesh data corresponding to an image of the foot after deformation are paired. Method for predicting foot deformity.

3. In paragraph 1, The deformation prediction of the above-mentioned foot includes information on the pressure received by the node and information on the displacement of the node due to the pressure. Method for predicting foot deformity.

4. In paragraph 1, The above deep learning model is learned by further including at least one body information among weight, gender, age, disease, body type, and walking habits corresponding to the foot, and when predicting the deformation of the foot, the body information is further input corresponding to the image of the foot before deformation. Method for predicting foot deformity.

5. A communication unit that receives an image of the user's foot from the user terminal; A mesh image generation unit that generates an input mesh image corresponding to the above-mentioned foot image; An artificial intelligence module configured to generate an input graph based on the input mesh image and, using a graph neural network (GNN), generate deformation prediction information corresponding to the foot image based on the coordinate values of nodes included in the input graph; and Including an insole manufacturing system that generates insole drawing information corresponding to the above foot image; The above deformation prediction information is, Includes pressure information received by each of the nodes and displacement information indicating the degree to which the coordinate values of each of the nodes change due to the pressure information received by each of the nodes, The above insole drawing information is, It includes a first insole pattern corresponding to the shape of the user's sole and a second insole pattern formed on a part of the first insole pattern and configured to disperse the pressure applied to the user's foot by reflecting the deformation prediction information. The above first insole pattern is, A foot deformation prediction system generated based on either a mesh image of the user's foot shape before being input to the artificial intelligence module or a foot shape in a state where pressure is applied based on the deformation prediction information, depending on the amount of change in the user's foot shape before and after walking.

6. In paragraph 5, The above artificial intelligence module, Generate first displacement information indicating the degree to which the coordinate values of each of the nodes change when a first time elapses due to the pressure information, and second displacement information indicating the degree to which the coordinate values of each of the nodes change when a second time elapses due to the pressure information, A foot deformation prediction system, characterized in that the first displacement information and the second displacement information are different from each other.

7. In paragraph 5, The above artificial intelligence module, A foot deformation prediction system characterized in that it generates deformation prediction information corresponding to the foot image by further using the user's walking pattern.

8. A foot deformation prediction system that receives a user's foot image from a user terminal and generates foot deformation prediction information including pressure information and displacement information about the degree to which the coordinate values of each node are changed by the pressure information based on the coordinate values of nodes included in an input graph corresponding to the foot image using a graph neural network (GNN); and An insole manufacturing system is provided that generates insole drawing information corresponding to the foot image based on the foot deformation prediction information. The above insole drawing information is, It includes a first insole pattern corresponding to the shape of the user's sole and a second insole pattern formed on a part of the first insole pattern and configured to disperse the pressure applied to the user's foot by reflecting the deformation prediction information. The above first insole pattern is, A foot health management system formed based on either a mesh image of the user's foot shape before being input to the graph neural network (GNN) or a foot shape in a state where pressure is applied based on the deformation prediction information, depending on the amount of change in the user's foot shape before and after walking.

9. In paragraph 8, A foot health management system characterized by including an exercise program providing system that generates an exercise program for a user's foot part based on the above foot deformation information.

10. In paragraph 8, A foot health management system characterized by including a medical information providing system that generates medical information for treating a user's foot part based on the above foot deformation information.

11. In paragraph 8, The above insole manufacturing system is, A foot health management system characterized in that the insole drawing information is generated to include an insole pattern with a lower density than other parts for the foot portions corresponding to the nodes among the above nodes, wherein the pressure received is higher than a reference pressure value and the degree of change in the coordinate value is higher than a reference displacement value.

12. In paragraph 11, The above insole manufacturing system is, A foot health management system characterized in that the insole drawing information is generated to include an insole pattern having higher elasticity than other parts for the foot portions corresponding to the nodes among the nodes where the pressure received is higher than the reference pressure value and the degree of change in the coordinate value is higher than the reference displacement value.

Images