Prediction device, prediction system, and prediction method

Abstract

To provide a prediction device, a prediction system, and a prediction method capable of predicting the shape of feet of a measured person in different load states.SOLUTION: A prediction device 1 includes: an input part for receiving an input of measurement data on the shape of feet of a measured person in a first load state; an arithmetic part for predicting the shape of the feet of the measured person in a second load state of a load different from the first load state from the measurement data received by the input part using a prediction model 134 that has been trained by machine learning; and an output part for outputting prediction data 41 on the shape of the feet of the measured person in the second load state predicted by the arithmetic part. The prediction model 134 is generated in the training by machine learned beforehand on the basis of measurement data for training of the shape of the feet of the measured person for training in the first load state, and shape data for training on the shape of the feet of the measured person for training in the second load state.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to a prediction device, a prediction system, and a prediction method for predicting the shape of a subject's foot. [Background technology]

[0002] Conventionally, a method for making an insole based on the shape of the foot under a low load is known (Patent Document 1). In the method described in Patent Document 1, an insole is made based on the difference between pressurized state data obtained by measuring the shape of the foot of a person standing on a transparent plate and non-pressurized state data obtained by measuring the shape of the foot of a person lightly touching the transparent plate. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 5717894 Summary of the Invention [Problem to be solved by the invention]

[0004] When making custom-made shoes or insoles or when selecting shoes or insoles with a better fit, it is preferable to measure the shape of the foot. However, in general foot shape measurements, the subject is in a standing position, so the shape of the foot is measured in a load-bearing state where the sole of the foot is under load. In order to make shoes or insoles that are more suitable for the shape of the foot, it is preferable to have information on the shape of the foot measured in a no-load state where the sole of the foot is under almost no load.

[0005] As a method for measuring the shape of the foot in a no-load state, for example, a method is known in which the subject lies face down on a bed, the subject's foot is wrapped in a plaster bandage (cast), and the hardened plaster bandage is poured to take a mold of the foot shape and measure it. However, this measurement method requires someone with the skills to wrap the plaster bandage around the foot so that no weight is applied to the sole, and the shape of the foot in a no-load state cannot be easily measured in a store or the like.

[0006] In addition, in the method disclosed in Patent Document 1, it is necessary to measure the shape of the foot while standing on a transparent plate, and further measure the shape of the foot while lightly touching the transparent plate. In addition, in this method, it is difficult for the subject to maintain a state of lightly touching the transparent plate, and the degree of contact with the transparent plate varies depending on the subject, making it difficult to accurately measure the shape of the foot in a low load state.

[0007] The present disclosure has been made to solve such problems, and its purpose is to provide a prediction device, a prediction system, and a prediction method that are capable of predicting the shape of a subject's foot under different loading conditions. [Means for solving the problem]

[0008] A prediction device according to an aspect of the present disclosure is a prediction device that predicts the shape of a subject's foot. The prediction device includes an input unit that accepts input of measurement data of the subject's foot shape in a first load state, a calculation unit that predicts the subject's foot shape in a second load state different from the first load state from the measurement data accepted by the input unit using a prediction model trained by machine learning, and an output unit that outputs predicted data of the subject's foot shape in the second load state predicted by the calculation unit. The prediction model is generated in advance by training using machine learning based on training measurement data of the foot shape of the training subject in the first load state and training shape data of the foot shape of the training subject in the second load state.

[0009] A prediction system according to an aspect of the present disclosure includes a measurement device that measures a shape of a subject's foot in a first load state, and the prediction device described above.

[0010] A prediction method according to an aspect of the present disclosure is a prediction method for predicting the shape of a subject's foot. The prediction method includes the steps of accepting input of measurement data of the subject's foot shape in a first load state, predicting the subject's foot shape in a second load state different from the first load state from the accepted measurement data using a prediction model trained by machine learning, and outputting predicted data of the subject's foot shape in the predicted second load state. The prediction model is generated in advance by training using machine learning based on training measurement data of the foot shape of the training subject in the first load state and training shape data of the foot shape of the training subject in the second load state. Effect of the Invention

[0011] According to the present disclosure, a prediction model trained by machine learning is used, so that the shape of the subject's foot in the second loading condition can be predicted from measurement data in the first loading condition. [Brief description of the drawings]

[0012] [Figure 1] 1 is a schematic diagram illustrating a configuration of a prediction system including a prediction device according to an embodiment. [Diagram 2] 1 is a block diagram showing a configuration of a prediction device according to an embodiment. [Diagram 3] FIG. 1 is a schematic diagram for explaining a method for acquiring training data. [Figure 4] 10 is a flowchart for explaining the process of a prediction method executed by a prediction device according to an embodiment. [Diagram 5] 13A and 13B are schematic diagrams for explaining a process of producing an insole using the prediction system according to the embodiment. [Figure 6] FIG. 2 is a diagram illustrating the calculation of a prediction model used in the prediction device according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] A prediction device according to an aspect of the present disclosure is a prediction device that predicts the shape of a subject's foot. The prediction device includes an input unit that accepts input of measurement data of the subject's foot shape in a first load state, a calculation unit that predicts the subject's foot shape in a second load state different from the first load state from the measurement data accepted by the input unit using a prediction model trained by machine learning, and an output unit that outputs predicted data of the subject's foot shape in the second load state predicted by the calculation unit. Here, as an embodiment, a prediction device, a prediction system, and a prediction method that can predict the subject's foot shape in a low load state (second load state) where the load is smaller than the load state from the subject's foot shape in the load state (first load state) are described. In particular, in this embodiment, a prediction device, a prediction system, and a prediction method that predict the foot shape in a no-load state where there is almost no load from measurement data obtained by measuring the subject's foot shape using a prediction model trained by machine learning and output predicted data are described. Training data is required to generate a predictive model trained by machine learning, but in order to generate a predictive model with high prediction accuracy, it is preferable to use training data that is a set of training measurement data measured under the same measurement conditions for multiple subjects, and training shape data of the shape of the foot in an unloaded state obtained under the same acquisition conditions.

[0014] Here, the measurement conditions for measuring the shape of the subject's foot include the subject's posture, such as a standing posture, a sitting posture, etc., and the subject's state, such as a stationary state, a moving state, etc. Furthermore, the measurement conditions also include conditions such as the type of measuring device used for measurement, a measurement method using an image captured by a smartphone or the like, etc.

[0015] Moreover, the conditions for obtaining the shape of the foot in a no-load state include whether to have the subject lie face down on a bed, wrap a plaster bandage (cast) around the subject's foot, pour plaster into the hardened plaster bandage to obtain a mold of the foot shape, or to measure the foot in a seated position with the subject's foot raised above the floor. These conditions allow the shape of the foot to be obtained in a nearly no-load state, but some weight may be applied to the sole of the foot.

[0016] In the following description, a prediction model is generated by training using machine learning using training data that is a set of training measurement data measured by a subject in a standing posture in a stationary state and training shape data of the foot shape in a no-load state obtained using a plaster bandage. However, if it is possible to predict the foot shape in a no-load state, data under different measurement conditions may be included in the training measurement data and data under different acquisition conditions may be included in the training shape data to generate the prediction model through training using machine learning.

[0017] In addition, the shape of the foot predicted by the prediction device or the like is not limited to the shape of the foot in a no-load state, but may be the shape of the foot of the subject in a low-load state where the load is smaller than the load state. For example, a prediction model may be generated by training using machine learning using training data that is a set of training measurement data measured by a subject in a sitting posture in a stationary state and training shape data of the shape of the subject's foot lightly touching a transparent plate. This makes it possible to generate a prediction model that predicts the shape of the subject's foot in a low-load state where the load is smaller than the load state in the sitting posture from the measurement data of the subject in a sitting posture. In this disclosure, the low-load state includes a no-load state where there is almost no load.

[0018] Hereinafter, a prediction device, a prediction system, and a prediction method according to the embodiments will be described with reference to the drawings. In the following description, the same components are denoted by the same reference numerals. Their names and functions are also the same. Therefore, detailed description thereof will not be repeated.

[0019] (Embodiment) [Configuration of the prediction system] FIG. 1 is a schematic diagram showing a configuration of a prediction system 10 including a prediction device 1 according to an embodiment. The prediction system 10 includes a measurement device 2 that measures the shape of the subject's foot in a loaded state (first loaded state), and a prediction device 1 that predicts the shape of the subject's foot in a no-load state (second loaded state) from the measurement data measured by the measurement device 2. The prediction device 1 uses a prediction model 134 trained by machine learning to predict the shape of the subject's foot in a no-load state. Therefore, the prediction device 1 needs to generate the prediction model 134 by training by machine learning using training data 132 before predicting the shape of the subject's foot in a no-load state. In this embodiment, a configuration in which the prediction device 1 generates the prediction model 134 using the training data 132 will be described, but the prediction device 1 may be configured to accept the prediction model 134 generated by another device.

[0020] Hereinafter, the prediction device 1 will be described as a device that generates a prediction model 134 by training by machine learning using training data, and predicts the shape of the subject's foot in a no-load state from measurement data using the trained prediction model 134. The prediction device 1 outputs the predicted shape of the subject's foot in a no-load state as prediction data 41. By using the prediction data 41 to create custom-made shoes or insoles, it becomes possible to create shoes or insoles that are more suitable for the subject's feet than shoes or insoles created based on the shape of the foot in a loaded state.

[0021] Fig. 2 is a block diagram showing a configuration of a prediction device 1 according to an embodiment. As shown in Fig. 2, the prediction device 1 includes a processor 11, a memory 12, a storage 13, an input interface 14, an output interface 15, a communication interface 16, and a media reader 17. These components are connected via a processor bus 19.

[0022] The processor 11 is a computer that reads out programs (for example, an OS (Operating System) 130, a training program 131, and a prediction program 133) stored in the storage 13, and deploys and executes the read out programs in the memory 12. The processor 11 is configured, for example, with a CPU (Central Processing Unit), an FPGA (Field Programmable Gate Array), a GPU (Graphics Processing Unit), or an MPU (Multi Processing Unit). The processor 11 may also be configured with a processing circuitry.

[0023] The memory 12 is composed of a volatile memory such as a dynamic random access memory (DRAM) or a static random access memory (SRAM), or a non-volatile memory such as a read only memory (ROM) or a flash memory.

[0024] The storage 13 is configured by a non-volatile storage device such as a hard disk drive (HDD) or a solid state drive (SSD), etc. The storage 13 stores an OS 130, a training program 131, a prediction program 133, training data 132, a prediction model 134, etc.

[0025] The training program 131 is a program for generating a prediction model 134 through training by machine learning using training data 132. The prediction program 133 is a program for predicting the shape of the subject's foot in an unloaded state from measurement data of the foot shape in a loaded state using the prediction model 134. The training program 131 and the prediction model 134 are used together to execute a prediction method.

[0026] The input interface 14 is an example of an input unit and accepts input operations from a keyboard, a mouse, a touch device, etc. The output interface 15 is an example of an output unit and outputs predicted data 41 of the foot shape in a no-load state predicted by the prediction device 1 to a display or the like.

[0027] The communication interface 16 is an example of an input section and an output section, and receives the input of measurement data from the measurement device 2 and transmits and receives data to and from other devices by performing wired or wireless communication. The communication interface 16 can receive the input of measurement data and output predicted prediction data 41 by transmitting and receiving data to and from other devices. The communication interface 16 may also receive the input of training data 132 from other devices, or receive an image of the shape of the subject's foot in a weight-bearing state captured by a smartphone or the like. When the prediction device 1 receives the image, it can obtain data (corresponding to measurement data) of the shape of the subject's foot in a weight-bearing state from the image based on a predetermined algorithm.

[0028] The media reading device 17 accepts a storage medium such as a removable disk 18, a memory chip, a USB memory, etc., and obtains data stored in the removable disk 18, the memory chip, the USB memory, etc. The media reading device 17 may read out the training data 132 stored in the removable disk 18, or store the predicted prediction data 41 in the removable disk 18, etc., and output it.

[0029] In the above description, the prediction device 1 generates the prediction model 134 through machine learning training using the training data 132, but if the prediction model 134 generated by another device is prepared in advance, it may be received from a server or the like via the communication interface 16. In addition, the prediction device 1 may read the prediction model 134 stored in a removable disk 18 or the like by the media reading device 17.

[0030] The measuring device 2 is, for example, a three-dimensional foot shape scanner using laser measurement, and includes a tabletop 21 and a laser measuring unit 22 installed so as to sandwich the tabletop 21. When the person to be measured places his / her foot on the tabletop 21 in a standing position, the weight of the person to be measured applies a load from the foot to the tabletop 21. That is, the person to be measured is in a state of being under load. With the weight applied to the person to be measured's foot, the measuring device 2 measures the shape of the foot while moving from the toe to the heel using the laser measuring unit 22. The measuring device 2 outputs measurement data (three-dimensional data) of the shape of the person to be measured, acquired by the laser measuring unit 22, to the prediction device 1. Note that the measurement data is sufficient as long as it includes at least the data of the shape of the foot acquired by the measuring device 2, and may also include other data (for example, attribute information such as the gender or age of the person to be measured).

[0031] [Training data] As the training data 132 for generating the prediction model 134 trained by machine learning, a set of training measurement data measured by a subject in a standing posture in a stationary state (hereinafter, although the term "stationary state" is not particularly used, the subject is assumed to be in a stationary state during measurement) and training shape data of the foot shape in a no-load state obtained using a plaster bandage is used. Fig. 3 is a schematic diagram for explaining a method of acquiring the training data.

[0032] A plurality of training subjects are prepared to obtain training data. The training subjects may include subjects for whom the shape of the foot in a non-loaded state is to be predicted. First, as shown in FIG. 3, the measurement device 2 measures the feet of the plurality of training subjects in a standing position. That is, the measurement device 2 measures the feet of the training subjects in a standing position, and obtains the shape of the feet of the training subjects in a loaded state. When the training subjects place their feet on the top board 21 in a standing position, the measurement device 2 measures the shape of the feet while moving from the toes to the heels of the feet using the laser measurement unit 22 in a state in which the weight of the training subjects applies a load from the feet to the top board 21. The data of the foot shape measured by the measurement device 2 is three-dimensional data. The three-dimensional data may be used as the training measurement data as it is, but in this embodiment, the three-dimensional data is converted into two-dimensional data including information on the height direction of the foot shape, and the converted two-dimensional data is used as the training measurement data 31.

[0033] For the same multiple training subjects, measurements are taken of the unloaded feet using a plaster bandage as shown in Fig. 3, and data on the shape of the training subjects' feet in an unloaded state is obtained. Specifically, data on the shape of the training subjects' feet in an unloaded state is obtained by measuring a plaster foot mold made using a plaster bandage with the measuring device 2. Therefore, the data on the foot shape obtained using the plaster bandage is three-dimensional data. The three-dimensional data may be used as training shape data as is, but in this embodiment, the three-dimensional data is converted to two-dimensional data including information on the height direction of the foot shape, and the converted two-dimensional data is used as training shape data 32.

[0034] Pairs of training measurement data 31 and training shape data 32 obtained from each of a plurality of training subjects to be measured are adopted as training data 132. Note that the training data 132 converted into two-dimensional data including information in the height direction of the foot shape is also referred to as two-dimensional training data. The training data 132 may be obtained by simply converting the three-dimensional training measurement data 31 and training shape data 32 into two-dimensional training measurement data 31 and training shape data 32, or may be only the data in the vicinity of the sole that is necessary for predicting the shape of the subject's foot in a non-loaded state. Therefore, the information in the height direction of the foot shape is, for example, the information on the height h from the reference plane S to the sole as shown in FIG. 3. The reference plane S is, for example, a plane parallel to the floor surface in a standing posture, and is a plane at a predetermined height in the direction from the ankle or instep of the training subject to the sole, or a plane having the maximum outer shape. When the reference plane S is set inside the foot of the training subject as shown in FIG. 3, the height h from the reference plane S to the sole can also be regarded as the depth from the reference plane S. Therefore, the training measurement data 31 and training shape data 32 converted into two-dimensional data are also referred to as sole depth images.

[0035] In addition, the unevenness of the sole in the training shape data 32 is larger than that in the training measurement data 31. Therefore, for the training measurement data 31, the data from the reference plane S to the height h1 (first height) in the three-dimensional training measurement data is converted into two-dimensional data, while for the training shape data 32, the data from the reference plane S to the height h2 (second height (h1 < h2)) higher than the height h1 in the training shape data is converted into two-dimensional data.

[0036] When the training measurement data 31 and the training shape data 32 are converted into sole depth images, information on the height h from the reference plane S to the sole is converted into two-dimensional data, for example, as grayscale information (information of 256 gradations from white to black). Therefore, the sole depth image is an image in which gray shading according to the height h is added in the two-dimensional shape of the foot. For example, the sole depth image expresses the two-dimensional shape of the foot with high height h in black, low height h in white, and intermediate height h in gray. There is no restriction on how the information on the height h is expressed in the sole depth image, and it may be expressed not only as gradation information as described above, but also as hue information or contour lines. For example, when the information on the height h is expressed as hue information, the sole depth image expresses the two-dimensional shape of the foot with high height h in red, low height h in blue, and intermediate height h in yellow. In any case, the training measurement data 31 and the training shape data 32 may be two-dimensional array data that stores information on height h at each point of the two-dimensional foot shape as a scalar value.

[0037] Specifically, the training measurement data 31 represents an XY plane of 400 mm×400 mm as a two-dimensional image of 256 pixels×256 pixels, and the height h information stored in each pixel is represented as a height of 0 mm to 20 mm from the reference plane S in a gray scale of 256 gradations. Note that a plane parallel to the floor surface in a standing posture is the XY plane, and a direction perpendicular to the XY plane is the Z direction, which is the direction of height h. The training shape data 32 represents an XY plane of 400 mm×400 mm as a two-dimensional image of 256 pixels×256 pixels, and the height h information stored in each pixel is represented as a height of 0 mm to 40 mm from the reference plane S in a gray scale of 256 gradations. Note that the range of height h information stored in the training measurement data 31 is narrower than that in the training shape data 32, being 0 mm to 20 mm from the reference plane S. However, like the training shape data 32, the training measurement data 31 expresses the information of the height h as a 256-level grayscale. Therefore, the amount of information of the training measurement data 31 converted into two-dimensional data is the same as the amount of information of the training shape data 32 converted into two-dimensional data. As a result, the training measurement data 31 has a larger amount of data per mm unit than the training shape data 32. Of course, the amount of data per mm unit of the training measurement data 31 and the amount of data per mm unit of the training shape data 32 may be the same. In addition, the range from the reference plane S stored as the information of the height h in the training measurement data 31 and the training shape data 32 is not particularly limited, and may be changed, for example, depending on the data of the target foot (e.g., shape and size).

[0038] In this way, the training data 132 converts the training measurement data 31 and the training shape data 32 into sole depth images, respectively, so that a prediction model with high prediction accuracy can be generated while suppressing the amount of data processed by the prediction device 1. Of course, the training data 132 may be three-dimensional data expressed by voxels, point clouds, meshes, boundary surfaces, etc., without converting the training measurement data 31 and the training shape data 32 into sole depth images, respectively. Since the prediction model is generated using the sole depth image as the training data 132, when the prediction device 1 predicts the shape of the foot in a no-load state using the prediction model, it is necessary to convert the measurement data input to the prediction device 1 into a sole depth image. In addition, since the shape of the foot in a no-load state predicted by the prediction device 1 is output as a sole depth image, it is necessary to restore the shape of the foot in a no-load state of the three-dimensional data from the output sole depth image.

[0039] [Forecasting method] Next, the process of the prediction method executed by the prediction system 10 will be described with reference to a flowchart. Fig. 4 is a flowchart for explaining the process of the prediction method executed by the prediction device 1 according to the embodiment. Note that each step shown in Fig. 4 is realized by the processor 11 of the prediction device 1 executing the training program 131 and the prediction program 133.

[0040] First, the prediction device 1 judges whether or not to perform the training phase processing by the training program 131 (step S101). When the prediction device 1 does not yet have the prediction model 134, or when sufficient training has not been performed on the prediction model 134, the prediction device 1 performs the training phase processing. When the training phase processing is not to be performed (NO in step S101), the prediction device 1 advances the processing to step S104. On the other hand, when the training phase processing is to be performed (YES in step S101), the prediction device 1 executes the training phase processing to generate a prediction model 134 for predicting the shape of the subject's foot in a no-load state from the measurement data of the shape of the subject's foot in a load state.

[0041] In the training phase process, input of training data 132 for N people is accepted (step S102) in order to generate a prediction model 134. The training data 132 accepted in step S102 includes training measurement data 31 and training shape data 32.

[0042] Next, the prediction device 1 generates a prediction model 134 that predicts the shape of the subject's foot in a no-load state through machine learning training using the training data 132 (step S103). Here, the prediction model 134 includes a network structure such as a known neural network, a support vector machine (SVM), or a Bayesian network, and internal parameters used by the network structure, and the internal parameters are optimized (adjusted) based on the predicted data of the foot shape in a no-load state predicted from the training measurement data 31 and the training shape data 32.

[0043] In particular, the training data 132 is pair data of the training measurement data 31 and the training shape data 32, and in training by machine learning based on the pair data, an algorithm for image-to-image translation is used with the training shape data 32 as teacher data. Specifically, as an algorithm for conversion between pair data (pair images) as supervised learning, an encoder-decoder model such as a convolutional autoencoder (CAE), a variational autoencoder (VAE), or U-Net can be used. Here, the variational autoencoder (VAE) is a model in which a constraint is imposed so that the latent variable of the convolutional autoencoder (CAE) is normally distributed, and the U-Net is a model in which the encoder and decoder of the convolutional autoencoder (CAE) are skip-connected. The encoder uses convolution, vision transformer, etc., and the decoder uses transposed convolution, etc.

[0044] In addition, a GAN (Generative Adversarial Network) model may be used as an algorithm used for conversion between paired data (paired images). For example, pix2pix is ​​used as the GAN model. Note that the algorithm used for conversion between paired data (paired images) is not limited to the above-mentioned algorithm, and any algorithm used for image generation, such as a flow-based model, a diffusion model, an autoregressive model (for example, pixelRNN, pixelCNN, ImageGPT, etc.), or an energy-based model, may be used as appropriate.

[0045] Next, the prediction device 1 judges whether or not to perform operation phase processing by the prediction program 133 (step S104). When a sufficiently trained prediction model 134 is generated in the training phase processing, the prediction device 1 performs operation phase processing. When the operation phase processing is not performed (NO in step S104), the prediction device 1 advances the processing to step S110. On the other hand, when the operation phase processing is performed (YES in step S104), the prediction device 1 executes operation phase processing to predict the shape of the subject's foot in a no-load state from only the measurement data of the subject's foot shape in a load state using the prediction model 134.

[0046] In the operation phase process, in order to predict the shape of the subject's foot in a no-load state, input of measurement data of the subject's foot shape in a loaded state is accepted (step S105). Note that in step S105, only the measurement data obtained by measuring the shape of the subject's foot with the measuring device 2 is required, and the shape of the foot in a no-load state obtained using a plaster bandage is not required.

[0047] Next, the prediction device 1 converts the received three-dimensional measurement data into two-dimensional data including information on the height direction of the foot shape (step S106) so that the data can be used in the prediction model 134. Specifically, the prediction device 1 converts information on the height h from the reference plane S of the measurement data to the sole of the foot into a sole depth image that is grayscale information.

[0048] Next, the prediction device 1 predicts the shape of the subject's foot in a no-load state using the prediction model 134 (step S107). The prediction device 1 converts the predicted data predicted in step S107 into three-dimensional data and outputs it to a device for designing shoes or insoles (step S108). Alternatively, the prediction device 1 may display the predicted data as an image on a display connected to the prediction device 1 for confirmation.

[0049] The prediction device 1 judges whether or not new measurement data has been input from the measurement device 2 (step S109). When the prediction device 1 receives new measurement data, it predicts the shape of the subject's foot in a no-load state based on the new measurement data. Therefore, when the prediction device 1 receives new measurement data (YES in step S109), the process returns to step S106.

[0050] On the other hand, if the prediction device 1 has not received an input of new measurement data (NO in step S109), the prediction device 1 determines whether or not to end the processing of the prediction method (step S110). If the prediction device 1 has not received an input to end the processing of the prediction method (NO in step S110), the prediction device 1 returns the processing to step S101. On the other hand, if the prediction device 1 has received an input to end the processing of the prediction method (YES in step S110), the prediction device 1 ends the processing of the prediction method.

[0051] Next, an example of making an insole based on the shape of the subject's foot in a no-load state predicted by a prediction system 10 including at least a prediction device 1 and a measurement device 2 will be described with reference to a schematic diagram. FIG. 5 shows a schematic diagram for explaining the process of making an insole in the prediction system 10 according to the embodiment. First, in the training data collection phase P1 shown in FIG. 5, in order to collect training data 132, the measurement device 2 is used to measure the standing foot of N training subjects, and measurement of the no-load foot is performed using a plaster bandage. This allows N sets of paired data to be collected, combining three-dimensional data of the foot shape measured by the measurement device 2 obtained from the same training subject with three-dimensional data of the foot shape in a no-load state.

[0052] In the training data pre-processing phase P2, the three-dimensional data 31a of the foot shape measured by the measuring device 2 and the three-dimensional data 32a of the foot shape in a no-load state collected in the training data collection phase P1 are converted into two-dimensional data including information on the foot shape in the height direction. Specifically, the three-dimensional data 31a of the foot shape measured by the measuring device 2 is converted into training measurement data 31 of a sole depth image, and the three-dimensional data 32a of the foot shape in a no-load state is converted into training shape data 32 of a sole depth image.

[0053] In the prediction model training phase P3, a prediction model 134 trained by machine learning is generated using N sets of paired data combining the training measurement data 31 and the training shape data 32. The processes from the training data collection phase P1 to the prediction model training phase P3 correspond to the processes in the training phase shown in FIG.

[0054] Next, in the measurement data collection phase P4, measurement data 33a of a subject (who may be different from the training subject or may be the same person) whose foot shape in a no-load state is to be predicted is collected. The measurement device 2 acquires the foot shape of the subject in a standing position with load as measurement data 33a. The measurement data 33a is three-dimensional data.

[0055] In the measurement data pre-processing phase P5, the three-dimensional measurement data 33a collected in the measurement data collection phase P4 is converted into two-dimensional data including information on the height direction of the foot shape. Specifically, the three-dimensional data of the measured foot shape is converted into measurement data 33 of a sole depth image.

[0056] In prediction phase P6, prediction model 134 is used to predict the shape of the subject's foot in a no-load state from measurement data 33 of the sole depth image. Prediction data 41 predicted in prediction phase P6 is shape data of the sole depth image, and is two-dimensional data including information on the foot shape in the height direction. The processes from measurement data collection phase P4 to prediction phase P6 correspond to the processes in the operation phase shown in FIG.

[0057] In the sole surface restoration phase P7, three-dimensional sole shape data 42 is restored from the predicted data 41, which is two-dimensional data including information on the height direction of the foot shape, to facilitate use in insole design. The sole shape data 42 is three-dimensional shape data from the reference plane S to the sole of the foot. In the sole surface restoration phase P7, shape data on the ankle side from the reference plane S may be extracted from the three-dimensional measurement data 33a and combined with the sole shape data 42 to restore the sole shape data of the entire foot.

[0058] In the insole design phase P8, the insole design data 51 is designed based on the sole shape data 42 restored in the sole surface restoration phase P7. Although the design of the insole has been described as an example in Fig. 5, the shoe design data may be designed based on the sole shape data 42 or the sole shape data of the entire foot.

[0059] [Prediction model] Next, the prediction model 134 will be described using a schematic diagram. FIG. 6 is a diagram illustrating the calculation of the prediction model used in the prediction device 1 according to the embodiment. A convolutional auto-encoder (CAE) is used for the prediction model 134, and the sole depth image of the training measurement data 31 is used as an input image, and predicted data 41 of the shape of the subject's foot in a no-load state is used as an output image. Here, the convolutional auto-encoder is a multi-layer neural network including a structure for encoding using one or more convolutional layers or pooling layers, and a structure for decoding using one or more deconvolutional layers.

[0060] Although it has been described that the prediction model 134 uses a convolutional autoencoder (CAE), which is an encoder-decoder model, it is possible to use, for example, a vision transformer instead of a convolution for the encoder. When a vision transformer is used for the encoder, the structure does not have a pooling layer like a convolution, so that in the output image of the predicted data 41 of the shape of the subject's foot in a no-load state, the contrast between the predicted shape of the subject's foot and the background is relatively clear, the predicted outline shape of the foot is less likely to be distorted, and noise is less likely to occur in some parts of the background.

[0061] Furthermore, a GAN (Generative Adversarial Network) model, such as pix2pix, may be used for the prediction model 134 instead of an encoder-decoder model. When pix2pix is ​​used for the prediction model 134, the contrast between the predicted subject's foot shape and the background becomes relatively clearer compared to a convolutional autoencoder (CAE) that uses convolution in the encoder, since it also has a structure that does not have a pooling layer like convolution.

[0062] Using the sole depth image of the training shape data 32 as correct answer data, the output image of the predicted prediction data 41 is compared with the correct answer data, and training is performed to adjust the prediction model 134 so that the loss function between the output image and the correct answer data is minimized. Therefore, by using the trained prediction model 134, the prediction device 1 can predict the shape of the subject's foot in an unloaded state from the shape of the subject's foot in a loaded state measured by the measurement device 2. As a result, by using the trained prediction model 134, the prediction device 1 can obtain the shape of the subject's foot in an unloaded state without using a plaster bandage.

[0063] [Variations] (1) In the above embodiment, training data 132 is collected from a plurality of subjects selected regardless of their attributes (e.g., gender, age, race, etc.), and the prediction model 134 is generated by training by machine learning using the collected training data 132. This is not limited to the above, and training data may be collected from a plurality of subjects divided by attribute, and a prediction model may be generated by training by machine learning using the training data divided by attribute. The prediction device 1 has a prediction model for each attribute, and by receiving attribute information of the subject, selects a prediction model corresponding to the attribute. Then, the prediction device 1 predicts the shape of the subject's foot in a no-load state from the measurement data using the selected prediction model. As a result, the prediction device 1 can accurately predict the shape of the subject's foot in a no-load state by selecting a prediction model with high prediction accuracy based on the attribute information of the subject.

[0064] (2) In the above embodiment, it has been described that the shape of the subject's foot in a no-load state is predicted from the measurement data of the shape of the subject's foot in a loaded state using a trained prediction model. However, the state in which the shape of the foot is predicted is not limited to a no-load state. For example, the shape of the subject's foot in a low-load state where the load is not zero may be predicted using a prediction model in machine learning training using training shape data of the shape of the subject's foot in a low-load state where the load is smaller than the loaded state. Furthermore, the shape of the subject's foot in a high-load state where the load is larger than the loaded state may be predicted using a prediction model in machine learning training using training shape data of the shape of the subject's foot in a high-load state where the load is larger than the loaded state. In other words, the prediction device 1 can predict the shape of the subject's foot in different load states by generating a prediction model in machine learning training based on the training measurement data of the shape of the subject's foot in a first load state and the training shape data of the shape of the subject's foot in a second load state where the load is different from the first load state.

[0065] [Aspects] (1) A prediction device according to the present disclosure includes: an input unit that accepts input of measurement data of a shape of a subject's foot in a first loading state; a calculation unit that predicts the shape of the subject's foot in a second load condition different from the first load condition from the measurement data received by the input unit using a prediction model trained by machine learning; an output unit that outputs predicted data of the subject's foot shape in the second load state predicted by the calculation unit, The prediction model is The training data is generated in advance through machine learning training based on training measurement data of the shape of the foot of the training subject in a first loading state and training shape data of the shape of the foot of the training subject in a second loading state.

[0066] As a result, the prediction device according to the present disclosure uses a prediction model that has been trained by machine learning, and is therefore able to predict the shape of the subject's foot in the second loading condition from the measurement data in the first loading condition.

[0067] (2) In the prediction device according to (1), the training measurement data and the training shape data may be three-dimensional data; The prediction model may be generated in advance by training using machine learning based on two-dimensional training data obtained by converting the training measurement data and the training shape data into two-dimensional data including information on the height direction of the shape of the foot of the training subject, The calculation unit may use a prediction model to predict the shape of the subject's foot in the second load-bearing state from the measurement data converted into two-dimensional data.

[0068] (3) In the prediction device described in (2), the two-dimensional training data may include information on the shape of the training subject's feet in the height direction as gradation information.

[0069] (4) In the prediction device according to (2) or (3), the information on the foot shape in the height direction may be information on the height from a reference plane to the sole of the foot.

[0070] In the prediction device described in (5)(4), the two-dimensional training data may be converted from the training measurement data up to a first height from the reference plane in the sole direction to two-dimensional data, or the training shape data up to a second height higher than the first height from the reference plane in the sole direction to two-dimensional data.

[0071] (6) In the prediction device described in (5), an amount of information of the training measurement data converted into two-dimensional data may be equal to an amount of information of the training shape data converted into two-dimensional data.

[0072] (7) In the prediction device according to any one of (2) to (6), predicted data of the two-dimensional data may be restored to three-dimensional data.

[0073] (8) In the prediction device according to any one of (1) to (7), the prediction model may be generated in advance by training using machine learning for each attribute information of the training subjects, The input unit may receive attribute information of the subject, and a prediction model corresponding to the attributes may be used to predict the shape of the subject's foot in the second load state from the measurement data received by the input unit.

[0074] (9) The prediction system according to the present disclosure is A measuring device for measuring the shape of the subject's foot in a first load state; The present invention includes a prediction device according to any one of (1) to (8).

[0075] (10) A prediction method according to the present disclosure is a method for predicting a shape of a subject's foot, comprising: receiving input of measurement data of the subject's foot shape in a first loading state; predicting, from the received measurement data, a shape of the subject's foot in a second load condition in which the load is smaller than that in the first load condition, using a prediction model trained by machine learning; and outputting predicted data of the subject's foot shape in the predicted second loading state, The prediction model is The training data is generated in advance through machine learning training based on training measurement data of the shape of the foot of the training subject in a first loading state and training shape data of the shape of the foot of the training subject in a second loading state.

[0076] The embodiments disclosed herein should be considered to be illustrative and not restrictive in all respects. The scope of the present invention is defined by the claims, not the above description, and is intended to include all modifications within the meaning and scope of the claims. [Explanation of symbols]

[0077] 1 prediction device, 2 measurement device, 10 prediction system, 11 processor, 12 memory, 13 storage, 14 input interface, 15 output interface, 16 communication interface, 17 media reader, 18 removable disk, 19 processor bus, 21 top plate, 22 laser measurement unit, 131 training program, 132 training data, 133 prediction program, 134 prediction model.

Claims

1. A prediction device for predicting the shape of a subject's foot, an input unit that receives input of measurement data of the subject's foot shape in a first loading state; a calculation unit that predicts a shape of the subject's foot in a second load condition different from the first load condition from the measurement data received by the input unit using a prediction model trained by machine learning; and an output unit that outputs predicted data of the foot shape of the subject in the second load-bearing state predicted by the calculation unit, The predictive model is A prediction device that is generated in advance through machine learning training based on training measurement data of the foot shape of a training subject in the first loading state and training shape data of the foot shape of the training subject in the second loading state.

2. the training measurement data and the training shape data are three-dimensional data; The prediction model is generated in advance by training using machine learning based on two-dimensional training data obtained by converting the training measurement data and the training shape data into two-dimensional data including information on the height direction of the shape of the foot of the training subject, The prediction device according to claim 1 , wherein the calculation unit predicts the shape of the subject's foot in the second load-bearing state from the measurement data converted into two-dimensional data, using the prediction model.

3. The prediction device according to claim 2 , wherein the two-dimensional training data includes information on the shape of the training subject's foot in a height direction as gradation information.

4. 3. The prediction device according to claim 2, wherein the information on the shape of the foot in the height direction is information on the height from a reference plane to the sole of the foot.

5. 5. The prediction device of claim 4, wherein the two-dimensional training data is obtained by converting data from the training measurement data up to a first height in the sole direction from the reference plane into two-dimensional data, and data from the training shape data up to a second height higher than the first height in the sole direction from the reference plane into two-dimensional data.

6. 6. The prediction device according to claim 5, wherein an amount of information of the training measurement data converted into two-dimensional data is the same as an amount of information of the training shape data converted into two-dimensional data.

7. The calculation unit is The prediction device according to claim 2 , wherein the predicted data of two-dimensional data is restored to three-dimensional data.

8. The prediction model is generated in advance by machine learning training for each attribute information of the training subjects, 2. The prediction device according to claim 1, wherein the input unit receives attribute information of the subject, and predicts the shape of the subject's foot in the second load condition from the measurement data received by the input unit using the prediction model corresponding to the attributes.

9. a measuring device for measuring a shape of the subject's foot in the first load-bearing state; A prediction system comprising: the prediction device according to any one of claims 1 to 8.

10. A method for predicting a shape of a subject's foot, comprising: receiving input of measurement data of the subject's foot shape in a first load state; predicting a shape of the subject's foot in a second loading state different from the first loading state from the received measurement data using a prediction model trained by machine learning; and outputting predicted data of the foot shape of the subject in the predicted second loading state, The predictive model is A prediction method in which training measurement data of the shape of the foot of a training subject in the first loading state and training shape data of the shape of the foot of the training subject in the second loading state are generated in advance through machine learning training.

Images