Prediction device, prediction system, and prediction method

Abstract

The present invention provides a prediction device, a prediction system, and a prediction method capable of predicting the shape of a foot of a subject in different load states. A prediction device (1) is provided with: an input unit that receives input of measurement data for the shape of a foot of a person to be measured in a first load state; a calculation unit that uses a prediction model (134) trained by machine learning to predict the shape of the foot of the person to be measured in a second load state different from the first load state on the basis of the measurement data received by the input unit; and an output unit that outputs prediction data (41) of the shape of the foot of the person to be measured in the second load state, the prediction data being predicted by the calculation unit. The prediction model (134) is generated in advance by training using machine learning on the basis of training measurement data for the shape of the foot of the training subject in a first load state and training shape data for the shape of the foot of the training subject in a second load state.

Description

Technical Field

[0001] The present disclosure relates to a prediction device, a prediction system, and a prediction method for predicting the shape of a foot of a person to be measured. Background Art

[0002] Conventionally, there is known a method for producing an insole based on the shape of a foot under a low load state (Japanese Patent No. 5717894). In the method described in Japanese Patent No. 5717894, an insole is produced based on the difference between pressurized state data obtained by measuring the shape of a subject's foot standing on a transparent plate and non-pressurized state data obtained by measuring the shape of the subject's foot lightly touching the transparent plate. Summary of the invention

[0003] When custom-made shoes or insoles (insoles) are made or when shoes or insoles with a better fit are selected, it is preferable to measure the shape of the foot. However, in the usual measurement of the shape of the foot, the person being measured is in a standing position, so the shape of the foot is measured in a loaded state where a load is applied to the sole of the foot. In order to make shoes or insoles that are more suitable for the shape of the foot, it is preferable to have information on the shape of the foot measured in a no-load state where almost no load is applied to the sole of the foot.

[0004] As a method for measuring the shape of the foot in a no-load state, for example, the following method is known: a person to be measured is placed in a prone position on a bed, a plaster bandage (gips) is wrapped around the foot of the person to be measured, and the plaster is poured into the hardened plaster bandage, thereby performing injection molding and measuring the shape of the foot. However, in this measurement method, a person with the skill to wrap the plaster bandage around the foot in a manner that does not apply a load to the sole of the foot is required, and the shape of the foot in a no-load state cannot be easily measured in a store or the like.

[0005] In addition, in the method disclosed in Japanese Patent No. 5717894, the shape of the foot needs to be measured while standing on a transparent plate, and then the shape of the foot needs to be measured while lightly touching the transparent plate. In addition, in the method, it is difficult for the subject to keep lightly touching the transparent plate, and it is difficult to accurately measure the shape of the foot in a low-load state depending on the degree of contact of the subject with the transparent plate.

[0006] The present disclosure is made to solve the above-mentioned problems, and an object of the present disclosure is to provide a prediction device, a prediction system, and a prediction method capable of predicting the shape of a foot of a person to be measured under different load states.

[0007] According to a certain aspect of the present disclosure, a prediction device is a prediction device for predicting the shape of a subject's foot. The prediction device includes: an input unit that receives input of measurement data of the shape of a subject's foot under a first load state; a calculation unit that uses a prediction model trained by machine learning to predict the shape of the subject's foot under a second load state different from the first load state based on the measurement data received by the input unit; and an output unit that outputs the prediction data of the shape of the subject's foot under the second load state predicted by the calculation unit. The prediction model is based on training measurement data of the shape of a training subject's foot under the first load state and training shape data of the shape of a training subject's foot under the second load state, and is pre-generated by training using machine learning.

[0008] A prediction system according to one aspect of the present disclosure includes: a measurement device that measures the shape of a foot of a person being measured in a first load state; and the prediction device.

[0009] The prediction method disclosed in one aspect is a prediction method for predicting the shape of a subject's foot. The prediction method includes: a step of receiving input of measurement data of the shape of a subject's foot under a first load state; a step of using a prediction model trained by machine learning to predict the shape of the subject's foot under a second load state different from the first load state based on the received measurement data; and a step of outputting the predicted data of the shape of the subject's foot under the second load state. The prediction model is based on training measurement data of the shape of a training subject's foot under the first load state and training shape data of the shape of a training subject's foot under the second load state, and is generated in advance by training using machine learning.

[0010] The above and other objects, features, aspects and advantages of the present invention will become more apparent from the following detailed description of the present invention when taken in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 This is a schematic diagram showing the configuration of a prediction system including a prediction device according to an embodiment.

[0012] Figure 2 This is a block diagram showing the structure of a prediction device according to an embodiment.

[0013] Figure 3 This is a schematic diagram for explaining a method of acquiring training data.

[0014] Figure 4 This is a flowchart for explaining the processing of the prediction method executed by the prediction device according to the embodiment.

[0015] Figure 5 A schematic diagram for explaining a process of producing an insole using the prediction system according to the embodiment is shown.

[0016] Figure 6 This is a diagram schematically illustrating the calculation of a prediction model used in the prediction device according to the embodiment. DETAILED DESCRIPTION

[0017] The prediction device according to one aspect of the present disclosure is a prediction device for predicting the shape of a subject's foot. The prediction device includes: an input unit that receives input of measurement data of the shape of a subject's foot under a first load state; a calculation unit that uses a prediction model trained by machine learning to predict the shape of the subject's foot under a second load state different from the first load state based on the measurement data received by the input unit; and an output unit that outputs the prediction data of the shape of the subject's foot under the second load state predicted by the calculation unit. Here, as an embodiment, a prediction device, a prediction system, and a prediction method are described that can predict the shape of the subject's foot under a low load state (second load state) with a load smaller than the load state based on the shape of the subject's foot under the load state (first load state). In particular, in this embodiment, a prediction device, a prediction system, and a prediction method are described that use a prediction model trained by machine learning and predict the shape of the foot under a no-load state with almost no load based on the measurement data obtained by measuring the shape of the subject's foot and output prediction data. In order to generate a prediction model trained by machine learning, training data is required. In order to generate a prediction model with high prediction accuracy, it is preferred to set a group of training data including training measurement data obtained by measuring multiple subjects under the same measurement conditions and training shape data of the shape of the foot in a no-load state obtained under the same acquisition conditions.

[0018] Here, the measurement conditions for measuring the shape of the foot of the person being measured include the posture of the person being measured, such as standing posture, sitting posture, etc.; the state of the person being measured, such as static state, moving state, etc. Furthermore, the measurement conditions also include the type of measurement device for measurement, the measurement method using an image captured by a smartphone, etc., etc.

[0019] In addition, the conditions for obtaining the shape of the foot in a no-load state include the following conditions: the subject is placed in a prone position on the bed, a plaster bandage (gips) is wrapped around the subject's foot, and the plaster is poured into the hardened plaster bandage to perform injection molding and measurement of the shape of the foot, or the subject is in a sitting position and the foot is floated from the ground. Under these conditions, the shape of the foot in a substantially no-load state can be obtained, but some load may be applied to the sole of the foot.

[0020] In the following description, training data is used in which training measurement data obtained by measuring a subject in a standing position in a static state and training shape data of the shape of a foot in a no-load state obtained using a plaster bandage are set as a set, and training is performed through machine learning to generate a prediction model. However, if the shape of the foot in a no-load state can be predicted, data under different measurement conditions can be included in the training measurement data, and data under different acquisition conditions can be included in the training shape data as training data, and a prediction model can be generated through training using machine learning.

[0021] In addition, the shape of the foot predicted by the prediction device is not limited to the shape of the foot in the no-load state, but may also be the shape of the foot of the subject in a low-load state where the load is smaller than the load state. For example, a set of training data may be used, which includes training measurement data obtained by measuring the subject in a sitting position in a static state and training shape data of the shape of the foot of the subject who lightly touches the transparent plate, and a prediction model may be generated by training through machine learning. Thus, the following prediction model may be generated: based on the measurement data of the subject in the sitting position, the shape of the foot of the subject in a low-load state where the load is smaller than the load state in the sitting position is predicted. In addition, in the present disclosure, the low-load state also includes a no-load state where there is almost no load.

[0022] Hereinafter, the prediction device, prediction system, and prediction method of the embodiment will be described based on the accompanying drawings. In the following description, the same symbols are marked on the same structures. Their names and functions are also the same. Therefore, the detailed description of them will not be repeated.

[0023] (Implementation Method)

[0024] [Structure of the prediction system]

[0025] Figure 11 is a schematic diagram showing the structure of a prediction system 10 including a prediction device 1 according to an embodiment. The prediction system 10 includes: a measuring device 2 for measuring the shape of a subject's foot under a load state (a first load state); and a prediction device 1 for predicting the shape of a subject's foot under a no-load state (a second load state) based on the measurement data measured by the measuring device 2. The prediction device 1 uses a prediction model 134 trained by machine learning in order to predict the shape of the subject's foot under a no-load state. Therefore, before predicting the shape of the subject's foot under a no-load state, the prediction device 1 needs to generate the prediction model 134 by training using machine learning using the training data 132. In addition, in the present embodiment, a structure in which the prediction model 134 is generated by the prediction device 1 using the training data 132 is described, but it may also be a structure in which the prediction device 1 accepts the prediction model 134 generated by another device.

[0026] Hereinafter, the prediction device 1 is described as a device that generates a prediction model 134 by training using machine learning using training data, and predicts the shape of the foot of the subject in the no-load state based on the measurement data using the trained prediction model 134. The prediction device 1 outputs the predicted shape of the foot of the subject in the no-load state as prediction data 41. By using the prediction data 41 to produce customized shoes or insoles, it is possible to produce shoes or insoles that are more suitable for the foot of the subject than shoes or insoles produced based on the shape of the foot in the loaded state.

[0027] Figure 2 is a block diagram showing the structure of the prediction device 1 according to the embodiment. Figure 2 As shown, the prediction device 1 includes a processor 11 , a memory 12 , a storage 13 , an input interface 14 , an output interface 15 , a communication interface 16 , and a media reader 17 . These components are connected via a processor bus 19 .

[0028] The processor 11 is a computer that reads out programs (e.g., an operating system (OS) 130, a training program 131, and a prediction program 133) stored in the storage 13, and expands and executes the read programs in the memory 12. The processor 11 is an example of a "computing unit", and includes, for example, a central processing unit (CPU), a field programmable gate array (FPGA), a graphics processing unit (GPU), or a multi-processing unit (MPU). In addition, the processor 11 may also include a computing circuit (processing circuit).

[0029] The memory 12 includes a volatile memory such as a dynamic random access memory (DRAM) or a static random access memory (SRAM), or a non-volatile memory such as a read only memory (ROM) or a flash memory.

[0030] The storage 13 includes, for example, a nonvolatile storage device such as a hard disk drive (HDD) or a solid state drive (SSD). The storage 13 stores the OS 130, the training program 131, and the prediction program 133 as well as the training data 132 and the prediction model 134.

[0031] The training program 131 is a program for generating a prediction model 134 by training using machine learning using the training data 132. The prediction program 133 is a program for predicting the shape of the foot in a no-load state based on the measurement data of the shape of the foot in a loaded state of the subject using the prediction model 134. The prediction program 133 is a program for executing a prediction method by combining the training program 131 and the prediction model 134.

[0032] The input interface 14 is an example of an input unit, and receives input operations from a keyboard, a mouse, a touch device, etc. The output interface 15 is an example of an output unit, and outputs the prediction data 41 of the foot shape in the unloaded state predicted by the prediction device 1 to a display or the like.

[0033] The communication interface 16 is an example of an input unit and an output unit, and receives input of measurement data from the measuring device 2 or transmits and receives data with other devices by performing wired communication or wireless communication. The communication interface 16 can receive input of measurement data or output predicted prediction data 41 by transmitting and receiving data with other devices. In addition, the communication interface 16 can also receive input of training data 132 from other devices, or receive an image of the shape of the foot of the person being measured under load conditions taken by a smartphone or the like. In addition, when receiving the image, the prediction device 1 can obtain data (equivalent to the measurement data) of the shape of the foot of the person being measured under load conditions from the image based on a prescribed algorithm.

[0034] The media reader 17 receives a storage medium such as a removable disk 18, a memory chip, a universal serial bus (USB) memory, etc., and acquires data stored in the removable disk 18, the memory chip, the USB memory, etc. In addition, the media reader 17 can also read the training data 132 stored in the removable disk 18, or save and output the predicted data 41 to the removable disk 18, etc.

[0035] Furthermore, although the prediction device 1 has been described as generating the prediction model 134 by training using machine learning using the training data 132, if a prediction model 134 generated by another device is prepared in advance, it may be received from a server or the like via the communication interface 16. In addition, the prediction device 1 may read the prediction model 134 stored in the removable disk 18 or the like through the media reading device 17.

[0036] The measuring device 2 is, for example, a three-dimensional foot scanner based on laser measurement, and includes a top plate 21 and a laser measuring unit 22 that is arranged in a manner of sandwiching the top plate 21. When the subject places his feet on the top plate 21 in a standing position, a load is applied from the feet to the top plate 21 due to the subject's weight. That is, a load is applied to the subject's feet. In a state where a load is applied to the subject's feet, the measuring device 2 measures the shape of the foot while moving from the toe to the heel of the foot using the laser measuring unit 22. The measuring device 2 outputs the measurement data (three-dimensional data) of the shape of the subject's foot obtained by the laser measuring unit 22 to the prediction device 1. In addition, the measurement data only needs to include at least the data of the shape of the foot obtained by the measuring device 2, and may also include other data (for example, attribute information such as the gender or age of the subject).

[0037] [Training data]

[0038] As training data 132 for generating a prediction model 134 trained by machine learning, training data is used which is a set of training measurement data obtained by measuring a subject in a standing position in a static state (hereinafter, not specifically recorded as a static state, but it is assumed that the subject is in a static state during measurement) and training shape data of the shape of the foot in a no-load state obtained using a plaster bandage. Figure 3 This is a schematic diagram for explaining a method of acquiring training data.

[0039] In order to obtain training data, a plurality of training subjects are prepared. In addition, the training subjects may also include subjects whose foot shapes under no-load conditions are predicted. Figure 3 As shown, the feet of multiple training subjects are measured in a standing position by the measuring device 2. That is, the feet of the training subjects in a standing position are measured by the measuring device 2 to obtain the shape of the feet of the training subjects under load. When the training subjects place their feet on the top plate 21 in a standing position, the measuring device 2 uses the laser measuring unit 22 to measure the shape of the feet while moving from the toe to the heel of the foot, while the load is applied from the feet to the top plate 21 due to the weight of the training subjects. The data of the shape of the feet measured by the measuring device 2 is three-dimensional data. The three-dimensional data can be directly used as training measurement data, but in this embodiment, the three-dimensional data is converted into two-dimensional data containing information on the height direction of the shape of the foot, and the converted two-dimensional data is used as training measurement data 31.

[0040] For the same multiple training subjects, such as Figure 3 As shown, a plaster bandage is used to measure a foot without a load, and data on the shape of the foot of a training subject under no load is obtained. Specifically, the measurement device 2 measures the shape of a foot made of plaster molded using a plaster bandage, thereby obtaining data on the shape of the foot of a training subject under no load. Therefore, the data on the shape of the foot obtained using the plaster bandage is three-dimensional data. The three-dimensional data can also be used directly as training shape data, but in this embodiment, the three-dimensional data is converted into two-dimensional data containing information on the height direction of the shape of the foot, and the converted two-dimensional data is used as training shape data 32.

[0041] Paired data of training measurement data 31 and training shape data 32 obtained from each of a plurality of training subjects are used as training data 132. In addition, training data 132 converted into two-dimensional data including information in the height direction of the shape of the foot is also referred to as two-dimensional training data. Training data 132 may be training measurement data 31 and training shape data 32 that are three-dimensional data simply converted into training measurement data 31 and training shape data 32 that are two-dimensional data, or may be data only near the sole of the foot that is required for predicting the shape of the subject's foot in a no-load state. Therefore, information in the height direction of the shape of the foot, for example, Figure 3 As shown in FIG. 1 , the information of the height h from the reference plane S to the sole of the foot is set. The reference plane S is, for example, a plane parallel to the ground in a standing position, and is a plane at a predetermined height from the ankle or instep of the training subject along the sole direction, or a plane with the largest outer shape. Figure 3 When the reference plane S is set on the inner side of the foot of the training subject, the height h from the reference plane S to the sole can also be regarded as the depth from the reference plane S. Therefore, the training measurement data 31 and the training shape data 32 converted into two-dimensional data are also called sole depth images.

[0042] In addition, the training shape data 32 has larger concavities and convexities of the sole than the training measurement data 31. Therefore, with respect to the training measurement data 31, the data from the reference plane S to the height h1 (first height) in the training measurement data of three-dimensional data is converted into two-dimensional data, and with respect to the training shape data 32, the data from the reference plane S to the height h2 (second height (h1<h2)) higher than the height h1 in the training shape data is converted into two-dimensional data.

[0043] When the training measurement data 31 and the training shape data 32 are respectively converted into a sole depth image, the information of the height h from the reference plane S to the sole is converted into two-dimensional data, for example, as grayscale information (information of 256 grayscales from white to black). Therefore, the sole depth image becomes an image in which the gray shades corresponding to the height h are added to the two-dimensional shape of the foot. For example, the sole depth image shows the part with high height h as black, the part with low height h as white, and the middle height h as gray in the two-dimensional shape of the foot. There is no restriction on how to show the information of height h in the sole depth image, and it can be shown not only as grayscale information as described above, but also as hue information or contour lines. For example, when the information of height h is shown as hue information, the sole depth image shows the part with high height h in red, the part with low height h in blue, and the middle height h in yellow in the two-dimensional shape of the foot. In any case, the training measurement data 31 and the training shape data 32 can be two-dimensional array data in which the information of height h is stored as a scalar value at each point of the two-dimensional shape of the foot.

[0044] Specifically, the training measurement data 31 represents the XY plane of 400mm×400mm as a two-dimensional image of 256 pixels×256 pixels, and the height h information stored in each pixel is represented by a grayscale of 256 gray levels from the reference plane S to 0mm~20mm. In addition, the plane parallel to the ground in the standing posture is set as the XY plane, and the direction perpendicular to the XY plane is set as the Z direction as the direction of the height h. The training shape data 32 represents the XY plane of 400mm×400mm as a two-dimensional image of 256 pixels×256 pixels, and the height h information stored in each pixel is represented by a grayscale of 256 gray levels from the reference plane S to 0mm~40mm. In addition, compared with the training shape data 32, the range of the height h information stored in the training measurement data 31 is narrower from the reference plane S to 0mm~20mm. However, the training measurement data 31 represents the height h information as a grayscale of 256 gray levels like the training shape data 32. Therefore, the amount of information of the training measurement data 31 converted into two-dimensional data is the same as the amount of information of the training shape data 32 converted into two-dimensional data. Therefore, the amount of data per 1 mm unit of the training measurement data 31 is greater than that of the training shape data 32. Of course, the amount of data per 1 mm unit of the training measurement data 31 and the amount of data per 1 mm unit of the training shape data 32 may be made the same. In addition, the range from the reference plane S stored as the information of the height h in the training measurement data 31 and the training shape data 32 is not particularly limited, and may be changed according to the data (e.g., shape or size) of the foot as the object.

[0045] In this way, the training data 132 converts the training measurement data 31 and the training shape data 32 into a sole depth image, thereby suppressing the amount of data processed by the prediction device 1 and generating a prediction model with high prediction accuracy. Of course, the training data 132 may be three-dimensional data represented by voxels, point groups, grids, boundary surfaces, etc. instead of converting the training measurement data 31 and the training shape data 32 into a sole depth image. In addition, the sole depth image is used as the training data 132 to generate the prediction model. Therefore, when the prediction device 1 uses the prediction model to predict the shape of the foot in the unloaded state, it is necessary to convert the measurement data input to the prediction device 1 into a sole depth image. In addition, the shape of the foot in the unloaded state predicted by the prediction device 1 is output as a sole depth image, so it is necessary to restore the shape of the foot in the unloaded state of the three-dimensional data from the output sole depth image.

[0046] [Prediction method]

[0047] Next, the processing of the prediction method executed by the prediction system 10 will be described with reference to the flowchart. Figure 4 1 is a flowchart for explaining the processing of the prediction method executed by the prediction device 1 according to the embodiment. Figure 4 The steps shown are implemented by the processor 11 of the prediction device 1 executing the training program 131 and the prediction program 133 .

[0048] First, the prediction device 1 determines whether to perform the training phase processing based on the training program 131 (step S101). In the case where the prediction device 1 does not yet have the prediction model 134, or in the case where the prediction model 134 has not been sufficiently trained, the prediction device 1 performs the training phase processing. In the case where the training phase processing is not performed (in the case of No in step S101), the prediction device 1 causes the processing to enter step S104. On the other hand, in the case of performing the training phase processing (in the case of Yes in step S101), the prediction device 1 executes the training phase processing to generate the prediction model 134, which is used to predict the shape of the subject's foot in a no-load state based on the measurement data of the shape of the subject's foot in a loaded state.

[0049] In the training phase process, in order to generate the prediction model 134, input of N training data 132 is accepted (step S102). The training data 132 accepted in step S102 includes the training measurement data 31 and the training shape data 32.

[0050] Next, the prediction device 1 generates a prediction model 134 for predicting the shape of the foot of the subject in the no-load state by training using machine learning using the training data 132 (step S103). Here, the prediction model 134 includes, for example, a well-known network structure such as a neural network or a support vector machine (SVM), or a Bayesian network, and internal parameters used by the network structure, and the internal parameters are optimized (adjusted) based on the prediction data of the shape of the foot in the no-load state predicted from the training measurement data 31 and the training shape data 32.

[0051] In particular, the training data 132 is paired data of the training measurement data 31 and the training shape data 32, and in the training using machine learning based on the paired data, an algorithm for image-to-image translation is used using the training shape data 32 as teacher data. Specifically, as an algorithm for conversion between paired data (paired images) as teacher learning (supervised learning), a convolutional autoencoder (CAE), a variational autoencoder (VAE), U-Net, etc. as encoder-decoder (Encoder-Decoder) models can be used. Here, the variational autoencoder (VAE) is a model that imposes constraints on the latent variables of the convolutional autoencoder (CAE) in a way that the latent variables become normally distributed, and the U-Net is a model in which the encoder (Encoder) and the decoder (Decoder) of the convolutional autoencoder (CAE) are connected via jumps. In addition, the encoder uses convolution, vision transformer, etc., and the decoder uses transposed convolution, etc.

[0052] In addition, as an algorithm for conversion between paired data (paired images), a Generative Adversarial Network (GAN) model may also be used. As a GAN model, for example, pix2pix is ​​used. In addition, the algorithm for conversion between paired data (paired images) is not limited to the above algorithm, and algorithms for image generation such as flow-based models, diffusion models, autoregressive models (e.g., pixelRNN, pixelCNN, ImageGPT, etc.), and energy-based models may be appropriately used.

[0053] Next, the prediction device 1 determines whether to perform the application phase processing based on the prediction program 133 (step S104). When a fully trained prediction model 134 is generated through the processing of the training phase, the prediction device 1 performs the application phase processing. When the application phase processing is not performed (in the case of No in step S104), the prediction device 1 causes the processing to enter step S110. On the other hand, when the application phase processing is performed (in the case of Yes in step S104), the prediction device 1 performs the following application phase processing: using the prediction model 134, the shape of the subject's foot in the unloaded state is predicted based only on the measurement data of the shape of the subject's foot in the loaded state.

[0054] In the application phase processing, in order to predict the shape of the subject's foot in the unloaded state, input of measurement data of the shape of the subject's foot in the loaded state is accepted (step S105). In step S105, only the measurement data obtained by measuring the shape of the subject's foot by the measurement device 2 is required, and the shape of the foot in the unloaded state obtained by using a plaster bandage is not required.

[0055] Next, the prediction device 1 converts the received three-dimensional data into two-dimensional data including information on the height direction of the foot shape so that it can be used in the prediction model 134 (step S106). Specifically, the prediction device 1 converts the information on the height h from the reference plane S to the sole of the foot in the measurement data into a sole depth image as grayscale information.

[0056] Next, the prediction device 1 predicts the shape of the subject's foot in the no-load state using the prediction model 134 (step S107). The prediction device 1 converts the prediction data predicted in step S107 into three-dimensional data and outputs it to a device for designing shoes or insoles, etc. (step S108). Alternatively, the prediction device 1 may display the prediction data as an image on a display connected to the prediction device 1 for confirmation.

[0057] The prediction device 1 determines whether new measurement data is received from the measurement device 2 (step S109). When the prediction device 1 receives the input of new measurement data, it predicts the shape of the foot of the person being measured in the unloaded state based on the new measurement data. Therefore, when the input of new measurement data is received (yes in step S109), the prediction device 1 returns the process to step S106.

[0058] On the other hand, when the input of new measurement data is not received (No in step S109), the prediction device 1 determines whether to end the processing of the prediction method (step S110). When the prediction device 1 does not receive the input to end the processing of the prediction method (No in step S110), the prediction device 1 returns the processing to step S101. On the other hand, when the prediction device 1 receives the input to end the processing of the prediction method (Yes in step S110), the prediction device 1 ends the processing of the prediction method.

[0059] Next, an example of manufacturing an insole based on the shape of the subject's foot in a no-load state predicted by the prediction system 10 including at least the prediction device 1 and the measurement device 2 will be described using schematic diagrams. Figure 5 Schematic diagram for explaining the process of making insoles using the prediction system 10 according to the embodiment. Figure 5 In the training data collection phase P1 shown, in order to collect training data 132, the standing foot measurement using the measurement device 2 and the unloaded foot measurement using a plaster bandage are performed on N training subjects. Thus, N sets of paired data can be collected: the three-dimensional data of the foot shape measured by the measurement device 2 and the three-dimensional data of the foot shape in the unloaded state obtained from the same training subject are combined.

[0060] In the training data preprocessing stage P2, the three-dimensional data 31a of the foot shape measured by the measuring device 2 and the three-dimensional data 32a of the foot shape in the no-load state collected in the training data collection stage P1 are respectively converted into two-dimensional data containing information in the height direction of the foot shape. Specifically, the three-dimensional data 31a of the foot shape measured by the measuring device 2 is converted into the training measurement data 31 of the sole depth image, and the three-dimensional data 32a of the foot shape in the no-load state is converted into the training shape data 32 of the sole depth image.

[0061] In the prediction model training phase P3, a prediction model 134 trained by machine learning is generated using N sets of paired data consisting of the training measurement data 31 and the training shape data 32. The processing from the training data collection phase P1 to the prediction model training phase P3 corresponds to Figure 4 Processing of the training phase is shown.

[0062] Next, in the measurement data collection phase P4, the measurement data 33a of the subject (who may be a different person from the training subject or the same person) for predicting the shape of the foot in the unloaded state is collected. The shape of the foot of the subject in the loaded state in the standing posture is obtained as the measurement data 33a by the measurement device 2. The measurement data 33a is three-dimensional data.

[0063] In the measurement data preprocessing stage P5, the three-dimensional data 33a collected in the measurement data collection stage P4 is converted into two-dimensional data including information in the height direction of the foot shape. Specifically, the measured three-dimensional data of the foot shape is converted into the measurement data 33 of the sole depth image.

[0064] In the prediction stage P6, the shape of the foot of the subject in the no-load state is predicted based on the measurement data 33 of the sole depth image using the prediction model 134. The prediction data 41 predicted in the prediction stage P6 is the shape data of the sole depth image, and is two-dimensional data including information in the height direction of the shape of the foot. The processing from the measurement data collection stage P4 to the prediction stage P6 corresponds to Figure 4 The processing of the application phase is shown.

[0065] In the sole surface restoration stage P7, in order to facilitate the design of the insole, the sole shape data 42 of the three-dimensional data is restored from the prediction data 41 which is the two-dimensional data including the information of the height direction of the shape of the foot. The sole shape data 42 is the shape data of the three-dimensional data from the reference plane S to the sole. In the sole surface restoration stage P7, the shape data of the ankle side may be cut out from the reference plane S from the measurement data 33a of the three-dimensional data and combined with the sole shape data 42 to restore the sole shape data of the entire foot.

[0066] In the insole design stage P8, the insole design data 51 is designed based on the sole shape data 42 restored in the sole surface restoration stage P7. Figure 5 In the description, the design of the insole is taken as an example, but the design data of the shoe may be designed based on the sole shape data 42 or the sole shape data of the entire foot.

[0067] [Prediction Model]

[0068] Next, the prediction model 134 will be described using a schematic diagram. Figure 6This is a diagram schematically showing the operation of the prediction model used in the prediction device 1 of the embodiment. The prediction model 134 uses a convolutional auto-encoder (CAE) and takes the sole depth image of the training measurement data 31 as an input image and the prediction data 41 of the shape of the foot of the person being measured in the unloaded state as an output image. Here, the convolutional auto-encoder is a multi-layer neural network including a structure for encoding using one or more convolutional layers or pooling layers and a structure for decoding using one or more deconvolutional layers.

[0069] The prediction model 134 is described as using a convolutional autoencoder (CAE) as an encoder-decoder model, but the encoder may use, for example, a vision transformer instead of a convolution. When the encoder uses a vision transformer, since it does not have a structure with a pooling layer like a convolution, for the prediction data 41 of the shape of the foot of the person being measured in a no-load state, in the output image, the contrast between the predicted shape of the foot of the person being measured and the background becomes clearer, the predicted contour shape of the foot is less likely to be confused, and part of the background is less likely to cause interference.

[0070] In addition, the prediction model 134 may use a GAN (Generative Adversarial Network) model instead of an encoder-decoder model, for example, pix2pix. When the prediction model 134 uses pix2pix, since it does not have a pooling layer structure like a convolution, the predicted shape of the subject's foot and the contrast with the background become clearer compared to a convolutional autoencoder (CAE) in which a convolution is used as an encoder.

[0071] The sole depth image of the training shape data 32 is used as the correct answer data, and the output image of the predicted prediction data 41 is compared with the correct answer data, so that the training of the adjustment prediction model 134 is performed in a manner that minimizes the loss function between the output image and the correct answer data. Therefore, by using the trained prediction model 134, the prediction device 1 can predict the shape of the subject's foot in the no-load state based on the shape of the subject's foot in the loaded state measured by the measurement device 2. Therefore, by using the trained prediction model 134, the prediction device 1 can obtain the shape of the subject's foot in the no-load state even without using a plaster bandage.

[0072] [Modifications]

[0073] (1) In the above embodiment, training data 132 is collected from a plurality of subjects selected regardless of their attributes (e.g., gender, age, race, etc.), and a prediction model 134 is generated by training using machine learning using the collected training data 132. Without limitation thereto, training data may be collected from a plurality of subjects divided for each attribute, and a prediction model may be generated by training using machine learning using the training data divided for each attribute. The prediction device 1 has a prediction model for each attribute, and selects a prediction model corresponding to the attribute by receiving attribute information of the subject. Then, the prediction device 1 uses the selected prediction model to predict the shape of the subject's foot in a no-load state based on the measurement data. Thus, the prediction device 1 can predict the shape of the subject's foot in a no-load state with good accuracy by selecting a prediction model with high prediction accuracy based on the subject's attribute information.

[0074] (2) In the above embodiment, the prediction of the shape of the foot of the subject under no-load state based on the measurement data of the shape of the foot of the subject under load state using the trained prediction model is described. However, the state of predicting the shape of the foot is not limited to the no-load state. For example, the shape of the foot of the subject under low-load state where the load is not zero can also be predicted by using a prediction model generated by machine learning training using training shape data of the shape of the foot of the training subject under low-load state where the load is smaller than the load state. Furthermore, the shape of the foot of the subject under high-load state can also be predicted by using a prediction model generated by machine learning training using training shape data of the shape of the foot of the training subject under high-load state where the load is larger than the load state. That is, the prediction device 1 generates a prediction model by using machine learning training based on the training measurement data of the shape of the foot of the training subject under the first load state and the training shape data of the shape of the foot of the training subject under the second load state where the load is different from the first load state, thereby predicting the shape of the foot of the subject under different load states.

[0075] [form]

[0076] (1) The prediction device of the present disclosure includes: an input unit that receives input of measurement data of the shape of a foot of a person being measured in a first load state;

[0077] a calculation unit that uses a prediction model trained by machine learning to predict the shape of the subject's foot under a second load state different from the first load state based on the measurement data received by the input unit; and

[0078] an output unit for outputting the prediction data of the shape of the foot of the person being measured under the second load state predicted by the calculation unit,

[0079] The prediction model is

[0080] The training measurement data of the shape of the foot of the training subject in the first load state and the training shape data of the shape of the foot of the training subject in the second load state are generated in advance through training using machine learning.

[0081] Thus, the prediction device of the present disclosure uses a prediction model trained by machine learning, and can therefore predict the shape of the subject's foot under the second load state based on the measurement data under the first load state.

[0082] (2) The prediction device according to (1), wherein the training measurement data and the training shape data are three-dimensional data,

[0083] The prediction model can be generated in advance by training using machine learning based on two-dimensional training data obtained by converting the training measurement data and the training shape data into two-dimensional data including information on the height direction of the shape of the training subject's foot.

[0084] (3) The prediction device according to (2), wherein the two-dimensional training data may include information on the height direction of the shape of the foot of the training subject as grayscale information.

[0085] (4) The prediction device according to (2) or (3), wherein the information in the height direction of the foot shape can be information at a height from a reference plane to a sole of the foot.

[0086] (5) A prediction device according to (4), wherein, with respect to two-dimensional training data, data from a reference plane toward the sole of the foot to a first height in the training measurement data can be converted into two-dimensional data, and data from a reference plane toward the sole of the foot to a second height higher than the first height in the training shape data can be converted into two-dimensional data.

[0087] (6) The prediction device according to (5), wherein the amount of information of the training measurement data converted into two-dimensional data and the amount of information of the training shape data converted into two-dimensional data may be the same.

[0088] (7) In the prediction device according to (2), the calculation unit may predict the shape of the subject's foot in the second load state based on the measurement data converted into two-dimensional data using a prediction model.

[0089] (8) The prediction device according to (7), wherein the predicted two-dimensional data can be restored to three-dimensional data.

[0090] (9) The prediction device according to any one of (1) to (8), wherein the prediction model can be generated in advance for each attribute information of the training subject by training using machine learning,

[0091] The input unit may receive attribute information of the subject, and use a prediction model corresponding to the attribute to predict the shape of the subject's foot in the second load state based on the measurement data received by the input unit.

[0092] (10) The prediction system of the present disclosure comprises:

[0093] a measuring device for measuring the shape of a foot of a person to be measured under a first load state; and

[0094] The prediction device according to any one of (1) to (9).

[0095] (11) The prediction method disclosed herein is a prediction method for predicting the shape of a subject's foot, comprising: a step of receiving input of measurement data of the shape of the subject's foot under a first load state; a step of using a prediction model trained by machine learning to predict the shape of the subject's foot under a second load state in which the load is smaller than the first load state based on the received measurement data; and a step of outputting the predicted data of the shape of the subject's foot under the second load state.

[0096] The prediction model is

[0097] The training measurement data of the shape of the foot of the training subject in the first load state and the training shape data of the shape of the foot of the training subject in the second load state are generated in advance through training using machine learning.

[0098] (12) The prediction device according to any one of (1) to (9), wherein the shape of the foot is a sole shape.

[0099] (13) The prediction device according to any one of (1) to (9), wherein the training measurement data and the training shape data are paired image data.

[0100] Although the embodiments of the present invention have been described, the embodiments disclosed this time should be considered to be illustrative in all aspects and not restrictive. The scope of the present invention is indicated by the claims, and is intended to include all modifications within the meaning and scope equivalent to the claims.

Claims

1. A prediction device for predicting the shape of a foot of a person to be measured, comprising: an input unit for receiving input of measurement data of the shape of the foot of the person being measured in a first load state; a calculation unit that uses a prediction model trained by machine learning to predict the shape of the foot of the subject under a second load state different from the first load state based on the measurement data received by the input unit; as well as an output unit for outputting prediction data of the shape of the foot of the subject under the second load state predicted by the calculation unit, The prediction model is The training measurement data is generated in advance by training using machine learning based on the training measurement data of the shape of the foot of the training subject in the first load state and the training shape data of the shape of the foot of the training subject in the second load state.

2. The prediction device according to claim 1, wherein: The training measurement data and the training shape data are three-dimensional data, The prediction model is generated in advance by training using machine learning based on two-dimensional training data obtained by converting the training measurement data and the training shape data into two-dimensional data including information on the height direction of the shape of the training subject's foot.

3. The prediction device according to claim 2, wherein: The two-dimensional training data includes information on the height direction of the shape of the foot of the training subject as grayscale information.

4. The prediction device according to claim 2, wherein: The information on the height direction of the foot shape is information on the height from the reference plane to the sole of the foot.

5. The prediction device according to claim 4, wherein: Regarding the two-dimensional training data, data from the reference plane toward the sole to a first height in the training measurement data is converted into two-dimensional data, and data from the reference plane toward the sole to a second height higher than the first height in the training shape data is converted into two-dimensional data.

6. The prediction device according to claim 5, wherein: The amount of information of the training measurement data converted into two-dimensional data is the same as the amount of information of the training shape data converted into two-dimensional data.

7. The prediction device according to claim 1, wherein: The calculation unit predicts the shape of the subject's foot in the second load state based on the measurement data converted into two-dimensional data using the prediction model.

8. The prediction device according to claim 7, wherein: The computing unit The predicted data of the two-dimensional data is restored to three-dimensional data.

9. The prediction device according to claim 1, wherein: The prediction model is generated in advance by training using machine learning for each attribute information of the training subject. The input unit receives attribute information of the subject, and uses the prediction model corresponding to the attribute to predict the shape of the subject's foot in the second load state based on the measurement data received by the input unit.

10. The prediction device according to any one of claims 1 to 9, wherein: The shape of the foot is the sole shape.

11. The prediction device according to any one of claims 1 to 9, wherein: The training measurement data and the training shape data are paired image data.

12. A prediction system comprising: a measuring device for measuring the shape of a foot of a person to be measured under a first load state; as well as The prediction device according to any one of claims 1 to 9.

13. A prediction method for predicting the shape of a foot of a person to be measured, comprising: a step of accepting input of measurement data of the shape of the foot of the subject in a first load state; A step of predicting the shape of the foot of the subject under a second load state different from the first load state based on the received measurement data using a prediction model trained by machine learning; as well as a step of outputting predicted data of the shape of the foot of the subject under the second load state, The prediction model is The training measurement data is generated in advance by training using machine learning based on the training measurement data of the shape of the foot of the training subject in the first load state and the training shape data of the shape of the foot of the training subject in the second load state.

Images