Heel position estimation device, heel position estimation method, and program

Abstract

To provide a technique that enables appropriate estimation of a heel position from an image of one foot of a subject under measurement.SOLUTION: In a heel position estimation device, an outline extraction unit 52 extracts an outline shape of a foot portion including at least a part of the heal from an image of one foot of a subject under measurement. A determination unit 54 determines the type of the outline shape. A heel estimation unit 56 estimates a position of a heel point in the outline shape on the basis of the determined type of the outline shape.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to a heel position estimation device, a heel position estimation method, and a program. [Background technology]

[0002] There is known a technique for measuring the foot size, such as the foot length, of a subject from an image including at least one foot of the subject (hereinafter referred to as a "single-foot image"). For example, Patent Document 1 describes a technique for obtaining a single-foot image from an imaging unit, in which the foot placement paper and the entire foot are photographed while the sole of the foot is placed on the foot placement paper with the edge of the heel aligned with the bottom side of the foot placement paper, and calculating an actual length ratio based on the single-foot image by dividing the actual length of the long side of the foot placement paper by the number of pixels of the long side of the foot placement paper. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2019-45463 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the technology described in Patent Document 1, the subject himself / herself needs to align the edge of his / her heel with the bottom side of the foot placement paper, so there is room for improvement in measurement accuracy. In order to accurately measure the foot length of the subject, it is necessary to estimate the position of the heel point of the subject in the image of one foot.

[0005] The present invention has been made in consideration of the above circumstances, and has a general purpose to provide a technique capable of appropriately estimating the position of the heel point from an image of one foot of a subject. [Means for solving the problem]

[0006] A heel position estimation device of one embodiment of the present invention comprises a contour extraction unit that extracts the contour shape of a foot part including at least a part of the heel from an image of one foot of a person to be measured, a determination unit that determines the type of the contour shape, and a heel estimation unit that estimates the position of the heel point in the contour shape based on the determined type of the contour shape.

[0007] Another aspect of the present invention is a heel position estimation method, comprising the steps of: extracting a contour shape of a foot part including at least a part of the heel from an image of one foot of a subject; determining a type of the contour shape; and estimating a position of the heel point in the contour shape based on the determined type of the contour shape.

[0008] Another aspect of the present invention is a program that causes a computer to realize a function of extracting a contour shape of a foot part including at least a part of a heel from an image of one foot of a subject, a function of determining a type of the contour shape, and a function of estimating a position of a heel point in the contour shape based on the determined type of the contour shape.

[0009] In addition, any combination of the above components, or mutual substitution of the components or expressions of the present invention among methods, devices, programs, temporary or non-temporary storage media storing programs, systems, etc., are also valid aspects of the present invention. Effect of the Invention

[0010] According to an aspect of the present invention, it is possible to provide a technique capable of appropriately estimating the position of the heel point from an image of one foot of a subject. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 is a schematic diagram showing an overview of a heel position estimation system. [Diagram 2] 2 is a functional block diagram showing a schematic configuration of the heel position estimation system of FIG. 1. [Diagram 3] 2 is a functional block diagram showing a schematic configuration of a calculation unit of the heel position estimation server of FIG. 1. [Figure 4] 4 is a schematic diagram showing an example of a contour shape of a foot part extracted from a single foot image by a contour extraction unit in FIG. 3. [Diagram 5] 2 is a flowchart showing an example of a process of the heel position estimation server of FIG. 1. [Figure 6] 6 is a flowchart specifically showing a process by a determination unit in step S14 and a process by a heel estimation unit in step S16 shown in FIG. 5, in an example of the operation of the heel position estimation server in FIG. [Figure 7] Fig. 7(a) is a diagram showing an example of a contour shape determined to be the first type, and Fig. 7(b) is a diagram showing an example of a contour shape determined to be the second type. [Figure 8] Fig. 8(a) is a diagram showing another example of a contour shape determined to be the second type, and Fig. 8(b) is a diagram showing an example of a contour shape determined to be the third type. [Figure 9] 4 is a schematic diagram illustrating correction of foot length by the foot length correction unit in FIG. 3. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, the present invention will be described based on preferred embodiments with reference to the drawings. In the embodiments and modifications, the same or equivalent components are given the same reference numerals, and duplicated descriptions are omitted as appropriate.

[0013] FIG. 1 is a schematic diagram showing an overview of a heel position estimation system 100. The heel position estimation system 100 includes a user terminal 10 and a heel position estimation server 20, which are connected to each other via wireless communication 16 and a network 18. The subject takes an image (one-foot image) including his / her own one foot 12 using the camera function of the user terminal 10. The user terminal 10 transmits the captured one-foot image to the heel position estimation server 20 via wireless communication 16 and the network 18. The heel position estimation server 20 estimates the position of the heel point in the received one-foot image. The heel position estimation server 20 may further estimate the toe position from the received one-foot image, and may calculate the foot length based on the estimated toe position and the estimated heel position. The heel position estimation server 20 may transmit the calculated foot size information such as the foot length to the user terminal 10 and display it on the screen of the user terminal 10.

[0014] The user terminal 10 and the heel position estimation server 20 may be configured as a mobile terminal or computer including a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a RAM (Random Access Memory), a ROM (Read Only Memory), an auxiliary storage device, a display device, a communication device, etc., and a program stored in the mobile terminal or computer. For example, the program executed on the heel position estimation server 20 side may be used from the user terminal 10 side via a network 18. However, the function of the heel position estimation system may be realized by a single device having the functions of the user terminal 10 and the heel position estimation server 20, and the measurement subject may directly operate the device to execute the heel position estimation program. The single device is not limited to a personal computer, and may be provided in the form of a mobile terminal such as a smartphone or a tablet terminal, and a program stored in the computer or terminal. Alternatively, it may be realized in the form of a terminal installed in a shoe store and operated by an assistant such as a sales clerk, and a program stored in the terminal.

[0015] In this specification, the "heel position estimation device" may refer to the entire heel position estimation system 100 or the heel position estimation server 20. In this embodiment, the heel position estimation server 20 is provided with many of the characteristic functions included in the "heel position estimation device", so the heel position estimation server 20 essentially corresponds to the "heel position estimation device". However, the characteristic functions of the "heel position estimation device" may be distributed between the user terminal 10 and the heel position estimation server 20, or may be provided in the user terminal 10.

[0016] Fig. 2 is a functional block diagram showing the schematic configuration of the heel position estimation system 100. Each functional block shown in each diagram including Fig. 2 can be realized in hardware terms by elements and mechanical devices such as a computer processor and memory, and in software terms by a computer program, etc., but here, functional blocks realized by cooperation of these are depicted. These functional blocks can be realized in various forms by hardware, software, or a combination of both.

[0017] The user terminal 10 includes an image acquisition unit 30, a display unit 32, a communication unit 34, and an operation processing unit 36. The operation processing unit 36 ​​accepts operation input from the subject. The operation processing unit 36 ​​accepts instructions from the subject regarding the start of a series of foot size measurement programs including a heel position estimation program. Based on the instructions accepted by the operation processing unit 36, the display unit 32 starts displaying the contents of the foot size measurement program on a screen and displays contents related to the measurement procedure. In terms of hardware, the operation processing unit 36 ​​and the display unit 32 may be configured, for example, by a touch panel.

[0018] The image acquisition unit 30 captures one foot of the subject to be measured to obtain a one-foot image. The one-foot image is an image of an area including at least the heel point of the one foot 12. In the case where the heel position estimation server 20 includes a toe estimation unit 58 described later, the one-foot image is an image of an area including at least the toe of the one foot 12. The one-foot image may be an image that can be captured by the subject to be measured operating the user terminal 10. In this case, for example, as shown in FIG. 1, the subject to be measured can take an image of his / her own one foot 12 taken from diagonally above the inner foot side as the one-foot image. The one-foot image may be taken from diagonally above the outer foot side of the one foot 12.

[0019] The single-foot image may be a photograph of one foot 12 together with a predetermined reference object, as described later. Here, the "reference object" is an object that is photographed together with the single foot in an image as a comparison object when measuring the foot length and foot width of the single foot based on the single-foot image, and is a planar or three-dimensional object whose size and shape are specified in advance. This "reference object" may be a general-purpose object whose size and shape are specified, or may be a dedicated object whose size and shape are specified specifically for the measurement in this embodiment. As the reference object in this embodiment, a planar object whose size and shape are specified in advance, such as a piece of paper of A4 size (210 mm x 297 mm), can be used. The color of the paper is preferably white, but paper of other colors may also be used. The single-foot image in this embodiment is an image in which one foot is placed on a piece of A4 white paper as a reference object and the entire white paper is photographed. By assuming that the A4 white paper is placed on a floor other than white when photographing, it can be accurately extracted from the single-foot image by image recognition. The single-leg image may be an image that includes, in addition to the subject's single leg 12 and the reference object, for example, a part of the subject's other leg.

[0020] The image acquisition unit 30 may be configured, in terms of hardware, by a camera module, for example. The communication unit 34 transmits the image of one foot to the heel position estimation server 20 via the network 18, receives foot size information from the heel position estimation server 20, and sends it to the display unit 32. The communication unit 34 may be configured, in terms of hardware, by a wireless communication module for wireless LAN communication, mobile phone communication, or the like. The display unit 32 displays the image of one foot and foot size information of the subject on a screen.

[0021] The heel position estimation server 20 includes a communication unit 40, a calculation unit 42, a storage unit 44, and an output unit 46. The communication unit 40 receives an image of one foot from the user terminal 10, and transmits the calculation results by the calculation unit 42, such as foot size information, to the user terminal 10. In terms of hardware, the communication unit 40 may be configured with a communication module such as a wired LAN.

[0022] The calculation unit 42 executes calculation processing including processing for estimating the position of the heel point based on the single-foot image received by the communication unit 40. The details of the calculation processing executed by the calculation unit 42 will be described later.

[0023] The calculation unit 42 stores the calculation result in the storage unit 44. The calculation result stored in the storage unit 44 is transmitted by the output unit 46 to the user terminal 10 via the communication unit 40.

[0024] 3 is a functional block diagram showing a schematic configuration of the calculation unit 42 of the heel position estimation server 20. The calculation unit 42 includes an image reading unit 50, a contour extraction unit 52, a determination unit 54, and a heel estimation unit 56.

[0025] The image reading unit 50 reads the image of one foot received by the communication unit 40 from the user terminal 10. The contour extraction unit 52 extracts a contour shape of the foot part including at least a part of the heel from the image of one foot of the subject read by the image reading unit 50. Fig. 4 is a schematic diagram showing an example of a contour shape 74 of the foot part extracted by the contour extraction unit 52 from the image 70 of one foot.

[0026] FIG. 4(a) is a schematic diagram showing the position of a foot part from which a contour shape 74 is extracted in a single-foot image 70. In the example shown in FIG. 4(a), the single-foot image 70 is a two-dimensional image taken from an obliquely upward direction on the inner side of the foot so that the entire foot is captured with the foot placed on a blank sheet of paper 80 as a reference object. As will be described later, the heel estimation unit 56 estimates a point along the contour shape 74 extracted from the single-foot image 70 by the contour extraction unit 52 as the position of the heel point. Therefore, the single-foot image 70 is an image taken from the inner or outer side of the foot so that the heel point is present on the contour of the captured single foot.

[0027] In the single-foot image 70, a region corresponding to a foot part from which a contour shape 74 is extracted is shown as a cutout region 72. The cutout region 72 includes at least a part of the heel, but may not include the entire heel. The cutout region 72 may also include a part of the single foot other than the heel. However, the cutout region 72 needs to include the heel point in order for the heel estimation unit 56 to estimate the position of the heel point. As will be described later, the heel estimation unit 56 estimates the position of the heel point according to the type of contour shape 74 determined by the determination unit 54, so that the position at which the heel point is estimated is unknown at the time when the contour extraction unit 52 extracts the contour shape 74. Therefore, the contour extraction unit 52 needs to set the cutout region 72 to a region that includes all the positions of the heel point that can be estimated regardless of the type of the contour shape 74 determined. The cutout region 72 can be set based on a predetermined algorithm such as pattern recognition. The shape of the cutout region 72 is not limited to a rectangle as shown in FIG. 4(a) and may be any shape.

[0028] FIG. 4B is a schematic diagram showing an example of a contour shape 74 extracted by the contour extraction unit 52. The contour extraction unit 52 detects the contour of one foot reflected in the one-foot image 70, and extracts the contour shape 74 by identifying a shape along the detected contour. For example, the contour extraction unit 52 can detect the contour between the area occupied by the one foot and the area occupied by the blank paper 80 in the cutout area 72 by binarizing the cutout area 72. The contour extraction unit 52 extracts the contour shape 74 along the contour detected in this manner. The data of the contour shape 74 may be in a raster format that represents the contour shape 74 with a set of points in a two-dimensional coordinate system, or in a vector format that represents the contour shape 74 with a set of lines in a two-dimensional coordinate system. When the data of the contour shape 74 is in a raster format as shown in FIG. 4B, the interval between the points constituting the contour shape 74 is arbitrary, and the number of points may vary depending on the shape of the contour shape 74 included in the cutout area 72. When extracting the contour shape 74, the contour extraction unit 52 may perform any smoothing process to remove noise from the one-leg image 70, for example.

[0029] The contour shape 74 thus extracted is two-dimensional information extracted from the single-foot image 70, as described above. Even if the single-foot image 70 is an image of the same single foot of the same person to be measured, it differs depending on the direction and angle at which it is photographed. In addition, there are individual differences in the three-dimensional shape of one foot of the person to be measured. In this way, the contour shape 74 may differ depending on the direction and angle at which the single-foot image 70 is photographed and the individual differences in the three-dimensional shape of one foot of the person to be measured.

[0030] 3, the determination unit 54 determines the type of the contour shape extracted by the contour extraction unit 52. The determination process by the determination unit 54 will be described in detail later.

[0031] The heel estimation unit 56 estimates the position of the heel point in the contour shape based on the type of contour shape determined by the determination unit 54. The estimation process by the heel estimation unit 56 will be described in detail later.

[0032] In this way, the heel position estimation server 20 of this embodiment uses the determination unit 54 to determine the type of the contour shape 74, and uses the heel estimation unit 56 to estimate the position of the heel point based on the type of the contour shape 74. This allows the heel position estimation server 20 to reduce the influence of the shooting conditions of the one-foot image 70 and individual differences in the three-dimensional shape of one foot, thereby improving the estimation accuracy of the heel point position.

[0033] As shown in FIG. 3, the calculation unit 42 of the heel position estimating server 20 may further include a toe estimating unit 58, a foot length calculating unit 60, and a foot length correcting unit 62.

[0034] The toe estimation unit 58 estimates the toe position from the image of one foot of the subject read by the image reading unit 50. For example, the toe estimation unit 58 may binarize any area on the toe side of the one foot image 70 and detect the contour between the area occupied by the one foot and the area occupied by the blank paper 80. Then, the point located closest to the toe side of the detected contour may be estimated as the toe position. In this case, the one foot image 70 is an image taken from the inner or outer side of one foot so that the toes are present on the contour of the captured one foot.

[0035] The foot length calculation unit 60 calculates the foot length of the subject based on the toe position estimated by the toe estimation unit 58 and the heel position estimated by the heel estimation unit 56. Specifically, when the foot length direction in the one-foot image 70 (left-right direction in FIG. 4(a)) is defined as the X direction, the foot length calculation unit 60 can calculate the absolute value of the difference between Xt and Xh as the foot length using Xt, which is the X coordinate of the estimated toe position, and Xh, which is the X coordinate of the estimated heel position. At this time, the foot length may be calculated based on a comparison with a reference object captured in the one-foot image 70.

[0036] The foot length correction unit 62 corrects the foot length calculated by the foot length calculation unit 60. For example, the foot length correction unit 62 corrects the foot length calculated by the foot length calculation unit by a multiple regression model using parameters based on the size of a reference object and the angle of a predetermined position in the image of one foot, information on the actual size of the reference object and information on the actual angle of the predetermined position, and coefficients based on the type of contour shape. The correction process by the foot length correction unit 62 will be described in detail later.

[0037] FIG. 5 is a flowchart showing an example of the process of the heel position estimation server 20. The image reading unit 50 reads the image of one foot received by the communication unit 40 from the user terminal 10 (S10). The contour extraction unit 52 extracts the contour shape of the foot part including at least a part of the heel from the image of one foot of the subject read by the image reading unit 50 (S12). The determination unit 54 determines the type of the contour shape extracted by the contour extraction unit 52 (S14). The heel estimation unit 56 estimates the position of the heel point in the contour shape based on the type of contour shape determined by the determination unit 54 (S16). After step S10, in parallel with steps S12 to S16, the toe estimation unit 58 estimates the toe position from the image of one foot of the subject read by the image reading unit 50 (S18). The foot length calculation unit 60 calculates the foot length of the subject based on the toe position estimated by the toe estimation unit 58 in step S18 and the heel position estimated by the heel estimation unit 56 in step S16 (S20). The foot length correction unit 62 corrects the foot length calculated by the foot length calculation unit 60 (S22) and ends the process.

[0038] Fig. 6 is a flow chart specifically illustrating the process by the determination unit 54 in step S14 shown in Fig. 5 and the process by the heel estimation unit 56 in step S16, among an example of the operation of the heel position estimation server 20. The determination unit 54 determines whether or not the contour shape is of the first type having a maximum point when the heel side in the foot length direction of the one-foot image is set as the positive direction (S30). When the determination unit 54 determines that it is of the first type (Y in S30), the heel estimation unit 56 estimates the maximum point of the contour shape as the position of the heel point (S32).

[0039] If the determination unit 54 determines that the shape is not the first type (N in S30), the determination unit 54 determines whether the magnitude of the inclination or the degree of change of the contour shape relative to the foot length direction is the second type that satisfies a predetermined condition (S34). If the determination unit 54 determines that the shape is the second type (Y in S34), the position that satisfies the predetermined condition is estimated to be the position of the heel point (S36).

[0040] If the determining unit 54 determines that the contour shape is not the second type (N in S34), the determining unit 54 determines that the contour shape is the third type (S38). If the determining unit 54 determines that the contour shape is the third type (S38), the heel estimating unit 56 estimates the position of the contour shape closest to the heel in the foot length direction as the position of the heel point (S40).

[0041] 7 and 8 are diagrams for explaining examples of each type of contour shape 74 determined by the determination unit 54. In FIG. 7 and FIG. 8, the contour shape 74 is shown as a set of points, but this does not mean that the data of the contour shape 74 is limited to being in raster format. Even if the data of the contour shape 74 is in vector format, it can be seen as a set of points calculated at predetermined intervals in a two-dimensional coordinate system based on the data in vector format. In FIG. 7(a), FIG. 7(b), and FIG. 8(a), the points constituting the contour shape 74 are shown as contour points 82a to 82e. When the contour points 82a to 82e are not distinguished from each other, they are also collectively referred to as contour points 82. Moreover, the contour points 82a to 82e in each drawing are described to distinguish the contour points 82 in each drawing, and the same reference numerals do not indicate the same contour points 82 between different drawings.

[0042] FIG. 7(a) is a diagram showing an example of a contour shape 74 determined to be the first type. Here, FIG. 7(a) shows an enlarged schematic view of a part of the entire contour shape 74 located closest to the heel in the foot length direction of the single foot image 70. The determination unit 54 selects contour points 82 constituting the contour shape 74, two at a time, from the sole side toward the ankle side. At this time, the contour points 82 on the ankle side of the two contour points 82 selected first and the contour point 82 on the sole side of the two contour points 82 selected immediately afterwards are selected to be the same contour point 82. The determination unit 54 sequentially calculates the angle θ between the vector from the selected sole side contour point 82 toward the ankle side contour point 82 and a straight line parallel to the direction from the sole side toward the ankle side (Y-axis direction). For example, the determination unit 54 calculates the angle θ between a vector directed from contour point 82a to contour point 82b and a line parallel to the Y axis, and then calculates the angle θ between a vector directed from contour point 82b to contour point 82c and a line parallel to the Y axis. The sign of the angle θ is positive when the vector is directed toward the toe side from the line parallel to the Y axis, and is negative when the vector is directed toward the heel side from the line parallel to the Y axis.

[0043] The determining unit 54 determines that the contour shape 74 is the first type when the sign of the angle θ becomes positive for the first time, that is, when the vector from the selected contour point 82 on the sole side toward the contour point 82 on the ankle side is more toward the toe side than the straight line parallel to the Y axis. The heel estimating unit 56 then estimates the contour point 82 on the sole side of the two contour points 82 selected at that time as the position of the heel point 76. In the example shown in FIG. 7(a), the vector from the contour point 82c toward the contour point 82d is more toward the toe side than the straight line parallel to the Y axis, so that the contour point 82c is estimated to be the position of the heel point 76. In other words, the determining unit 54 determines that the contour shape 74 is the first type when the heel side in the foot length direction of the one-foot image 70 is the positive direction and the contour shape 74 has a maximum point. The heel estimating unit 56 then estimates the maximum point as the position of the heel point. When there are multiple contour points 82 that satisfy the above condition, the heel estimation section 56 may estimate the contour point 82 that is located closest to the sole of the foot as the position of the heel point 76 .

[0044] When the sign of the angle θ becomes positive for the first time, if the angle θ exceeds a predetermined upper limit, the determination unit 54 may not determine that the angle is the first type, and may proceed to the selection of the next contour point 82. This can prevent the position of the heel point from being erroneously estimated due to, for example, noise in the one-foot image 70. The upper limit of the angle θ is, for example, 45°.

[0045] FIG. 7(b) is a diagram showing an example of a contour shape 74 determined to be the second type. Here, FIG. 7(b) shows an enlarged schematic view of a portion of the entire contour shape 74 that continues from the position closest to the sole side of the one-foot image 70 toward the heel side. After determining that the contour shape 74 is not the first type, the determination unit 54 selects contour points 82 that constitute the contour shape 74, two at a time, from the sole side toward the ankle side, in the same manner as described above. The determination unit 54 sequentially calculates the angle θ between the vector from the selected sole side contour point 82 toward the ankle side contour point 82 and a straight line parallel to the direction from the sole side toward the ankle side (Y-axis direction).

[0046] The determining unit 54 may determine that the contour shape 74 is of the second type when the sign of the angle θ remains negative and the angle θ is equal to or greater than a predetermined angle (the absolute value of the angle θ is equal to or less than the absolute value of the predetermined angle). The heel estimating unit 56 then estimates the contour point 82 on the sole side of the two contour points 82 selected at that time as the position of the heel point 76. In the example shown in FIG. 7(b), the angle θ between the vector from the contour point 82c toward the contour point 82d and a straight line parallel to the Y axis is equal to or greater than a predetermined angle, so that the contour point 82c is estimated to be the position of the heel point 76. The predetermined angle in this case is, for example, -5°. In other words, the determining unit 54 determines that the contour shape 74 is of the second type when the contour shape 74 has a position at an inclination of a predetermined threshold value or greater with respect to the foot length direction (X-axis direction). The predetermined threshold in this case is a value obtained by subtracting the above-mentioned predetermined angle from 90°, for example, 85°. When there are multiple contour points 82 that satisfy the above condition, the heel estimation section 56 may estimate the contour point 82 that is located closest to the sole of the foot as the position of the heel point 76 .

[0047] FIG. 8(a) is a diagram showing another example of a contour shape 74 determined to be the second type. Here, FIG. 8(a) shows an enlarged schematic view of a portion of the entire contour shape 74 that continues from the position closest to the sole side of the one-foot image 70 toward the heel side. After determining that the contour shape 74 is not the first type, the determination unit 54 selects contour points 82 that constitute the contour shape 74, two at a time, from the sole side toward the ankle side, in the same manner as described above. The determination unit 54 sequentially calculates an angle θ2 (not shown) between a vector from the selected contour point 82 on the sole side toward the contour point 82 on the ankle side and a straight line parallel to the foot length direction (X-axis direction).

[0048] The determination unit 54 determines that the contour point 82 is of the second type when the angle θ2 between the two selected contour points 82 is within the first gradient range α and the angle θ2 between the two next selected contour points 82 is within the second gradient range β. The heel estimation unit 56 then estimates the contour point 82 on the sole side of the two subsequently selected contour points 82 (i.e., the contour point 82 on the ankle side of the two previously selected contour points 82) as the position of the heel point 76. In the example shown in FIG. 8(a), the angle θ2 between the vector from the contour point 82b to the contour point 82c and a line parallel to the X-axis is within the first gradient range α, and the angle θ2 between the vector from the contour point 82c to the contour point 82d and a line parallel to the X-axis is within the second gradient range β, so that the contour point 82c is estimated as the position of the heel point 76. Here, the second inclination range β is greater than the first inclination range α by a predetermined amount. For example, the first inclination range α may be greater than 45° and less than 85°, and the second inclination range β may be greater than 85° and less than 90°. If there are multiple contour points 82 that satisfy the above condition, the heel estimation unit 56 may estimate the contour point 82 located closest to the sole of the foot as the position of the heel point 76.

[0049] The determination unit 54 may selectively perform the second type determination described with reference to Fig. 7(b) and the second type determination described with reference to Fig. 8(a) or may perform them step by step. That is, the determination unit 54 may perform only one of the second type determination described with reference to Fig. 7(b) and the second type determination described with reference to Fig. 8(a), or may perform the other determination when the second type is not determined in one determination.

[0050] Fig. 8(b) is a diagram showing an example of a contour shape 74 determined to be the third type. When the determining unit 54 determines that the contour shape 74 is neither the first type nor the second type, it determines that the contour shape 74 is the third type. In this case, the heel estimating unit 56 estimates that the contour point 82 located at the position closest to the heel in the foot length direction (X direction) of the contour shape 74 is the position of the heel point 76, as shown in Fig. 8(b).

[0051] FIG. 9 is a schematic diagram for explaining the correction of the foot length by the foot length correction unit 62. The foot length calculated by the foot length calculation unit 60 is likely to be affected by the shooting angle of the one-foot image 70, etc. In particular, in this embodiment, the position of the heel point estimated depending on the type of the determined contour shape 74 differs in the direction perpendicular to the foot length direction, so it is easily affected by the shooting angle of the one-foot image 70. Therefore, the foot length correction unit 62 corrects the foot length calculated by the foot length calculation unit 60 by a multiple regression model using parameters based on the length of the blank paper 80 as a reference object in the one-foot image 70 along the foot length direction and the angle of the corner of the blank paper 80, information on the actual length of the blank paper 80 along the foot length direction and information on the actual angle of the corner of the blank paper 80, and coefficients based on the type of the contour shape 74.

[0052] The foot length (length_raw) of one foot in the one foot image 70 is calculated based on the coordinates of the toe 78 and the coordinates of the heel point 76. The foot length (length_raw) is standardized based on the length of the blank paper 80 in the one foot image 70 along the foot length direction and the information on the actual length of the blank paper 80 along the foot length direction, and the standardized foot length (Xstd_length_raw) is calculated. The angle (deg_corner) of the corner of the blank paper 80 in the one foot image 70 is calculated by contour extraction or the like. The corner of the blank paper 80 here may be any corner among the four corners of the blank paper 80, but in the example shown in FIG. 9, it is the lower right corner. The angle (deg_corner) of the corner of the blank paper 80 is standardized using the angle of the actual corner of the blank paper 80, for example, 90° which is the angle when the blank paper 80 is viewed in a plan view, and the standardized angle (Xstd_deg_corner) is calculated.

[0053] The multiple regression equation with the standardized foot length (Xstd_length_raw) and the standardized angle (Xstd_deg_corner) as explanatory variables and the corrected foot length (Ylength) as a response variable is expressed by the following equation (1). Ylength=a*Xstd_length_raw + b*Xstd_deg_corner + c ···(1) Here, a, b, and c are coefficients, which can be different for each type of contour shape 74.

[0054] As described above, according to this embodiment, the contour extraction unit 52 extracts the contour shape 74 of the foot part including at least a part of the heel from the one-foot image 70 of the subject. The determination unit 54 determines the type of the contour shape 74. The heel estimation unit 56 estimates the position of the heel point in the contour shape based on the determined type of contour shape. This allows the heel position estimation server 20 to appropriately estimate the position of the heel point while reducing the influence of the shooting conditions of the one-foot image 70 and individual differences in the three-dimensional shape of one foot.

[0055] Furthermore, according to this embodiment, the determination unit 54 determines whether the contour shape 74 is of the first type having a maximum point when the heel side in the foot length direction of the one-foot image 70 is set as the positive direction. If it is determined to be of the first type, the heel estimation unit 56 estimates the maximum point as the position of the heel point 76. This allows the heel position estimation server 20 to appropriately estimate the position of the heel point 76 when the heel shape is a standard heel shape in which the position of the heel point 76 is the maximum point on the heel side.

[0056] Furthermore, according to this embodiment, when the determining unit 54 determines that the contour shape 74 is not the first type, it determines whether the magnitude of the inclination or the degree of change of the contour shape 74 relative to the foot length direction is the second type that satisfies a predetermined condition. When the contour shape 74 is determined to be the second type, the heel estimation unit 56 estimates the position where the contour shape 74 satisfies the predetermined condition as the position of the heel point 76. This allows the heel position estimation server 20 to appropriately estimate the position of the heel point 76 even if the contour shape 74 does not have a maximum point on the heel side.

[0057] Furthermore, according to this embodiment, when tracing the contour shape 74 from the sole side toward the heel side in the foot length direction, the determining unit 54 determines that the contour shape 74 is of the second type if the inclination of the contour shape 74 with respect to the foot length direction changes from the first inclination range α to a second inclination range β that is greater than the first inclination range α by a predetermined amount. When the contour shape 74 is determined to be of the second type, the heel estimation unit 56 estimates that the position where the contour shape 74 changes from the first inclination range α to the second inclination range β is the position of the heel point 76. This allows the heel position estimation server 20 to appropriately estimate the position of the heel point 76 in accordance with the change in inclination of the contour shape 74.

[0058] Furthermore, according to this embodiment, the determining unit 54 determines that the contour shape 74 is of the second type when it has a position where the contour shape 74 has an inclination with respect to the foot length direction that is equal to or greater than a predetermined threshold value. When it is determined that the contour shape 74 is of the second type, the heel estimating unit 56 estimates that the position closest to the sole of the foot where the contour shape 74 has an inclination that is equal to or greater than the threshold value is the position of the heel point 76. This allows the heel position estimating server 20 to appropriately estimate the position of the heel point 76 according to the inclination of the contour shape 74.

[0059] Furthermore, according to this embodiment, if the determining unit 54 determines that the shape is neither the first type nor the second type, the heel estimating unit 56 determines that the shape is the third type. If the shape is determined to be the third type, the heel estimating unit 56 estimates the position of the contour shape 74 that is closest to the heel in the foot length direction as the position of the heel point 76. This allows the heel position estimating server 20 to appropriately estimate the position of the heel point 76 within the range where the contour shape 74 is extracted, even if the contour shape 74 cannot be classified into the first type or the second type.

[0060] The above-described embodiment may be a program for making a computer realize the functions for implementing the above-described method, or a recording medium for storing the program. Such a recording medium for storing the program may be a non-transitory and tangible computer-readable storage medium, and may be a non-volatile memory, a magnetic recording medium such as a magnetic tape or a magnetic disk, or an optical recording medium such as an optical disk.

[0061] The present invention has been described above based on the embodiment. This embodiment is merely an example, and it will be understood by those skilled in the art that various modifications are possible in the combination of each component and each processing step, and that such modifications are also within the scope of the present invention. Furthermore, the above-described embodiment can be generalized to obtain the following aspects.

[0062] [Aspect 1] A contour extraction unit that extracts a contour shape of a foot part including at least a part of a heel from an image of one foot of a subject; A determination unit for determining a type of the contour shape; a heel estimation unit that estimates a position of a heel point in the contour shape based on the determined type of the contour shape; A heel position estimation device comprising:

[0063] [Aspect 2] the determining unit determines whether the contour shape is of a first type having a maximum point when a heel side in a foot length direction of the one-foot image is defined as a positive direction; When the type is determined to be the first type, the heel estimation unit estimates the maximum point as the position of the heel point. A heel position estimation device as described in embodiment 1.

[0064] [Aspect 3] when it is determined that the contour shape is not of the first type, the determination unit determines whether or not a magnitude of inclination or a degree of change of the contour shape with respect to the foot length direction is of a second type that satisfies a predetermined condition; When the contour shape is determined to be the second type, the heel estimation unit estimates a position where the contour shape satisfies the predetermined condition as a position of the heel point. A heel position estimation device according to embodiment 2.

[0065] Aspect 4 the determining unit determines that the contour shape is the second type when an inclination of the contour shape with respect to the foot length direction changes from a first inclination range to a second inclination range that is greater than the first inclination range by a predetermined amount or more when tracing the contour shape from a sole side to the heel side in the foot length direction; When the contour shape is determined to be the second type, the heel estimation unit estimates a position where the contour shape changes from the first gradient range to the second gradient range as a position of the heel point. A heel position estimation device according to embodiment 3.

[0066] Aspect 5 the determining unit determines that the contour shape is of the second type when the contour shape has a position where the inclination with respect to the foot length direction is equal to or greater than a predetermined threshold value; When the contour shape is determined to be the second type, the heel estimation unit estimates a position on the plantar side where the contour shape has a gradient equal to or greater than the threshold value as the position of the heel point. A heel position estimation device according to embodiment 3.

[0067] Aspect 6 When the determination unit determines that the type is neither the first type nor the second type, the determination unit determines that the type is a third type, When the foot is determined to be the third type, the heel estimation unit estimates a position of the contour shape that is closest to the heel in the foot length direction as the position of the heel point. A heel position estimation device according to any one of aspects 3 to 5.

[0068] Aspect 7 a toe estimation unit that estimates a toe position from the one-foot image; a foot length calculation unit that calculates a foot length based on the estimated toe position and the estimated heel position; 7. The heel position estimation device according to any one of aspects 1 to 6, further comprising:

[0069] Aspect 8 a foot length correction unit that corrects the foot length calculated by the foot length calculation unit using a multiple regression model using parameters based on a size of a reference object in the one-foot image and an angle formed by a predetermined part of the reference object, information on an actual size of the reference object and information on an actual angle formed by the predetermined part of the reference object, and a coefficient based on a type of the contour shape. A heel position estimation device as described in embodiment 7.

[0070] Aspect 9 Extracting a contour shape of a foot part including at least a part of a heel from an image of one foot of a subject; determining a type of the contour shape; estimating a position of a heel point on the contour shape based on the determined type of the contour shape; A heel position estimation method comprising:

[0071] Aspect 10 A function of extracting a contour shape of a foot part including at least a part of a heel part from an image of one foot of a measurement subject; determining a type of the contour shape; A function of estimating a position of a heel point in the contour shape based on the determined type of the contour shape; A program to make the above happen on a computer. [Explanation of symbols]

[0072] 10 user terminal, 12 one foot, 20 heel position estimation server, 52 contour extraction unit, 54 determination unit, 56 heel estimation unit, 58 toe estimation unit, 60 foot length calculation unit, 62 foot length correction unit, 70 one foot image, 74 contour shape, 76 heel point, 78 toe, 100 heel position estimation system.

Claims

1. A contour extraction unit that extracts a contour shape of a foot part including at least a part of a heel from an image of one foot of a subject; A determination unit for determining a type of the contour shape; a heel estimation unit that estimates a position of a heel point in the contour shape based on the determined type of the contour shape; A heel position estimation device comprising:

2. the determining unit determines whether the contour shape is of a first type having a maximum point when a heel side in a foot length direction of the one-foot image is defined as a positive direction; When the type is determined to be the first type, the heel estimation unit estimates the maximum point as the position of the heel point.

2. The heel position estimation device according to claim 1,

3. when it is determined that the contour shape is not of the first type, the determination unit determines whether or not a magnitude of inclination or a degree of change of the contour shape with respect to the foot length direction is of a second type that satisfies a predetermined condition; When the contour shape is determined to be the second type, the heel estimation unit estimates a position where the contour shape satisfies the predetermined condition as a position of the heel point.

3. The heel position estimating device according to claim 2 .

4. the determining unit determines that the contour shape is the second type when, when tracing the contour shape from a sole side toward the heel side in the foot length direction, an inclination of the contour shape with respect to the foot length direction changes from a first inclination range to a second inclination range that is greater than the first inclination range by a predetermined amount or more; When the contour shape is determined to be the second type, the heel estimation unit estimates a position where the contour shape changes from the first gradient range to the second gradient range as a position of the heel point.

4. The heel position estimating device according to claim 3,

5. the determining unit determines that the contour shape is of the second type when the contour shape has a position where the inclination with respect to the foot length direction is equal to or greater than a predetermined threshold value; When the contour shape is determined to be the second type, the heel estimation unit estimates a position on the plantar side where the contour shape has a gradient equal to or greater than the threshold value as the position of the heel point.

4. The heel position estimating device according to claim 3,

6. When the determination unit determines that the type is neither the first type nor the second type, the determination unit determines that the type is a third type; When the foot is determined to be the third type, the heel estimation unit estimates a position of the contour shape that is closest to the heel in the foot length direction as the position of the heel point. The heel position estimating device according to any one of claims 3 to 5.

7. a toe estimation unit that estimates a toe position from the one-foot image; a foot length calculation unit that calculates a foot length based on the estimated toe position and the estimated heel position; The heel position estimation device according to claim 1 , further comprising:

8. a foot length correction unit that corrects the foot length calculated by the foot length calculation unit using a multiple regression model using parameters based on a size of a reference object in the one-foot image and an angle formed by a predetermined part of the reference object, information on an actual size of the reference object and information on an actual angle formed by the predetermined part of the reference object, and a coefficient based on a type of the contour shape. The heel position estimating device according to claim 7.

9. Extracting a contour shape of a foot part including at least a part of a heel from an image of one foot of a subject; determining a type of the contour shape; estimating a position of a heel point on the contour shape based on the determined type of the contour shape; A heel position estimation method comprising:

10. A function of extracting a contour shape of a foot part including at least a part of a heel part from an image of one foot of a measurement subject; determining a type of the contour shape; A function of estimating a position of a heel point in the contour shape based on the determined type of the contour shape; A program to make the above happen on a computer.

Images