Heel position estimation device, heel position estimation method, and computer program product

Abstract

Provided is a technique capable of appropriately estimating the position of a heel point on the basis of a single-foot image of a measurement target. The invention provides a heel position estimation device, a heel position estimation method and a computer program product. In a heel position estimation device, a contour extraction unit (52) extracts a contour shape of a foot portion including at least a part of a heel portion from a single-foot image of a measurement target. A determination unit (54) determines the type of the contour shape. A heel estimation unit (56) estimates the position of a heel point in the contour shape on the basis of the determined type of contour shape.

Description

Technical Field

[0001] The present invention relates to a heel position estimating device, a heel position estimating method and a computer program product. Background Art

[0002] A technique for measuring the size of a foot such as the foot length of a measurement object based on an image including at least one foot of the measurement object (hereinafter referred to as a "single foot image") is known. For example, Patent Document 1 describes the following technique: a single foot image is obtained from a photographing unit by photographing the foot placement paper and the entire foot in a state where the sole of the foot is placed on the foot placement paper so that the end of the heel is aligned with the lower edge of the foot placement paper, and an actual length ratio is calculated based on the single foot image by dividing the actual length of the long side of the foot placement paper by the number of pixels of the long side of the foot placement paper.

[0003] [Prior art literature]

[0004] [Patent Document]

[0005] [Patent Document 1] Japanese Patent Application Publication No. 2019-45463 Summary of the invention

[0006] [Problems to be solved by the invention]

[0007] However, in the technology described in Patent Document 1, the end of the heel of the subject to be measured must be aligned with the bottom edge of the foot placement paper, so there is room for improvement in measurement accuracy. In order to accurately measure the foot length of the subject to be measured, it is required to estimate the position of the heel point of the subject to be measured in the single-foot image.

[0008] The present invention has been made in view of the above situation, and an object of the present invention is to provide a technique capable of appropriately estimating the position of the heel point from an image of a single foot of a measurement target.

[0009] [Technical means to solve the problem]

[0010] A heel position estimation device according to one embodiment of the present invention includes: a contour extraction unit, which extracts a contour shape of a foot portion including at least a portion of a heel portion from an image of a single foot of a measurement object; a determination unit, which determines a type of the contour shape; and a heel estimation unit, which estimates a position of a heel point in the contour shape based on the determined type of the contour shape.

[0011] Another embodiment of the present invention is a heel position estimation method, which includes: extracting a contour shape of a foot region including at least a portion of a heel from a single-foot image of a measurement object; determining the type of the contour shape; and estimating the position of a heel point in the contour shape based on the determined type of the contour shape.

[0012] Another embodiment of the present invention is a computer program product, including a computer program. When the computer program is executed by a computer, the computer implements the following functions: a function of extracting a contour shape of a foot portion including at least a portion of a heel portion from an image of a single foot of a measurement object; a function of determining the type of the contour shape; and a function of estimating the position of a heel point in the contour shape based on the determined type of the contour shape.

[0013] In addition, any combination of the above structural elements, or replacement of the structural elements or expressions of the present invention between methods, devices, programs, transient or non-transient storage media storing programs, systems, etc. are also valid as embodiments of the present invention.

[0014] [Effects of the Invention]

[0015] According to one embodiment of the present invention, it is possible to provide a technique capable of appropriately estimating the position of the heel point from an image of a single foot of a measurement target. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 : is a schematic diagram showing the outline of the heel position estimation system.

[0017] Figure 2 Yes means Figure 1 A functional block diagram showing a schematic structure of a heel position estimation system.

[0018] Figure 3 Yes means Figure 1 A functional block diagram showing a schematic structure of a computing unit of a heel position estimation server.

[0019] Figure 4 Yes means Figure 3 Schematic diagram of an example of the contour shape of the foot part extracted from the single foot image by the contour extraction unit of FIG.

[0020] Figure 5 Yes means Figure 1 A flowchart of an example of processing by a heel position estimation server.

[0021] Figure 6 It is a specific expression Figure 1 In an example of the operation of the heel position estimation server, Figure 5 Flowchart showing the process performed by the determination unit in step S14 and the process performed by the heel estimation unit in step S16.

[0022] Figure 7 (a) is a diagram showing an example of a contour shape determined to be of the first type. Figure 7 (b) is a diagram showing an example of a contour shape determined to be the second type.

[0023] Figure 8 (a) is a diagram showing another example of a contour shape determined to be of the second type. Figure 8 (b) is a diagram showing an example of a contour shape determined to be the third type.

[0024] Fig. 9 It is to explain the use Figure 3 Schematic diagram of foot length correction performed by the foot length correction department.

[0025] [Explanation of Symbols]

[0026] 10: User terminal

[0027] 12: Single Foot

[0028] 20: Heel position estimation server

[0029] 52: Contour extraction unit

[0030] 54: Judgment Department

[0031] 56: Heel estimation department

[0032] 58: Toe estimation

[0033] 60: Foot length calculation

[0034] 62: Foot length correction

[0035] 70: Single-foot image

[0036] 74: Outline Shape

[0037] 76: Heel point

[0038] 78: Pointe

[0039] 100: Heel position estimation system DETAILED DESCRIPTION

[0040] Hereinafter, the present invention will be described based on preferred embodiments with reference to the accompanying drawings. In the embodiments and modifications, the same or equivalent components are denoted by the same reference numerals, and repeated descriptions are appropriately omitted.

[0041] Figure 11 is a schematic diagram showing an overview of a heel position estimation system 100. The heel position estimation system 100 includes a user terminal 10 and a heel position estimation server 20 which are connected to each other via wireless communication 16 and a network 18. The subject of measurement uses the camera function of the user terminal 10 to capture an image (single-foot image) containing the subject's single foot 12. The user terminal 10 sends the captured single-foot image to the heel position estimation server 20 via the wireless communication 16 and the network 18. The heel position estimation server 20 estimates the position of the heel point in the received single-foot image. In addition, the heel position estimation server 20 can further estimate the toe position based on the received single-foot image, and can also calculate the foot length based on the estimated toe position and the estimated heel point position. The heel position estimation server 20 can send the foot size information such as the calculated foot length to the user terminal 10, and display it on the screen of the user terminal 10.

[0042] The user terminal 10 and the heel position estimation server 20 may include a portable terminal or a computer, and a program stored in the portable terminal or the computer, respectively. The portable terminal or the computer includes a central processing unit (CPU), a graphics processing unit (GPU), a random access memory (RAM), a read only memory (ROM), an auxiliary storage device, a display device, a communication device, etc. For example, it can be configured in the form of a program executed on the heel position estimation server 20 side from the user terminal 10 side via the network 18. However, it can also be configured in the following manner, that is, a single device having the functions of the user terminal 10 and the heel position estimation server 20 is used to realize the function of the heel position estimation system, and the measurement subject can execute the heel position estimation program by directly operating the device. As a single device, it is not limited to a personal computer, and it can also be provided in the form of a portable terminal or a tablet terminal such as a smartphone and a program stored in these computers or terminals. Or it can be implemented in the form of a terminal installed in a shoe sales store and operated by an assistant such as a store clerk and a program stored in the terminal.

[0043] In this specification, the "heel position estimation device" may refer to the entire heel position estimation system 100 or the heel position estimation server 20. In this embodiment, the heel position estimation server 20 has most of the characteristic functions included in the "heel position estimation device" and implements these characteristic functions, so the heel position estimation server 20 substantially corresponds to the "heel position estimation device". However, the characteristic functions of the "heel position estimation device" may be distributed between the user terminal 10 and the heel position estimation server 20, and the user terminal 10 may have most of these characteristic functions.

[0044] Figure 2 1 is a functional block diagram showing a schematic structure of the heel position estimation system 100. Figure 2 Each functional block shown in each figure can be implemented in hardware by an element or mechanical device represented by a computer processor or memory, and in software by a computer program, etc., but here, the functional blocks implemented by their cooperation are depicted. These functional blocks can be implemented in various forms by hardware, software, or a combination thereof.

[0045] The user terminal 10 includes an image acquisition unit 30, a display unit 32, a communication unit 34, and an operation processing unit 36. The operation processing unit 36 ​​receives an operation input from the measurement subject. The operation processing unit 36 ​​receives an instruction from the measurement subject to start a series of foot size measurement programs including a heel position estimation program. The display unit 32 starts to display the content of the foot size measurement program on the screen based on the instruction received by the operation processing unit 36, and displays the content related to the measurement sequence. In terms of hardware, the operation processing unit 36 ​​and the display unit 32 may include, for example, a touch panel.

[0046] The image acquisition unit 30 captures a single foot of the subject to be measured to acquire a single foot image. The single foot image is an image of a region including at least the heel point of the single foot 12. When the heel position estimation server 20 includes a toe estimation unit 58 described later, the single foot image is an image of a region including at least the toe point of the single foot 12. The single foot image can be an image that the subject can capture by operating the user terminal 10. In this case, for example, Figure 1 As shown in FIG. 1 , the image of the subject's own single foot 12 taken from obliquely above the inner foot side can be used as the single foot image. The single foot image can also be an image taken from obliquely above the outer foot side of the single foot 12.

[0047] Regarding the single-leg image, a predetermined reference object and the single leg 12 may also be photographed together as described later. Here, the "reference object" is an object that is taken into the image together with the single leg as a comparison object when measuring the foot length or foot width of the single leg based on the single-leg image, and is a plane or three-dimensional object whose size or shape is predetermined. The "reference object" may be a general-purpose object as long as it is an object whose size or shape has been predetermined, or it may be a special object, and the special object has a size and shape that is predetermined to be used exclusively for the measurement in this embodiment. As the reference object in this embodiment, a plane object with a predetermined size and shape may be used, for example, A4 size (210mm×297mm) paper may be used. The color of the paper is preferably white, but it may also be paper of other colors. The single-leg image of this embodiment is an image obtained by photographing the single leg placed on the A4 white paper as a reference object together with the white paper as a whole. The A4 white paper is placed on a ground other than white when photographing, so that it can be extracted from the single-leg image with good accuracy through image recognition. The single-leg image may be an image including, for example, a part of the other leg of the measurement object in addition to the single leg 12 of the measurement object and the reference object.

[0048] In terms of hardware, the image acquisition unit 30 may include, for example, a camera module. The communication unit 34 sends the single foot image to the heel position estimation server 20 via the network 18, receives the foot size information from the heel position estimation server 20, and sends it to the display unit 32. In terms of hardware, the communication unit 34 may include, for example, a wireless communication module such as a wireless local area network (LAN) communication or a mobile phone communication. The display unit 32 displays the single foot image and the foot size information of the measurement object on the screen.

[0049] The heel position estimation server 20 includes a communication unit 40, a calculation unit 42, a storage unit 44, and an output unit 46. The communication unit 40 receives a single foot image from the user terminal 10, and transmits calculation results such as foot size information obtained by the calculation unit 42 to the user terminal 10. In terms of hardware, the communication unit 40 includes a communication module such as a wired LAN.

[0050] The calculation unit 42 performs calculation processing including a process of estimating the position of the heel point based on the single-foot image received by the communication unit 40. Details of the calculation processing performed by the calculation unit 42 will be described later.

[0051] The calculation unit 42 stores the calculation result in the storage unit 44. The calculation result stored in the storage unit 44 is transmitted to the user terminal 10 via the communication unit 40 through the output unit 46.

[0052] Figure 31 is a functional block diagram showing a schematic configuration of the calculation unit 42 of the heel position estimation server 20. The calculation unit 42 includes an image reading unit 50, a contour extraction unit 52, a determination unit 54, and a heel estimation unit 56.

[0053] The image reading unit 50 reads the single foot image received by the communication unit 40 from the user terminal 10. The contour extraction unit 52 extracts the contour shape of the foot region including at least a part of the heel from the single foot image of the measurement object read by the image reading unit 50. Figure 4 1 is a schematic diagram showing an example of the contour shape 74 of the foot region extracted by the contour extraction unit 52 from the single-foot image 70 .

[0054] Figure 4 (a) is a schematic diagram showing the position of the foot part from which the contour shape 74 is extracted in the single-foot image 70. Figure 4 In the example shown in (a) of FIG. 1 , the single foot image 70 is a two-dimensional image captured from the oblique upper side of the inner foot side in a state where the single foot is placed on a white paper 80 as a reference object so as to capture the entire single foot. As described later, the heel estimation unit 56 estimates a point along the contour shape 74 extracted from the single foot image 70 by the contour extraction unit 52 as the position of the heel point. Therefore, the single foot image 70 is an image captured from the inner foot side or the outer foot side of the single foot so that the heel point exists on the contour of the captured single foot.

[0055] In the single-foot image 70, the area corresponding to the foot part from which the contour shape 74 is extracted is represented as a cut-out area 72. The cut-out area 72 includes at least a portion of the heel portion, but may not include the entire heel portion. In addition, the cut-out area 72 may also include portions other than the heel portion of the single foot. However, the cut-out area 72 needs to include the heel point so that the position of the heel point can be estimated using the heel estimation unit 56. As described later, the heel estimation unit 56 estimates the position of the heel point based on the type of contour shape 74 determined by the determination unit 54. Therefore, at the moment when the contour extraction unit 52 extracts the contour shape 74, it can be estimated that the position of the heel point is unclear. Therefore, the contour extraction unit 52 needs to set the following area for the cut-out area 72: the area includes all the positions of the heel point that can be estimated when the contour shape 74 is determined to be any type. The cut-out area 72 can be set based on a predetermined algorithm such as pattern recognition. The shape of the cut-out area 72 is not limited to. Figure 4 The rectangle shown in (a) may be any shape.

[0056] Figure 4(b) is a schematic diagram showing an example of a contour shape 74 extracted by the contour extraction unit 52. The contour extraction unit 52 detects the contour of the single foot captured in the single foot image 70, and determines the shape along the detected contour, thereby extracting the contour shape 74. For example, the contour extraction unit 52 performs binarization processing on the cutout area 72, thereby detecting the contours of the area occupied by the single foot and the area occupied by the white paper 80 in the cutout area 72. The contour extraction unit 52 extracts the contour shape 74 along the contour detected in this way. The data of the contour shape 74 may be in the form of a raster in which the contour shape 74 is represented by a set of points in a two-dimensional coordinate system, or in the form of a vector in which the contour shape 74 is represented by a set of lines in a two-dimensional coordinate system. When the data of the contour shape 74 is as follows: Figure 4 In the case of the grid form shown in (b), the intervals between the points constituting the contour shape 74 are arbitrary, and the number of points may be different depending on the shape of the contour shape 74 included in the clipping area 72. The contour extraction unit 52 may also perform arbitrary smoothing processing when extracting the contour shape 74 to remove noise from the single-leg image 70, etc.

[0057] As described above, the contour shape 74 extracted in this way is two-dimensional information extracted from the single-leg image 70. Regarding the single-leg image 70, even if it is obtained by photographing the same single leg of the same measurement subject, it may be different depending on the direction and angle of photography. In addition, there are individual differences in the three-dimensional shape of the single leg of the measurement subject. In this way, the contour shape 74 may be different depending on the shooting direction or angle of the single-leg image 70 and the individual differences in the three-dimensional shape of the single leg of the measurement subject.

[0058] Back to Figure 3 The determination unit 54 determines the type of the contour shape extracted by the contour extraction unit 52. Details of the determination process performed by the determination unit 54 will be described later.

[0059] The heel estimating unit 56 estimates the position of the heel point in the contour shape based on the type of the contour shape determined by the determining unit 54. Details of the estimation process performed by the heel estimating unit 56 will be described later.

[0060] In this way, the heel position estimation server 20 of the present embodiment determines the type of the contour shape 74 using the determination unit 54, and estimates the position of the heel point based on the type of the contour shape 74 using the heel estimation unit 56. Thus, the heel position estimation server 20 can reduce the influence of the shooting conditions of the single foot image 70 or the individual differences in the three-dimensional shape of the single foot, and improve the estimation accuracy of the heel point position.

[0061] like Figure 3As shown, the calculation unit 42 of the heel position estimation server 20 may further include a toe estimation unit 58 , a foot length calculation unit 60 , and a foot length correction unit 62 .

[0062] The toe estimating unit 58 estimates the toe position based on the single foot image of the measurement object read by the image reading unit 50. For example, the toe estimating unit 58 may perform binarization processing on an arbitrary area on the toe side in the single foot image 70, and detect the contours of the area occupied by the single foot and the area occupied by the white paper 80. In addition, the point located closest to the toe side in the detected contour may be estimated as the toe position. In this case, the single foot image 70 is an image captured from the inner foot side or the outer foot side of the single foot in such a way that the toe exists on the contour of the captured single foot.

[0063] The foot length calculation unit 60 calculates the foot length of the object to be measured based on the toe position estimated by the toe estimation unit 58 and the heel point position estimated by the heel estimation unit 56. Specifically, the foot length direction ( Figure 4 In the case where the left-right direction in (a) is set as the X direction, the foot length calculation unit 60 can use the estimated X coordinate of the toe position, that is, Xt, and the estimated X coordinate of the heel point position, that is, Xh, to calculate the absolute value of the difference between Xt and Xh as the foot length. In this case, the foot length can also be calculated based on the comparison with the reference object captured in the single foot image 70.

[0064] The foot length correction unit 62 corrects the foot length calculated by the foot length calculation unit 60. For example, the foot length correction unit 62 corrects the foot length calculated by the foot length calculation unit using a multiple regression model that uses parameters based on the size of the reference object and the angle of a predetermined position in the single foot image, information on the actual size of the reference object and the actual angle of the predetermined position, and a coefficient based on the type of contour shape. Details of the trimming process performed by the foot length correction unit 62 will be described later.

[0065] Figure 51 is a flowchart showing an example of processing of the heel position estimation server 20. The image reading unit 50 reads the single foot image received by the communication unit 40 from the user terminal 10 (S10). The contour extraction unit 52 extracts the contour shape of the foot part including at least a part of the heel part from the single foot image of the measurement object read by the image reading unit 50 (S12). The determination unit 54 determines the type of the contour shape extracted by the contour extraction unit 52 (S14). The heel estimation unit 56 estimates the position of the heel point in the contour shape based on the type of the contour shape determined by the determination unit 54 (S16). After step S10, in parallel with steps S12 to S16, the toe estimation unit 58 estimates the toe position based on the single foot image of the measurement object read by the image reading unit 50 (S18). The foot length calculation unit 60 calculates the foot length of the measurement object based on the toe position estimated by the toe estimation unit 58 in step S18 and the heel point position estimated by the heel estimation unit 56 in step S16 (S20). The foot length correction unit 62 corrects the foot length calculated by the foot length calculation unit 60 (S22) and then ends the process.

[0066] Figure 6 This is a specific example of the operation of the heel position estimation server 20. Figure 5 Flowchart of the process performed by the determination unit 54 in step S14 and the process performed by the heel estimation unit 56 in step S16 is shown. The determination unit 54 determines whether it is the first type in which the contour shape has a maximum point when the heel side in the foot length direction of the single foot image is set as the positive direction (S30). When the determination unit 54 determines that it is the first type (Y in S30), the heel estimation unit 56 estimates the maximum point of the contour shape as the position of the heel point (S32).

[0067] If the determination unit 54 determines that it is not the first type (N in S30), the determination unit 54 determines whether it is the second type as follows: the size of the inclination width of the contour shape with respect to the foot length direction or the degree of change satisfies a predetermined condition (S34). If the determination unit 54 determines that it is the second type (Y in S34), the position that satisfies the predetermined condition is estimated as the position of the heel point (S36).

[0068] If the determination unit 54 determines that the contour shape is not the second type (N in S34), the determination unit 54 determines that the contour shape is the third type (S38). If the determination unit 54 determines that the contour shape is the third type (S38), the heel estimation unit 56 estimates the position closest to the heel in the foot length direction in the contour shape as the position of the heel point (S40).

[0069] Figure 7 (a) Figure 7 (b) and Figure 8 (a) Figure 8(b) is a diagram for explaining examples of each type of the contour shape 74 determined by the determination unit 54. Figure 7 (a) Figure 7 (b) and Figure 8 (a) Figure 8 In (b), the contour shape 74 is shown as a collection of points, but this does not limit the data of the contour shape 74 to a grid format. Even when the data of the contour shape 74 is in a vector format, it can be regarded as a collection of points calculated at predetermined intervals in a two-dimensional coordinate system based on the data in the vector format. Figure 7 (a) Figure 7 (b) and Figure 8 In (a), the points constituting the outline shape 74 are represented by outline points 82a to 82e. When the outline points 82a to 82e are not distinguished, they are collectively referred to as outline points 82. In addition, the outline points 82a to 82e in each drawing are symbols recorded to distinguish the outline points 82 in each drawing, and even if they are the same symbols, they do not represent the same outline point 82 in different drawings.

[0070] Figure 7 (a) is a diagram showing an example of a contour shape 74 determined to be of the first type. Figure 7 In (a), a portion of the entire contour shape 74 that is located closest to the heel side in the foot length direction of the single foot image 70 is enlarged and schematically shown. The determination unit 54 selects two contour points 82 constituting the contour shape 74 from the sole side to the ankle side. At this time, the selection is performed in such a manner that the contour point 82 on the ankle side of the two contour points 82 selected first and the contour point 82 on the sole side of the two contour points 82 selected immediately thereafter become the same contour point 82. The determination unit 54 sequentially calculates the angle θ formed by the vector from the selected sole side contour point 82 toward the ankle side contour point 82 and the straight line parallel to the direction from the sole side toward the ankle side (Y-axis direction). For example, the determination unit 54 calculates the angle θ formed by the vector from the contour point 82a toward the contour point 82b and the straight line parallel to the Y-axis, and then calculates the angle θ formed by the vector from the contour point 82b toward the contour point 82c and the straight line parallel to the Y-axis. The sign of the angle θ is positive when the vector is directed toward the toe side relative to the straight line parallel to the Y axis, and is negative when the vector is directed toward the heel side relative to the straight line parallel to the Y axis.

[0071] When the sign of the angle θ is positive for the first time, that is, when the vector from the selected sole side contour point 82 toward the ankle side contour point 82 is more toward the toe side than the straight line parallel to the Y axis, the determination unit 54 determines that the contour shape 74 is the first type. Then, the heel estimation unit 56 estimates the sole side contour point 82 of the two contour points 82 selected at this time as the position of the heel point 76. Figure 7 In the example shown in (a), since the vector from the contour point 82c to the contour point 82d is more toward the toe side than the straight line parallel to the Y axis, the contour point 82c is estimated as the position of the heel point 76. In other words, the determination unit 54 determines that the contour shape 74 is of the first type when the heel side in the foot length direction of the single foot image 70 is set as the positive direction. Then, the heel estimation unit 56 estimates the maximum point as the position of the heel point. In addition, when there are multiple contour points 82 that meet the above conditions, the heel estimation unit 56 can estimate the contour point 82 that is located closest to the sole side as the position of the heel point 76.

[0072] When the sign of the angle θ is positive for the first time, if the angle θ exceeds a predetermined upper limit, the determination unit 54 does not determine the first type and may shift to the next selection of the contour point 82. This can suppress the position of the heel point from being erroneously estimated due to noise in the single-foot image 70, for example. The upper limit of the angle θ is, for example, 45°.

[0073] Figure 7 (b) is a diagram showing an example of a contour shape 74 determined to be the second type. Figure 7 In (b), a portion of the entire contour shape 74 that is connected from the position closest to the sole side of the single-foot image 70 toward the heel side is enlarged and schematically shown. After determining that the contour shape 74 is not of the first type, the determination unit 54 selects two contour points 82 constituting the contour shape 74 from the sole side to the ankle side in the same manner as described above. The determination unit 54 sequentially calculates the angle θ formed by the vector from the selected sole side contour point 82 toward the ankle side contour point 82 and the straight line parallel to the direction from the sole side to the ankle side (Y-axis direction).

[0074] The determination unit 54 may determine that the contour shape 74 is of the second type when the sign of the angle θ remains negative and the angle θ is greater than or equal to the predetermined angle (the absolute value of the angle θ is less than or equal to the absolute value of the predetermined angle). Then, the heel estimation unit 56 estimates the contour point 82 on the sole side of the two contour points 82 selected at this time as the position of the heel point 76. Figure 7In the example shown in (b), since the angle θ formed by the vector from the contour point 82c toward the contour point 82d and the straight line parallel to the Y-axis is greater than a predetermined angle, the contour point 82c is estimated to be the position of the heel point 76. The predetermined angle in this case is, for example, -5°. In other words, the determination unit 54 determines that the contour shape 74 is of the second type when it has a position where the inclination amplitude relative to the foot length direction (X-axis direction) is greater than a predetermined threshold. The predetermined threshold in this case is a value obtained by subtracting the predetermined angle from 90°, for example, 85°. In addition, when there are multiple contour points 82 that meet the above conditions, the heel estimation unit 56 can estimate the contour point 82 that is located closest to the sole of the foot as the position of the heel point 76.

[0075] Figure 8 (a) is a diagram showing another example of the contour shape 74 determined to be the second type. Figure 8 In (a), a portion of the entire contour shape 74 that is connected from the position closest to the sole side of the single-foot image 70 toward the heel side is enlarged and schematically shown. After determining that the contour shape 74 is not of the first type, the determination unit 54 selects two contour points 82 constituting the contour shape 74 from the sole side to the ankle side in the same manner as described above. The determination unit 54 sequentially calculates the angle θ2 (not shown) formed by the vector from the selected sole side contour point 82 toward the ankle side contour point 82 and the straight line parallel to the foot length direction (X-axis direction).

[0076] The determination unit 54 determines that the position is the second type when the angle θ2 about the two selected contour points 82 is within the first inclination range α and the angle θ2 about the next two selected contour points 82 is within the second inclination range β. Then, the heel estimation unit 56 estimates the contour point 82 on the sole side of the two contour points 82 selected later (that is, the contour point 82 on the ankle side of the two contour points 82 selected earlier) as the position of the heel point 76. Figure 8 In the example shown in (a), since the angle θ2 formed by the vector from the contour point 82b toward the contour point 82c and the straight line parallel to the X-axis is within the range of the first inclination range α, and the angle θ2 formed by the vector from the contour point 82c toward the contour point 82d and the straight line parallel to the X-axis is within the range of the second inclination range β, the contour point 82c is estimated to be the position of the heel point 76. Here, the inclination amplitude of the second inclination range β is greater than the first inclination range α by a predetermined degree or more. For example, the first inclination range α may be greater than 45° and less than 85°, and the second inclination range β may be greater than 85° and less than 90°. In addition, when there are multiple contour points 82 that meet the above conditions, the heel estimating unit 56 may estimate the contour point 82 located closest to the sole side as the position of the heel point 76.

[0077] The determination unit 54 may selectively refer to Figure 7 (b) The second type of determination and reference Figure 8 One of the second type of determination described in (a) may also be referred to in stages. Figure 7 (b) The second type of determination and reference Figure 8 That is, the determination unit 54 may only refer to Figure 7 (b) The second type of determination and reference Figure 8 In any one of the second type determinations described in (a), when one of the determinations is not determined as the second type, the other determination may be performed.

[0078] Figure 8 (b) is a diagram showing an example of the contour shape 74 determined to be the third type. When the determination unit 54 determines that the contour shape 74 is not the first type or the second type, it determines that the contour shape 74 is the third type. In this case, Figure 8 As shown in FIG. 2( b ), the heel estimating unit 56 estimates the contour point 82 located closest to the heel side in the foot length direction (X direction) of the contour shape 74 as the position of the heel point 76 .

[0079] Fig. 9 1 is a schematic diagram for explaining the foot length correction performed by the foot length correction unit 62. The foot length calculated by the foot length calculation unit 60 is prone to errors due to the shooting angle of the single foot image 70, etc. In particular, in the present embodiment, the position of the heel point estimated according to the type of the determined contour shape 74 is different in the direction orthogonal to the foot length direction, and is therefore easily affected by the shooting angle of the single foot image 70. Therefore, the foot length correction unit 62 corrects the foot length calculated by the foot length calculation unit 60 using a multiple regression model, the multiple regression model using parameters based on information on the length of a white paper 80 as a reference object in the single foot image 70 along the foot length direction and the angle formed by the corners of the white paper 80, and information on the actual length of the white paper 80 along the foot length direction and the actual angle formed by the corners of the white paper 80, and coefficients based on the type of the contour shape 74.

[0080] The foot length (length_raw) of the single foot in the single foot image 70 can be calculated based on the coordinates of the toe point 78 and the coordinates of the heel point 76. The foot length (length_raw) is standardized based on the length of the white paper 80 in the single foot image 70 along the foot length direction and the actual length of the white paper 80 along the foot length direction, thereby calculating the standardized foot length (Xstd_length_raw). In addition, the angle (deg_corner) formed by the corners of the white paper 80 in the single foot image 70 can be calculated by contour extraction or the like. The corner of the white paper 80 here can be any corner of the four corners of the white paper 80, but in Fig. 9 The angle formed by the corners of the white paper 80 (deg_corner) is normalized using the actual angle formed by the corners of the white paper 80, for example, 90 degrees as the angle when looking down at the white paper 80, to calculate the normalized angle (Xstd_deg_corner).

[0081] The multiple regression equation using the standardized foot length (Xstd_length_raw) and the standardized angle (Xstd_deg_corner) as explanatory variables and the corrected foot length (Ylength) as the target variable is represented by the following equation (1).

[0082] Ylength=a*Xstd_length_raw+b*Xstd_deg_corner+c…(1)

[0083] Here, a, b, and c are coefficients, respectively, and different coefficients may be set for each type of the outline shape 74 .

[0084] As described above, according to the present embodiment, the contour extraction unit 52 extracts the contour shape 74 of the foot region including at least a part of the heel portion from the single foot image 70 of the measurement object. The determination unit 54 determines the type of the contour shape 74. The heel estimation unit 56 estimates the position of the heel point in the contour shape based on the determined type of the contour shape. Thus, the heel position estimation server 20 can reduce the influence of the shooting conditions of the single foot image 70 or the individual differences in the three-dimensional shape of the single foot, and appropriately estimate the position of the heel point.

[0085] In addition, according to the present embodiment, the determination unit 54 determines whether the first type is as follows: when the heel side in the foot length direction of the single foot image 70 is set as the positive direction, the contour shape 74 has a maximum point. When it is determined to be the first type, the heel estimation unit 56 estimates the maximum point as the position of the heel point 76. Thus, in the case of a standard heel shape such that the position of the heel point 76 is the maximum point on the heel side, the heel position estimation server 20 can appropriately estimate the position of the heel point 76.

[0086] In addition, according to the present embodiment, when the determination unit 54 determines that the first type is not present, the determination unit 54 determines whether the second type is present: the size of the inclination width of the contour shape 74 with respect to the foot length direction or the degree of change satisfies a predetermined condition. When the second type is determined, the heel estimation unit 56 estimates the position of the contour shape 74 that satisfies the predetermined condition as the position of the heel point 76. Thus, even when the contour shape 74 does not have a maximum point on the heel side, the heel position estimation server 20 can appropriately estimate the position of the heel point 76.

[0087] In addition, according to the present embodiment, when the inclination amplitude of the contour shape 74 with respect to the foot length direction changes from the first inclination range α to the second inclination range β having an inclination amplitude greater than the first inclination range α by a predetermined degree or more when the contour shape 74 is drawn from the sole side toward the heel side in the foot length direction, the determination unit 54 determines that it is the second type. When it is determined that it is the second type, the heel estimation unit 56 estimates the position where the contour shape 74 changes from the first inclination range α to the second inclination range β as the position of the heel point 76. Thus, the heel position estimation server 20 can appropriately estimate the position of the heel point 76 based on the change in the inclination amplitude of the contour shape 74.

[0088] In addition, according to the present embodiment, the determination unit 54 determines that the contour shape 74 is of the second type when the contour shape 74 has a position where the inclination amplitude with respect to the foot length direction is equal to or larger than a predetermined threshold value. When the contour shape 74 is determined to be of the second type, the heel estimation unit 56 estimates the position closest to the sole of the foot where the contour shape 74 has an inclination amplitude equal to or larger than the threshold value as the position of the heel point 76. Thus, the heel position estimation server 20 can appropriately estimate the position of the heel point 76 according to the inclination amplitude of the contour shape 74.

[0089] According to the present embodiment, the determination unit 54 determines that the contour shape 74 is not the first type or the second type, but determines that the contour shape 74 is the third type. When the contour shape 74 is determined to be the third type, the heel estimation unit 56 estimates the position closest to the heel in the foot length direction as the position of the heel point 76. Thus, even when the contour shape 74 cannot be classified into the first type or the second type, the heel position estimation server 20 can appropriately estimate the position of the heel point 76 within the range where the contour shape 74 is extracted.

[0090] The embodiment may be a program for causing a computer to implement the functions for implementing the method, or a recording medium storing the program. The recording medium storing such a program may be a non-transitory and tangible computer-readable recording medium, or a non-volatile memory, a magnetic recording medium such as a tape or a disk, or an optical recording medium such as an optical disk.

[0091] The present invention has been described above based on the embodiments. The embodiments are illustrative, and those skilled in the art will appreciate that the combination of these various structural elements or various processing techniques can achieve various variations, and such variations are also within the scope of the present invention. In addition, if the embodiments are summarized, the following examples can be obtained.

[0092] [Example 1]

[0093] A heel position estimation device, comprising:

[0094] a contour extraction unit that extracts a contour shape of a foot portion including at least a part of a heel portion from an image of a single foot of a measurement object;

[0095] a determination unit that determines the type of the contour shape; and

[0096] The heel estimating unit estimates a position of a heel point in the contour shape based on the determined type of the contour shape.

[0097] [Example 2]

[0098] The heel position estimating device according to embodiment 1, wherein:

[0099] The determination unit determines whether the contour shape is of the first type having a local maximum point when the heel side in the foot length direction of the single-foot image is set as a positive direction,

[0100] When it is determined to be the first type, the heel estimating unit estimates the maximum point as the position of the heel point.

[0101] [Example 3]

[0102] The heel position estimating device according to embodiment 2, wherein:

[0103] When the determination unit determines that the contour shape is not the first type, the determination unit determines whether the contour shape is the second type in which the size of the inclination width or the degree of change in the foot length direction satisfies a predetermined condition.

[0104] When the second type is determined, the heel estimating unit estimates a position where the contour shape satisfies the predetermined condition as the position of the heel point.

[0105] [Example 4]

[0106] According to the heel position estimating device described in Example 3, wherein:

[0107] The determination unit determines that the contour shape is of the second type when the inclination amplitude of the contour shape relative to the foot length direction changes from a first inclination range to a second inclination range with an inclination amplitude greater than a predetermined degree than the first inclination range when the contour shape is drawn from the sole side toward the heel side in the foot length direction.

[0108] When the second type is determined, the heel estimating unit estimates a position where the contour shape changes from the first inclination range to the second inclination range as the position of the heel point.

[0109] [Example 5]

[0110] According to the heel position estimating device described in Example 3, wherein:

[0111] The determination unit determines that the contour shape is the second type when the contour shape has a position where the inclination width with respect to the foot length direction is equal to or larger than a predetermined threshold value,

[0112] When the second type is determined, the heel estimating unit estimates the position on the most plantar side where the contour shape has an inclination amplitude equal to or larger than the threshold value as the position of the heel point.

[0113] [Example 6]

[0114] The heel position estimating device according to any one of Embodiments 3 to 5, wherein:

[0115] The determination unit determines that the object is a third type when determining that the object is neither the first type nor the second type.

[0116] When the third type is determined, the heel estimating unit estimates a position of the contour shape that is closest to the heel in the foot length direction as the position of the heel point.

[0117] [Example 7]

[0118] The heel position estimating device according to any one of embodiments 1 to 6 is characterized by further comprising:

[0119] a toe estimating unit that estimates a toe position based on the single-foot image; and

[0120] The foot length calculation unit calculates the foot length based on the estimated toe position and the estimated heel point position.

[0121] [Example 8]

[0122] The heel position estimating device according to embodiment 7 is characterized in that:

[0123] It also includes a foot length correction unit, which uses a multivariate regression model to correct the foot length calculated by the foot length calculation unit. The multivariate regression model uses parameters based on the size of the reference object in the single-foot image and the angle formed by a predetermined part of the reference object, and information on the actual size of the reference object and the actual angle formed by the predetermined part of the reference object, and uses coefficients based on the type of the contour shape.

[0124] [Example 9]

[0125] A heel position estimation method, characterized by comprising:

[0126] A step of extracting a contour shape of a foot region including at least a part of a heel portion from an image of a single foot of a measurement subject;

[0127] a step of determining the type of the contour shape; and

[0128] The step of estimating a position of a heel point in the contour shape based on the determined type of the contour shape.

[0129] [Example 10]

[0130] A computer program product, comprising a computer program, wherein when the computer program is executed by a computer, the computer is enabled to implement the following functions:

[0131] A function of extracting a contour shape of a foot region including at least a part of a heel portion from an image of a single foot of a measurement subject;

[0132] A function for determining the type of the contour shape; and

[0133] A function of estimating a position of a heel point in the contour shape based on the determined type of the contour shape.

Claims

1. A heel position estimation device, characterized in that: include: a contour extraction unit that extracts a contour shape of a foot portion including at least a part of a heel portion from an image of a single foot of a measurement object; A determination unit, which determines the type of the contour shape; as well as The heel estimating unit estimates a position of a heel point in the contour shape based on the determined type of the contour shape.

2. The heel position estimating device according to claim 1, characterized in that: The determination unit determines whether the contour shape is of the first type having a local maximum point when the heel side in the foot length direction of the single-foot image is set as a positive direction, When it is determined to be the first type, the heel estimating unit estimates the maximum point as the position of the heel point.

3. The heel position estimating device according to claim 2, characterized in that: When the determination unit determines that the contour shape is not the first type, the determination unit determines whether the contour shape is the second type in which the size of the inclination width or the degree of change in the foot length direction satisfies a predetermined condition. When the second type is determined, the heel estimating unit estimates a position where the contour shape satisfies the predetermined condition as the position of the heel point.

4. The heel position estimating device according to claim 3, characterized in that: The determination unit determines that the contour shape is of the second type when the inclination amplitude of the contour shape relative to the foot length direction changes from a first inclination range to a second inclination range with an inclination amplitude greater than a predetermined degree than the first inclination range when the contour shape is drawn from the sole side toward the heel side in the foot length direction. When the second type is determined, the heel estimating unit estimates a position where the contour shape changes from the first inclination range to the second inclination range as the position of the heel point.

5. The heel position estimating device according to claim 3, characterized in that: The determination unit determines that the contour shape is the second type when the contour shape has a position where the inclination width with respect to the foot length direction is equal to or larger than a predetermined threshold value. When the second type is determined, the heel estimating unit estimates the position on the most plantar side where the contour shape has an inclination amplitude equal to or larger than the threshold value as the position of the heel point.

6. The heel position estimating device according to any one of claims 3 to 5, characterized in that: The determination unit determines that the object is a third type when determining that the object is neither the first type nor the second type. When the third type is determined, the heel estimating unit estimates a position of the contour shape that is closest to the heel in the foot length direction as the position of the heel point.

7. The heel position estimating device according to claim 1, characterized in that: Also includes: a toe estimating unit, for estimating a toe position based on the single foot image; as well as The foot length calculation unit calculates the foot length based on the estimated toe position and the estimated heel point position.

8. The heel position estimating device according to claim 7, characterized in that: Also includes: The foot length correction unit corrects the foot length calculated by the foot length calculation unit using a multivariate regression model, wherein the multivariate regression model uses parameters based on the angle formed by the size of the reference object in the single-foot image and the predetermined part of the reference object, and information on the actual size of the reference object and the actual angle formed by the predetermined part of the reference object, and uses coefficients based on the type of the contour shape.

9. A heel position estimation method, characterized in that: include: A step of extracting a contour shape of a foot region including at least a part of a heel portion from an image of a single foot of a measurement subject; a step of determining the type of the contour shape; as well as The step of estimating a position of a heel point in the contour shape based on the determined type of the contour shape.

10. A computer program product, characterized in that The invention comprises a computer program, wherein when the computer program is executed by a computer, the computer is enabled to implement the following functions: A function of extracting a contour shape of a foot region including at least a part of a heel portion from an image of a single foot of a measurement subject; A function for determining the type of the contour shape; as well as A function of estimating a position of a heel point in the contour shape based on the determined type of the contour shape.

Images