Method and device for providing foot information

Abstract

The present disclosure relates to a method and device for providing foot information A method for providing foot information, according to one embodiment of the present disclosure, comprises: acquiring images composed of a plurality of frames each including a user's foot; and generating, on the basis of the images, a model corresponding to the user's foot, wherein the images may include images of the user's foot viewed at a plurality of angles, respectively.

Description

Method and device for providing foot information

[0001] The present disclosure relates to a method and apparatus for providing information on a foot.

[0002] Most shoe manufacturers produce shoes based solely on foot measurements in 5mm increments, without classifying foot shapes. Furthermore, due to the lack of standardized sizing specifications across companies, shoes of the same size from different brands may not fit properly. Wearing ill-fitting shoes can lead to foot deformities, such as hallux valgus, caused by continuous pressure; these foot deformities can ultimately harm overall physical health.

[0003] Recently, in order to prevent such risks, technology has been developed and is being used to 3D scan the shape of a user's foot to produce shoes that fit the user's foot perfectly, taking into account various requirements such as the size of the user's foot, arch, width of the ball of the foot, toe length, and height of the instep.

[0004] The purpose of the present disclosure is to provide a method and apparatus for providing information on foot. The problems to be solved by the present disclosure are not limited to those mentioned above, and other problems and advantages of the present disclosure not mentioned can be understood from the following description and will be more clearly understood by the embodiments of the present disclosure. Furthermore, it will be understood that the problems and advantages to be solved by the present disclosure can be realized by the means and combinations thereof set forth in the claims.

[0005] A first aspect of the present disclosure may provide a method for providing information about a foot, comprising: acquiring an image composed of a plurality of frames, each including a user's foot; and generating a model corresponding to the user's foot based on the image, wherein the image includes an image of the user's foot viewed from each of a plurality of angles.

[0006] A second aspect of the present disclosure may provide a device comprising: a memory in which at least one program is stored; and a processor that operates by executing the at least one program, wherein the processor acquires an image composed of a plurality of frames, each including a user's foot, and generates a model corresponding to the user's foot based on the image, wherein the image includes images of the user's foot viewed from each of a plurality of angles, and provides information of the foot.

[0007] A third aspect of the present disclosure may provide a computer-readable recording medium having a program for executing a method according to a first aspect on a computer.

[0008] According to various embodiments of the present disclosure, foot size and / or shape can be measured through an image taken by a user.

[0009] Accordingly, users can not only easily obtain information about their feet but also receive information about shoes that fit their feet.

[0010] In addition, the corresponding model of the foot can be appropriately simplified, making it effective in user environments where specifications are insufficient, such as mobile terminals.

[0011] The scope of the present disclosure is not limited by the aforementioned effects.

[0012] FIG. 1 is a flowchart of a method for providing information on a foot according to one embodiment of the present disclosure.

[0013] FIG. 2 is a drawing showing a user image capturing interface according to one embodiment of the present disclosure.

[0014] FIG. 3 is a drawing showing a user image capturing interface according to one embodiment of the present disclosure.

[0015] FIGS. 4a to 4c are drawings illustrating an interface for a user to take an image according to another embodiment of the present disclosure.

[0016] FIG. 5 is a conceptual diagram for explaining a first process for processing an image according to one embodiment of the present disclosure.

[0017] FIG. 6 is a conceptual diagram for explaining a second process for processing an image according to one embodiment of the present disclosure.

[0018] FIG. 7 is a conceptual diagram for explaining a third process for processing an image according to one embodiment of the present disclosure.

[0019] FIG. 8 is a drawing showing an exemplary model for explaining a simplification process according to one embodiment of the present disclosure.

[0020] FIG. 9 is a drawing showing information on a measured foot according to one embodiment of the present disclosure.

[0021] FIG. 10 is a drawing showing an interface for recommending shoes according to one embodiment of the present disclosure.

[0022] FIG. 11 is a block diagram of an apparatus according to one embodiment of the present disclosure.

[0023] A method for providing information about a foot according to one embodiment of the present disclosure acquires an image composed of a plurality of frames, each including a user's foot, and generates a model corresponding to the user's foot based on the image, wherein the image may include images of the user's foot viewed from each of a plurality of angles.

[0024] The advantages and features of the present invention, and the methods for achieving them, will become clear by referring to the embodiments described in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments presented below, but can be implemented in various different forms and should be understood to include all modifications, equivalents, and substitutions that fall within the spirit and scope of the present invention. The embodiments presented below are provided to ensure that the disclosure of the present invention is complete and to fully inform those skilled in the art of the scope of the invention. In describing the present invention, detailed descriptions of related known technologies are omitted if it is determined that such detailed descriptions may obscure the essence of the present invention.

[0025] The terms used in this application are used merely to describe specific embodiments and are not intended to limit the invention. The singular expression includes the plural expression unless the context clearly indicates otherwise. In this application, terms such as "comprising" or "having" are intended to indicate the presence of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0026] Some embodiments of the present disclosure may be represented by functional block configurations and various processing steps. Some or all of these functional blocks may be implemented by various numbers of hardware and / or software configurations that perform specific functions. For example, the functional blocks of the present disclosure may be implemented by one or more microprocessors or by circuit configurations for a specific function. Additionally, for example, the functional blocks of the present disclosure may be implemented in various programming or scripting languages. The functional blocks may be implemented as algorithms executed on one or more processors. Furthermore, the present disclosure may employ prior art for electronic configuration, signal processing, and / or data processing, etc. Terms such as "mechanism," "element," "means," and "configuration" may be used broadly and are not limited to mechanical and physical configurations.

[0027] Furthermore, the connecting lines or connecting members between the components depicted in the drawings are merely illustrative of functional connections and / or physical or circuit connections. In the actual device, connections between components may be represented by various alternative or added functional connections, physical connections, or circuit connections.

[0028] Additionally, terms including ordinal numbers, such as "first" or "second," used in the specification may be used to describe various components, but said components shall not be limited by said terms. Such terms may be used for the purpose of distinguishing one component from another.

[0029] In the present disclosure, "foot information" may mean any information that can be measured or expressed by the external shape of the foot. For example, foot information may include, but is not limited to, the length of the foot, the height of the arch of the foot, the width of the foot, the circumference of the foot, the distance between a specific point on the foot and another point, the surface area of ​​the foot, the angles of three specific points on the foot, etc.

[0030] A method for providing information on a foot according to various embodiments of the present disclosure may be performed by a device for providing information on a foot according to the present disclosure. Through the device for providing information on a foot, a user may input an image, and the device for providing information on a foot may calculate information on a foot based on the input image and provide the calculated information on a foot to the user.

[0031] In the present disclosure, the method of providing foot information performed by the device providing foot information may be understood as being performed by a processor included in the device providing foot information.

[0032] In the present disclosure, a user may input an image of their foot through a device that provides foot information and receive foot information. In order to enable a user to use or access the method of providing foot information of the present disclosure or a service based thereon, a solution that provides foot information, such as an application, may be provided to the user, and the solution that provides foot information may be installed on a device that provides foot information. As another example, the method of providing foot information of the present disclosure may be implemented through a web service.

[0033] FIG. 1 is a flowchart of a method for providing information on a foot according to one embodiment of the present disclosure.

[0034] Referring to FIG. 1, in step 110, the device providing information about the foot can acquire an image composed of a plurality of frames, each containing the user's foot.

[0035] In one embodiment, an image composed of a plurality of frames may include images of the user's foot viewed from each of a plurality of angles.

[0036] In one embodiment, the device providing information about the foot may display an image input interface configured to allow a user to input an image. The image input interface will be described later with reference to FIGS. 2 to 4c.

[0037] In one embodiment, the device providing foot information can calculate foot information based on a model corresponding to the user's foot. The process of calculating foot information will be described later with reference to FIG. 9.

[0038] In one embodiment, an image composed of a plurality of frames may include a reference object.

[0039] The image composed of a plurality of frames of the present disclosure will be described in detail below.

[0040] Referring to FIG. 1, in step 120, the device providing information about the foot can generate a model corresponding to the user's foot based on an image.

[0041] In one embodiment, step 120 may include the step of selecting a plurality of preferred images among an image composed of a plurality of frames. The preferred images will be described in detail later.

[0042] In one embodiment, step 120 may include the step of generating a plurality of foot images composed only of feet from an image composed of a plurality of frames, each including a user's foot. In one embodiment, the step of generating a plurality of foot images may include the step of generating a mask for the foot from a first frame among the image composed of a plurality of frames, and the step of removing the remaining area excluding the foot from the first frame based on the first frame and the mask for the foot. The process of generating a plurality of foot images composed only of feet according to the present disclosure will be described in detail later with reference to FIGS. 5 and 6.

[0043] In one embodiment, step 120 may further include the step of generating a point cloud based on images of a plurality of feet. In one embodiment, step 120 may further include the step of generating a mesh by simplifying the point cloud. The process of generating a point cloud and a mesh according to the present disclosure will be described in detail later with reference to FIGS. 6 and 7.

[0044] In addition to the embodiments described above, various embodiments may be included within the scope of the present disclosure, and below, various embodiments of a method for providing information on a foot will be described in detail with reference to FIGS. 2 to 10.

[0045] FIG. 2 is a drawing showing an interface that allows a user to input an image according to one embodiment of the present disclosure.

[0046] As described above, the device providing information about the foot can acquire an image composed of multiple frames, and each of the multiple frames may include the user's foot. In one embodiment, the image composed of multiple frames may include images of the user's foot viewed from each of multiple angles.

[0047] In one embodiment, a user can input an image into a device that provides foot information. To enable a user to input an image into a device that provides foot information, the device that provides foot information can generate and display an interface through which the user can input an image.

[0048] Referring to FIG. 2, an image input interface (200) is shown, which is an interface configured to allow a user to input an image into a device that provides information about the foot.

[0049] In one embodiment, the user may input an image by directly capturing the image, and in another embodiment, the user may input an image by uploading a captured image. The device providing foot information can acquire the image input by the user.

[0050] Referring to FIG. 2, in one embodiment, the image input interface (200) may include an image capture object (210) and an image upload object (220). The image capture object (210) may be an object that a user can interact with to take an image. The image upload object (220) may mean an object that a user can interact with to upload a captured image. A user can interact with the image capture object (210) or the image upload object (220) through input to the image capture object (210) or the image upload object (220), for example, the input to the image capture object (210) or the image upload object (220) may be a touch input to a device that provides information about the foot.

[0051] In one embodiment, the device providing information about the foot based on the user interacting with an image capturing object (210) may create and provide an image capturing interface. The image capturing interface may refer to an interface for the user to capture an image. The image capturing interface will be described in detail later.

[0052] In one embodiment, the device providing foot information based on a user interacting with an image upload object (220) may create and provide an image upload interface. The image upload interface may mean an interface for a user to upload an image. For example, the image upload interface may be implemented as an interface for a user to attach a file.

[0053] When a user inputs an image through an image capture interface or an image upload interface, the device providing foot information can acquire the image.

[0054] Meanwhile, in one embodiment, the input image may be a plurality of single-frame images. That is, the input image may be a plurality of photographic images. In another embodiment, the input image may be an image composed of a plurality of frames. That is, the input image may be a single video image. The "image composed of a plurality of frames" mentioned below may be understood to mean either a plurality of single-frame images or an image composed of a plurality of frames. The device providing foot information of the present disclosure may provide foot information based on an image composed of a plurality of frames.

[0055] In one embodiment, a device providing foot information may select a plurality of preferred images from an image composed of a plurality of input frames. In the present disclosure, a preferred image may refer to an image suitable for the device providing foot information to generate a model corresponding to the user's foot and to acquire and provide foot information. As one example, the clarity of an image input by a user may vary depending on the limitations of the device that captured the image, the image capture environment, the shooting method, etc. As another example, some frames among the images input by a user may not include the foot or may not clearly include the foot. Accordingly, the device providing foot information may select a plurality of preferred images by generating a model corresponding to the user's foot and excluding images unsuitable for acquiring and providing foot information.

[0056] In one embodiment, the number of multiple preferred images may be pre-set. That is, the device providing foot information can select a pre-set number of preferred images from among images composed of multiple input frames. By selecting a pre-set number of preferred images, the processing speed or efficiency of the device providing foot information may be increased, or the performance error of the device providing foot information may be minimized.

[0057] As described above, a device that provides information about the foot based on a user interacting with an image capturing object (210) can create and provide an image capturing interface. Hereinafter, an image capturing interface will be described with reference to FIGS. 3 and FIGS. 4a to 4c. A device that provides information about the foot can obtain an image from a user by creating and providing an image capturing interface (300) shown in FIG. 3 or an image capturing interface (400) shown in FIGS. 4a to 4c.

[0058] Meanwhile, in one embodiment, an image composed of multiple frames may include a reference object. That is, each of the images composed of multiple frames may include a user's foot and a reference object. The reference object may refer to an object whose size is standardized and whose length is known. For example, the reference object may be a flat object commonly used in daily life, such as paper, an envelope, a notebook, or a sketchbook. Typically, the reference object may be an A4 sheet of paper measuring 210 mm * 297 mm. When an image composed of multiple frames includes a reference object, information about the user's foot can be provided more accurately.

[0059] FIG. 3 is a drawing showing an image capturing interface according to one embodiment of the present disclosure.

[0060] Referring to FIG. 3, an image capturing interface (300) according to one embodiment is illustrated. The image capturing interface (300) may include a screen captured by an image capturing device, such as a camera, which is mounted on or connected to a device that provides foot information. Referring to FIG. 3, the image capturing interface (300) is illustrated as including a user's foot (F) within the screen being captured.

[0061] The image capturing interface (300) illustrated in FIG. 3 may be an interface for capturing video. That is, a video image composed of multiple frames may be input through the image capturing interface (300).

[0062] The user can capture video images through a device that provides foot information, and the device that provides foot information can assist the user in taking proper shots to acquire suitable images.

[0063] Referring to FIG. 3, the image shooting interface (300) may include a first shooting guide (310). The first shooting guide (310) may consist of a guide phrase, and the guide phrase may include text summarizing the shooting method. As illustrated in FIG. 3, the first shooting guide (310) may include the text "Shoot while rotating the camera." The first shooting guide (310) may be superimposed on the screen being shot and included in the image shooting interface (300) and displayed.

[0064] Referring to FIG. 3, the image shooting interface (300) may include a second shooting guide (320). The second shooting guide (320) may be composed of an arrow object, and the arrow object may include an image indicating the shooting direction. As illustrated in FIG. 3, the second shooting guide (320) may include an arrow bent downward from left to right, centered on the user's foot. The second shooting guide (320) may be superimposed on the screen being shot and included in the image shooting interface (300) for display.

[0065] The text of the first shooting guide (310) or the shape of the arrow of the second shooting guide (320) shown in FIG. 3 does not limit the present disclosure.

[0066] FIGS. 4a to 4c are drawings illustrating an image capturing interface according to another embodiment of the present disclosure.

[0067] The image capturing interface (400) illustrated in FIGS. 4a to 4c may be an interface for capturing a plurality of single-frame images. That is, a plurality of single-frame images, i.e., a plurality of photographic images, may be input through the image capturing interface (400).

[0068] Referring to FIGS. 4a to 4c, an image capturing interface (400) according to one embodiment is illustrated. The image capturing interface (400) may include a guide to help the user take a proper shot so that a device providing information about the foot can acquire a suitable image.

[0069] To explain the embodiments of the present disclosure, the left foot being photographed will be described as an example, and it will be readily understood by a person skilled in the art that the embodiments of the present disclosure may be similarly applied to the right foot.

[0070] In one embodiment, a guide to assist the user in taking a correct shot may be composed of lines schematically illustrating the shape of the foot.

[0071] Referring to FIG. 4a, the image capturing interface (400) may include a third shooting guide (410). Specifically, the third shooting guide (410) may be composed of lines illustrating the shape of the foot as seen from above.

[0072] Referring to FIG. 4b, the image capturing interface (400) may include a fourth shooting guide (420). Specifically, the fourth shooting guide (420) may be composed of lines that illustrate the shape of the foot as seen from the left side.

[0073] Referring to FIG. 4c, the image capturing interface (400) may include a fifth shooting guide (430). Specifically, the fifth shooting guide (430) may be composed of lines illustrating the shape of the foot as seen from the right side of the left foot.

[0074] Photographic images of the left foot viewed from three angles can be obtained through the third shooting guide (410) to the fifth shooting guide (430) illustrated in FIGS. 4a to 4c. In addition to the third shooting guide (410) to the fifth shooting guide (430), a guide can be generated based on the external shape of the foot viewed from various angles, and through this, photographic images of the foot viewed from various angles can be obtained. That is, the direction of the foot or the angle at which the foot is viewed does not limit the present disclosure.

[0075] Although not illustrated in FIGS. 4a to 4c, the third shooting guide (410), the fourth shooting guide (420), and the fifth shooting guide (430) may be displayed superimposed on a screen captured by an image capturing device, such as a camera, which is mounted on or connected to the device providing information about the foot.

[0076] Meanwhile, even if the user does not directly take a photo image through the shooting button, the device providing foot information can take a shot and receive an image when the user's foot is appropriately included in the photo image. That is, in one embodiment, the device providing foot information can automatically take a shot and receive an image when the user's foot is included in the guide within the image shooting interface (400) while the screen being shot and the guide within the image shooting interface (400) (e.g., a third shooting guide (410), a fourth shooting guide (420), or a fifth shooting guide (430)) are superimposed and displayed.

[0077] FIG. 5 is a conceptual diagram for explaining a first process for processing an image according to one embodiment of the present disclosure.

[0078] In the present disclosure, the first process may mean a process in which a device providing information about a foot analyzes an object included in each of a plurality of images and each of a plurality of unknowns, and consequently obtains information regarding a part of the foot within each of the plurality of images. Specifically, the first process may include extracting information regarding one or more objects included in an image and generating a mask regarding one or more objects.

[0079] As described above, the device providing foot information can generate a model corresponding to the user's foot based on an image. In one embodiment, the device providing foot information generating a model corresponding to the user's foot may include performing segmentation on the image.

[0080] The device providing information about the foot may include a segmentation unit (500). The segmentation unit (500) can be understood as a functional block that performs segmentation in the device providing information about the foot. Referring to FIG. 5, the segmentation unit (500) may receive a single image (510).

[0081] In the present disclosure, a single image (510) may mean an image of a single frame. In one embodiment, the single image (510) may be one frame of an image composed of a plurality of frames. In one embodiment, the single image (510) may be one image of a plurality of preferred images.

[0082] In one embodiment, the segmentation unit (500) can generate and output the location (521) of an object, the class (522) of an object, and a mask (523) based on an input single image (510).

[0083] In the present disclosure, the location of an object (521) may mean the location where an object exists within a single image (510) and may be understood as a bounding box. That is, the segmentation unit (500) can detect one or more objects within a single image (510) and determine the location within the single image (510) of each of the one or more objects included in the input single image (510).

[0084] In the present disclosure, the class (522) of an object may refer to the class to which the object belongs and may also be understood as a label. That is, the segmentation unit (500) can determine what kind of object each of one or more objects included in the input single image (510) is. In one example, the segmentation unit (500) can determine whether any object corresponds to a foot or not, i.e., perform binary classification. In another example, the segmentation unit (500) can determine that any object corresponds to one of a plurality of pre-set classes, i.e., perform multi-class classification.

[0085] In the present disclosure, the mask (523) may represent a pixel-level classification result indicating the region to which an object belongs within a single image (510). That is, the segmentation unit (500) may assign a label of an object to each pixel constituting the single image (510), thereby allowing the shape, size, and / or location of the object to be expressed in detail. In one embodiment, the segmentation unit (500) may detect an object as a bounding box and then generate a mask (523) within the region of the bounding box. In one embodiment, the type of mask (523) output by the segmentation unit (500) may be based on a binary class mask, a multi-class mask, or an instance mask.

[0086] In one embodiment, generating the mask (523) may include extracting features of an input single image (510). In one embodiment, generating the mask (523) may include proposing one or more regions of interest (ROIs) based on the extracted features. In one embodiment, generating the mask (523) may include aligning the proposed one or more regions of interest to a fixed size and maintaining pixel precision. In one embodiment, generating the mask (523) may include generating a pixel-level mask for each of the one or more regions of interest. Generating the mask (523) may include mapping the generated mask to the input single image (510).

[0087] In one embodiment, the segmentation unit (500) may include an artificial intelligence learning and inference model for performing the operation of the present disclosure. The segmentation unit (500) may be trained to detect an object (e.g., a user's foot) in an input single image (510) and to infer the location (521) of the object, the class (522) of the object, and the mask (523) of the object. In one embodiment, the segmentation unit (500) may include a Mask R-CNN.

[0088] As described above, in one embodiment, an image composed of a plurality of frames may include a reference object. In one embodiment, the segmentation unit (500) may also generate a mask (523) for the reference object. Information regarding the specifications of the reference object and the mask (523) of the reference object may be stored and subsequently used to calculate information about the foot.

[0089] The device providing the information on the initiative of the present disclosure can generate the location of an object, the class of an object, and a mask for each of a plurality of single images by performing the first process of FIG. 5 for each of a plurality of single images.

[0090] FIG. 6 is a conceptual diagram for explaining a second process for processing an image according to one embodiment of the present disclosure.

[0091] In the present disclosure, the second process may mean generating a point cloud for an object. Consequently, through the second process, a point cloud for a foot may be generated.

[0092] As described above, the device providing foot information can generate a model corresponding to the user's foot based on images. In one embodiment, the device providing foot information generating a model corresponding to the user's foot may include generating a point cloud based on a plurality of foot images.

[0093] The device providing foot information may include a point cloud generation unit (600). The point cloud generation unit (600) can be understood as a functional block that performs the generation of a point cloud in the device providing foot information. Referring to FIG. 6, the point cloud generation unit (600) may receive a plurality of foot images (610).

[0094] In the present disclosure, the plurality of foot images (610) may include a plurality of frames. In one embodiment, the plurality of foot images (610) may be at least some of a plurality of single-frame images. In one embodiment, the plurality of foot images (610) may be at least some of an image composed of a plurality of frames. In one embodiment, the plurality of foot images (610) may be a plurality of preferred images.

[0095] In a preferred embodiment, each of the plurality of foot images (610) may be an image in which the area excluding the foot is removed from each of the images composed of a plurality of frames. That is, the device providing foot information may remove the area excluding the foot from each of the acquired images composed of a plurality of frames and acquire the plurality of foot images (610).

[0096] Specifically, the device providing foot information can obtain information regarding a part of the foot within each of a plurality of images through the first process described above with reference to FIG. 5, and such information regarding a part of the foot can correspond to a mask (523) for the foot. In one embodiment, the device providing foot information can generate an image composed only of a foot based on a single image (510) and a mask (523) for the foot. For example, the device providing foot information can perform filtering on each of a plurality of pixels included in a single image (510) using a mask (523) for the foot. The device providing foot information can perform filtering on each of a plurality of single images (510), and each of the plurality of foot images (610) of FIG. 6 may be an image composed only of a foot generated through this process.

[0097] In one embodiment, the point cloud generation unit (600) can generate and output a point cloud (620) based on a plurality of input foot images (610).

[0098] In one embodiment, the point cloud generation unit (600) may determine information regarding the model of the image capturing device. That is, generating the point cloud (620) may include determining information regarding the model of the image capturing device. Information regarding the model of the image capturing device may include intrinsic parameters, extrinsic parameters, and a projection matrix. Information regarding the model of the image capturing device may be stored in advance in a device that provides information about the foot.

[0099] In one embodiment, the point cloud generation unit (600) can detect feature points in each of the images (610) of a plurality of feet. That is, generating the point cloud (620) may include detecting feature points. The point cloud generation unit (600) can detect feature points based on algorithms such as SIFT, SURF, and ORB.

[0100] In one embodiment, the point cloud generation unit (600) can match feature points between a plurality of foot images (610). That is, generating a point cloud (620) may include matching feature points. The point cloud generation unit (600) may match feature points based on algorithms such as brute-force matching, FLANN-based matching, and distance-based matching using k-NN. In one embodiment, matching feature points may include the point cloud generation unit (600) removing outliers. The point cloud generation unit (600) may remove outliers based on the RANSAC algorithm.

[0101] In one embodiment, the point cloud generation unit (600) may perform epipolar geometry analysis. That is, generating the point cloud (620) may include performing epipolar geometry analysis.

[0102] In one embodiment, the point cloud generation unit (600) may perform the restoration of the pose of the image capturing device. That is, generating the point cloud (620) may include performing the restoration of the pose of the image capturing device. The restoration of the pose of the image capturing device may be for the purpose of establishing the camera coordinate system of the image.

[0103] In one embodiment, the point cloud generation unit (600) may perform triangulation. That is, generating the point cloud (620) may include performing triangulation. Triangulation may mean estimating the location of feature points in three-dimensional space when the same feature point is observed at different points in time. The point cloud (620) may be generated by combining the feature points in three-dimensional space restored through triangulation.

[0104] In one embodiment, the point cloud generation unit (600) may further perform post-processing on the generated point cloud (620). For example, the post-processing may include scale correction (normalization) and noise removal.

[0105] As described above, in one embodiment, an image composed of a plurality of frames may include a reference object. In one embodiment, a device providing information about the foot may perform scale correction of the point cloud (620) based on the specifications of the reference object.

[0106] FIG. 7 is a conceptual diagram for explaining a third process for processing an image according to one embodiment of the present disclosure.

[0107] In the present disclosure, the third process may mean simplifying the generated point cloud. As a result, a mesh may be generated as the simplified point cloud.

[0108] As described above, the device providing foot information can generate a model corresponding to the user's foot based on an image. In one embodiment, the device providing foot information generating a model corresponding to the user's foot may include simplifying the generated point cloud to generate a mesh.

[0109] A device providing foot information may include a simplification unit (700). The simplification unit (700) can be understood as a functional block that performs simplification of a point cloud in a device providing foot information. Referring to FIG. 7, the simplification unit (700) may receive a point cloud (710).

[0110] The point cloud (710) of Fig. 7 can correspond to the point cloud (620) of Fig. 6.

[0111] In one embodiment, the simplification unit (700) can generate and output a mesh (720) based on an input point cloud (710). The mesh (720) can be generated by simplifying the point cloud (710). That is, generating the mesh (720) may include simplifying the point cloud (710). In one embodiment, the simplification unit (700) can simplify the point cloud (710) using Delauni triangulation.

[0112] In one embodiment, simplifying the point cloud (710) may include performing triangulation on the point cloud (710). Specifically, a plurality of triangles (or tetrahedrons) may be generated based on the points constituting the point cloud (710). Through the plurality of triangles, the spatial relationships between the points can be clearly expressed, and structural connectivity can be secured in the point cloud (710).

[0113] In one embodiment, simplifying the point cloud (710) may include removing some of the points constituting the point cloud (710) on which triangulation has been performed. When a point is removed, all lines connected to that point are removed, and as a result, the number of triangles may be reduced.

[0114] In one embodiment, the simplification unit (700) can remove points based on the quality of each of a plurality of triangles. The quality of the triangles may include the angles, size, and ratios of the triangles.

[0115] In one embodiment, the simplification unit (700) can remove points based on the length of the sides of a triangle. For example, if the length of a side of a triangle is longer than a preset value, that side can be removed.

[0116] In one embodiment, the simplification unit (700) can remove points based on the density of the points. For example, the simplification unit (700) can reduce the density of the points by removing points in an area where the density of the points is greater than a preset value.

[0117] In one embodiment, the simplification unit (700) may repeatedly perform the process of simplifying the aforementioned point cloud (710). The simplification unit (700) may repeatedly perform the process of simplifying the point cloud (710) until the mesh (720) generated by simplifying the point cloud (710) satisfies a preset condition. For example, the preset condition may be a condition regarding at least one of the size of the data in the mesh (720), the number of points included in the mesh (720), the number of triangles included in the mesh (720), etc. The preset condition may be set so that, considering the environment in which a device or method for providing foot information is used, the size of the data in the mesh (720) is not large, but the mesh (720) is generated sufficiently to represent the user's foot and determine information about the user's foot.

[0118] As described above, the device providing foot information can generate a model corresponding to the user's foot based on an image. The mesh (720) generated by the device providing foot information can be understood as a model corresponding to the user's foot.

[0119] FIG. 8 is a drawing showing an exemplary mesh for explaining a simplification process according to one embodiment of the present disclosure.

[0120] Referring to FIG. 8, an exemplary simplified mesh is illustrated. The mesh on the right can be understood as the result of simplifying the mesh on the left. It is shown that the mesh on the right contains fewer points and triangles compared to the mesh on the left. Although the mesh on the right does not represent the surface of the foot as finely as the mesh on the left, the data size of the mesh on the right may be smaller than the data size of the mesh on the left.

[0121] FIG. 9 is a drawing showing information on a calculated foot according to one embodiment of the present disclosure.

[0122] In one embodiment, the device providing foot information can calculate foot information based on a model corresponding to the user's foot. The device providing foot information can calculate the user's foot information using a generated mesh (e.g., mesh (720)).

[0123] Referring to FIG. 9, the device providing foot information can calculate the length of the foot (901). Alternatively, the device providing foot information can calculate the height of the arch (902).

[0124] Additionally, referring to FIG. 9, the device providing information about the foot can calculate the width (903) of the foot. Alternatively, the device providing information about the foot can calculate the circumference (904) of the foot.

[0125] In addition to the example illustrated in FIG. 9, the device providing foot information can produce various types of foot information based on a model corresponding to the user's foot.

[0126] In one embodiment, the device providing foot information can determine the type of foot shape. The device providing foot information can determine the type of user's foot shape using a generated mesh (e.g., mesh (720)).

[0127] For example, a device providing foot information can calculate the length of each toe based on a generated mesh. A device providing foot information can determine the type of foot shape based on the length relationships of each toe. For example, a user's foot may be classified into Type 1 if the index toe is the longest, Type 2 if the big toe is the longest and the lengths gradually decrease toward the little toe, and Type 3 if the lengths of the big toe, index toe, and middle toe are similar (e.g., the difference in length is below a threshold) and the lengths of the ring toe and little toe are short. For example, a user's foot may be classified into Type 4 if the big toe is the longest and the lengths of the remaining toes are similar (e.g., the difference in length is below a threshold), and Type 5 if the lengths of each toe are similar to one another (e.g., the difference in length is below a threshold). The aforementioned types of feet do not limit the present disclosure.

[0128] As described above, in one embodiment, an image composed of a plurality of frames may include a reference object. In one embodiment, a device providing foot information may calculate foot information based on the specifications of the reference object. When an image composed of a plurality of frames includes a reference object, the user's foot information may be calculated more accurately.

[0129] FIG. 10 is a drawing showing an interface for recommending shoes according to one embodiment of the present disclosure.

[0130] In one embodiment, a device providing foot information can generate and provide an interface for shoe recommendations based on the foot information.

[0131] Referring to FIG. 10, a device providing information about a foot may generate and provide a model display interface (1010) configured to display a model corresponding to the generated foot. The model corresponding to the foot displayed in the model interface (1010) may be a mesh generated by the device providing information about the foot.

[0132] Referring to FIG. 10, a device providing foot information may generate and provide an information summary interface (1020) configured to display calculated foot information. The foot information displayed in the information summary interface (1020) is foot information calculated by the device providing foot information, and may include, for example, the length of the foot, the width of the foot, etc.

[0133] In one embodiment, a device providing foot information can determine shoes recommended to a user based on a model corresponding to the generated foot and the calculated foot information.

[0134] In one embodiment, the device providing foot information may determine shoes to recommend to the user by receiving information regarding brands preferred by the user in advance or additionally. In one embodiment, the device providing foot information may receive information regarding various shoes of various brands, as well as the internal and external structures and specifications of the shoes, from a pre-stored database or server. In one embodiment, the device providing foot information may determine shoes having an inner surface suitable for the shape of the user's foot based on a model corresponding to the generated foot. That is, the shoes recommended to the user determined by the device providing foot information may be shoes having an inner surface suitable for the shape of the user's foot.

[0135] Referring to FIG. 10, a device providing foot information may generate and provide a shoe recommendation interface (1030) configured to display shoes recommended to a determined user. The shoes displayed in the shoe recommendation interface (1030) may include shoes recommended to a user determined by the device providing foot information.

[0136] FIG. 11 is a block diagram of an apparatus according to one embodiment of the present disclosure.

[0137] The device (1100) illustrated in FIG. 11 may be a device that provides information on the aforementioned foot.

[0138] Referring to FIG. 11, the device (1100) may include a communication unit (1110), a processor (1120), and a DB (1130). Only the components related to the embodiment are shown in the device (1100) of FIG. 11. Therefore, a person skilled in the art will understand that other general-purpose components may be included in addition to the components shown in FIG. 11.

[0139] The communication unit (1110) may include one or more components that enable wired / wireless communication with an external server or external device. For example, the communication unit (1110) may include at least one of a short-range communication unit (not shown), a mobile communication unit (not shown), and a broadcast receiving unit (not shown).

[0140] The DB (1130) is hardware that stores various data processed within the device (1100) and can store programs for processing and controlling the processor (1120). The DB (1130) can store payment information, user information, etc.

[0141] DB (1130) may include RAM (random access memory), such as DRAM (dynamic random access memory) and SRAM (static random access memory), ROM (read-only memory), EEPROM (electrically erasable programmable read-only memory), CD-ROM, Blu-ray or other optical disc storage, HDD (hard disk drive), SSD (solid state drive), or flash memory.

[0142] The processor (1120) controls the overall operation of the device (1100). For example, the processor (1120) can control the input unit (not shown), display (not shown), communication unit (1110), DB (1130), etc., by executing programs stored in the DB (1130). The processor (1120) can control the operation of the device (1100) by executing programs stored in the DB (1130).

[0143] The processor (1120) can control at least some of the operations of the device (1100) described above in FIGS. 1 to 10.

[0144] The processor (1120) may be implemented using at least one of ASICs (application specific integrated circuits), DSPs (digital signal processors), DSPDs (digital signal processing devices), PLDs (programmable logic devices), FPGAs (field programmable gate arrays), controllers, microcontrollers, microprocessors, and other electrical units for performing functions.

[0145] In one embodiment, the device (1100) may be a mobile electronic device. For example, the device (1100) may be implemented as a smartphone, tablet PC, PC, smart TV, PDA (personal digital assistant), laptop, media player, navigation, a device equipped with a camera, and other mobile electronic devices. Additionally, the device (1100) may be implemented as a wearable device such as a watch, glasses, a hair band, and a ring equipped with communication functions and data processing functions.

[0146] An embodiment according to the present invention may be implemented in the form of a computer program that can be executed through various components on a computer, and such a computer program may be recorded on a computer-readable medium. In this case, the medium may include a magnetic medium such as a hard disk, a floppy disk, and a magnetic tape, an optical recording medium such as a CD-ROM and a DVD, a magneto-optical medium such as a floptical disk, and a hardware device specifically configured to store and execute program instructions, such as a ROM, RAM, or flash memory.

[0147] Meanwhile, the above-mentioned computer program may be one specifically designed and configured for the present invention, or one known and available to those skilled in the art of computer software. Examples of computer programs may include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc.

[0148] According to one embodiment, the method according to various embodiments of the present disclosure may be provided by being included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or distributed online (e.g., download or upload) through an application store (e.g., Play Store™) or directly between two user devices. In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily created in a device-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0149] Unless explicitly stated otherwise regarding the steps constituting the method according to the present invention, said steps may be performed in a suitable order. The present invention is not necessarily limited by the order in which said steps are described. The use of all examples or exemplary terms (e.g., etc.) in the present invention is merely for the purpose of describing the present invention in detail, and the scope of the present invention is not limited by said examples or exemplary terms unless limited by the claims. Furthermore, those skilled in the art will understand that various modifications, combinations, and changes may be made according to design conditions and factors within the scope of the claims or equivalents to which they are added.

[0150] Accordingly, the scope of the present invention should not be limited to the embodiments described above, and all scopes equivalent to or equivalently modified from the claims set forth below, as well as the claims set forth below, shall be considered to fall within the scope of the concept of the present invention.

Claims

1. As a method of providing information on the foot, A step of acquiring an image composed of multiple frames, each including the user's foot; and A step of generating a model corresponding to the user's foot based on the above image; Includes, The image above is, including images of the user's foot viewed from each of multiple angles, method.

2. In Paragraph 1, The step of generating a model corresponding to the user's foot is, A step of selecting a plurality of preferred images among the images composed of the plurality of frames above; including, method.

3. In Paragraph 1, The step of generating a model corresponding to the user's foot is, A step of generating multiple images of feet composed only of feet from an image composed of multiple frames, each of which includes the user's feet; including, method.

4. In Paragraph 3, The step of generating the above-mentioned images of multiple feet is A step of generating a mask for the foot from a first frame among the images composed of the plurality of frames above; and Based on the first frame and the mask for the foot, the step of removing the remaining area excluding the foot from the first frame; including, method.

5. In Paragraph 3, The step of generating a model corresponding to the user's foot is, A step of generating a point cloud based on the above-mentioned images of the plurality of feet; including, method.

6. In Paragraph 5, The step of generating a model corresponding to the user's foot is, A step of generating a mesh by simplifying the above point cloud; including, method.

7. In Paragraph 1, A step of displaying an image input interface configured to allow the user to input an image; including, method.

8. In Paragraph 1, A step of calculating information about the foot based on a model corresponding to the user's foot; including, method.

9. In Paragraph 1, The image composed of the above plurality of frames is, including a reference object, method.

10. As a device that provides information on the shot, Memory in which at least one program is stored; and A processor that operates by executing at least one of the above programs; comprising, The above processor is, Acquire an image composed of multiple frames, each including the user's foot, and Based on the above image, generate a model corresponding to the user's foot, The image above is, including images of the user's foot viewed from each of multiple angles, device.

11. A computer-readable recording medium storing a program for executing the method according to paragraph 1 on a computer.

Images