System for providing online fitting glasses manufacturing service using 3D head shape

Abstract

Provided is a system for providing an online fitting glasses manufacturing service using a 3D head shape, comprising: a user terminal that photographs a face and a head shape, selects a glasses frame, and uploads an optometry report; and a manufacturing service providing server including a modeling unit for generating a 3D head shape on the basis of the face and the head shape photographed by the user terminal, a selection unit for selecting a sample head shape most similar to the 3D head shape from among pre-manufactured sample head shapes, an output unit that compares the selected sample head shape with the 3D head shape and outputs parts having different sizes to a store terminal, and a fitting guide unit that outputs a pre-stored glasses frame fitting process to the store terminal if the size of the part of the sample head shape has been adjusted like the 3D head shape.

Description

System providing online fitting eyeglasses production services using 3D head images

[0001] The present invention relates to a system for providing an online fitting eyeglasses manufacturing service using a 3D head. The system selects a sample head similar to the 3D head, adjusts the sample head to fit the 3D head, and then fits an eyeglass frame selected by the user to complete the manufacturing of eyeglasses.

[0002] Online eyewear sales services are businesses that manufacture and deliver glasses ordered after users have virtually fitted them via an application. To produce the glasses, eye exam results from within the last six months must be entered online, and opticians affiliated with the retailer manufacture prescription lenses based on this data. Subsequently, the glasses are adjusted and completed to match face size, pupil position, interpupillary distance, and height measured via the application, and then shipped via courier. If the prescription glasses received via courier are uncomfortable, they can be returned or refunded free of charge. However, when glasses are manufactured online, the fitting service tailored to the individual's face—the most critical aspect for ensuring vision correction, the core function of eyewear—cannot be provided. Consequently, attempts are being made to send multiple frames for customers to try on and select, or to produce frames using 3D printing; however, these efforts have failed to gain traction and have not resolved the limitations of the online platform.

[0003] At this time, methods for performing 3D face modeling or virtually trying on glasses to produce custom glasses have been researched and developed. In this regard, prior art Korean Published Patent No. 2024-0158068 (published November 4, 2024) and Korean Registered Patent No. 10-2451690 (published October 7, 2022) each disclose a configuration for obtaining a 3D face model for glasses fitting by photographing a face wearing glasses and generating a real glasses model by modifying a standard glasses model corresponding to 3D data using glasses feature points, and a configuration for obtaining a face shape from an image of a user's face, predicting glasses frames corresponding to the face shape using an AI model, and overlaying the glasses frames recommended by the AI ​​model onto the user's face.

[0004] However, in the former case, it discloses only a configuration for 3D modeling a human face, but does not disclose a configuration for actually fitting eyeglass frames onto a head on which a human face has been 3D printed. In the latter case as well, it is merely a configuration for overlaying 3D eyeglass data onto 3D face data on a computer, and does not implement the process of fitting eyeglasses onto a head created to resemble a human face. Eyeglass fitting refers to the process of adjusting and fitting eyeglasses to the wearer's face shape and head to ensure accurate vision correction and comfortable wear. When eyeglass fitting is not done properly, it leads to negative optical and aesthetic results, causing not only poor comfort but also pain. Therefore, research and development of a system are required to enable eyeglass fitting in the online market as well.

[0005] One embodiment of the present invention provides a system for providing an online fitting eyeglasses manufacturing service using a 3D head, wherein a standard head shape for each race is prepared as a sample head, a device capable of adjusting and measuring the size of each part of the sample head in millimeters is embedded within the sample head, and when a face and head are photographed by a user terminal, a 3D head shape is generated and the most similar sample head shape is selected, the size difference between the selected sample head shape and the 3D head shape is identified and the size of each part of the sample head shape is adjusted, and a fitting process is provided to a store terminal to fit an eyeglass frame selected by the user onto the adjusted sample head shape, thereby providing the same effect as fitting to the actual user's face without the user actually visiting an optician to receive fitting. However, the technical problem that this embodiment aims to solve is not limited to the technical problem described above, and other technical problems may exist.

[0006] As a technical means for achieving the technical problem described above, one embodiment of the present invention includes a manufacturing service providing server comprising: a user terminal that photographs a face and head, selects an eyeglass frame, and uploads an eye examination report; a modeling unit that generates a 3D head based on the face and head photographed from the user terminal; a selection unit that selects a sample head most similar to the 3D head among previously produced sample heads; an output unit that compares the selected sample head and the 3D head and outputs parts with different sizes to a store terminal; and a fitting guide unit that outputs a previously stored eyeglass frame fitting process to a store terminal when the size of a part in the sample head is adjusted to be like the 3D head.

[0007] According to any one of the means for solving the problem of the present invention described above, a standard head shape for each race is prepared as a sample head shape, and a device capable of adjusting and measuring the size of each part of the sample head shape in millimeters is embedded within the sample head shape. When a face and head shape are captured by a user terminal, a 3D head shape is generated, and the most similar sample head shape is selected. After identifying the size difference between the selected sample head shape and the 3D head shape, the size of each part of the sample head shape is adjusted. By providing a fitting process to a store terminal to fit an eyeglass frame chosen by the user onto the adjusted sample head shape, the same effect as fitting to the actual user's face can be provided without the user actually visiting an optician to receive fitting.

[0008] FIG. 1 is a drawing for explaining an online fitting glasses manufacturing service provision system using a 3D head image according to an embodiment of the present invention.

[0009] Figure 2 is a block diagram illustrating a production service provision server included in the system of Figure 1.

[0010] FIGS. 3 and 4 are drawings for explaining an embodiment in which an online fitting glasses manufacturing service using a 3D head image is implemented according to an embodiment of the present invention.

[0011] FIG. 5 is a flowchart illustrating a method for providing an online fitting glasses manufacturing service using a 3D head image according to an embodiment of the present invention.

[0012] Embodiments of the present invention are described below with reference to the attached drawings so that those skilled in the art can easily implement the invention. However, the present invention may be embodied in various different forms and is not limited to the embodiments described herein. Furthermore, in order to clearly explain the present invention in the drawings, parts unrelated to the explanation have been omitted, and similar parts throughout the specification are denoted by similar reference numerals.

[0013] Throughout the specification, when a part is described as being "connected" to another part, this includes not only cases where they are "directly connected" but also cases where they are "electrically connected" with other elements interposed between them. Furthermore, when a part is described as "including" a component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components, and it should be understood that this does not preclude the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0014] Terms such as “about,” “substantially,” etc., used throughout the specification, are used to mean at or near the stated value when inherent manufacturing and material tolerances are presented in the stated meaning, and are used to prevent unscrupulous infringers from unfairly exploiting the disclosure in which precise or absolute values ​​are mentioned to aid in understanding the invention. Terms such as “step” or “step of” used throughout the specification of the invention do not mean “step for”.

[0015] In this specification, the term "part" includes a unit realized by hardware, a unit realized by software, and a unit realized using both. Additionally, one unit may be realized using two or more pieces of hardware, and two or more units may be realized by one piece of hardware. Meanwhile, "part" is not limited to software or hardware, and "part" may be configured to reside in an addressable storage medium or configured to run on one or more processors. Accordingly, as an example, "part" includes components such as software components, object-oriented software components, class components, and task components, as well as processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. The functions provided within the components and "parts" may be combined into a smaller number of components and "parts" or further separated into additional components and "parts." In addition, the components and '~parts' may be implemented to play one or more CPUs within the device or secure multimedia card.

[0016] Some of the operations or functions described herein as being performed by a terminal, device, or device may instead be performed by a server connected to said terminal, device, or device. Likewise, some of the operations or functions described as being performed by a server may also be performed by a terminal, device, or device connected to said server.

[0017] In this specification, some of the operations or functions described as mapping or matching with a terminal may be interpreted as meaning mapping or matching the terminal's unique number or personal identification information, which is the terminal's identifying data.

[0018] The present invention will be described in detail below with reference to the attached drawings.

[0019] FIG. 1 is a diagram illustrating an online fitting glasses manufacturing service provision system using a 3D head image according to an embodiment of the present invention. Referring to FIG. 1, the online fitting glasses manufacturing service provision system (1) using a 3D head image may include at least one user terminal (100), a manufacturing service provision server (300), at least one sample head image (400), and at least one store terminal (500). However, since the online fitting glasses manufacturing service provision system (1) using a 3D head image of FIG. 1 is merely an embodiment of the present invention, the present invention is not to be interpreted as being limited by FIG. 1.

[0020] At this time, each component of FIG. 1 is generally connected through a network (Network, 200). For example, as shown in FIG. 1, at least one user terminal (100) can be connected to a production service provider server (300) through the network (200). And, the production service provider server (300) can be connected to at least one user terminal (100), at least one sample head (400), and at least one store terminal (500) through the network (200). Also, at least one sample head (400) can be connected to the production service provider server (300) through the network (200). And, at least one store terminal (500) can be connected to at least one user terminal (100), the production service provider server (300), and at least one sample head (400) through the network (200).

[0021] Here, a network refers to a connection structure capable of exchanging information among individual nodes, such as multiple terminals and servers. Examples of such networks include Local Area Networks (LANs), Wide Area Networks (WANs), the World Wide Web (WWW), wired and wireless data networks, telephone networks, and wired and wireless television networks. Examples of wireless data communication networks include, but are not limited to, 3G, 4G, 5G, 3GPP (3rd Generation Partnership Project), 5GPP (5th Generation Partnership Project), 5G NR (New Radio), 6G (6th Generation of Cellular Networks), LTE (Long Term Evolution), WIMAX (World Interoperability for Microwave Access), Wi-Fi, Internet, LAN (Local Area Network), Wireless LAN (Wireless Local Area Network), WAN (Wide Area Network), PAN (Personal Area Network), RF (Radio Frequency), Bluetooth network, NFC (Near-Field Communication) network, satellite broadcasting network, analog broadcasting network, DMB (Digital Multimedia Broadcasting) network, etc.

[0022] In the following, the term "at least one" is defined as a term including both singular and plural forms, and it will be obvious that even if the term "at least one" does not exist, each component may exist in a singular or plural form and may mean singular or plural. Furthermore, whether each component is provided in a singular or plural form may be changed according to the embodiment.

[0023] At least one user terminal (100) may be a user terminal that uploads an image of a face and head taken using a web page, app page, program, or application related to an online fitting glasses manufacturing service using a 3D head image to a manufacturing service providing server (300), selects a glasses frame, and then uploads an eye examination report.

[0024] Here, at least one user terminal (100) may be implemented as a computer capable of connecting to a remote server or terminal via a network. Here, the computer may include, for example, a navigation system, a laptop equipped with a web browser, a desktop, a laptop, etc. At this time, at least one user terminal (100) may be implemented as a terminal capable of connecting to a remote server or terminal via a network. At least one user terminal (100) may include all kinds of handheld-based wireless communication devices, such as navigation, PCS (Personal Communication System), GSM (Global System for Mobile communications), PDC (Personal Digital Cellular), PHS (Personal Handyphone System), PDA (Personal Digital Assistant), IMT (International Mobile Telecommunication)-2000, CDMA (Code Division Multiple Access)-2000, W-CDMA (W-Code Division Multiple Access), Wibro (Wireless Broadband Internet) terminal, smartphone, smartpad, tablet PC, etc.

[0025] The production service providing server (300) may be a server that provides a web page, app page, program, or application for an online fitting glasses production service using a 3D head. Additionally, the production service providing server (300) may be a server that generates a 3D head through 3D modeling when an image is received from a user terminal (100), checks the similarity between the 3D head and a previously stored sample head, and extracts the sample head (400) with the highest similarity. Furthermore, the production service providing server (300) may be a server that outputs the difference between the extracted sample head (400) and the 3D head and transmits it to a store terminal (500) so that an employee or optician can directly adjust the size of each part of the sample head (400) (manual), or adjust the size of each part using a device provided within the sample head (400) (automatic). And, the production service providing server (300) may be a server that transmits a fitting process to a store terminal (500) and, when the store terminal (500) outputs a fitting completion event, outputs a shipping label so that it can be transported to the user's delivery address.

[0026] Here, the production service providing server (300) may be implemented as a computer capable of connecting to a remote server or terminal via a network. Here, the computer may include, for example, a navigation system, a laptop equipped with a web browser, a desktop, a laptop, etc.

[0027] At least one sample head (400) may be a device that adjusts the size of each part by receiving adjustment values ​​from a production service provider server (300) so that the size of each part is the same as the 3D head using a web page, app page, program, or application related to an online fitting glasses production service using a 3D head. In order to increase or decrease the size of each part, the length of the adjustment rod supporting each part may be increased or decreased by a gear that is rotated by a motor, but the embodiment in which the size of each part is increased or decreased is not limited thereto.

[0028] Here, at least one sample head (400) may be implemented as a computer capable of connecting to a remote server or terminal via a network. Here, the computer may include, for example, a navigation system, a laptop equipped with a web browser, a desktop, a laptop, etc. At this time, at least one sample head (400) may be implemented as a terminal capable of connecting to a remote server or terminal via a network. At least one sample head (400) may include all kinds of handheld-based wireless communication devices, such as navigation, PCS (Personal Communication System), GSM (Global System for Mobile communications), PDC (Personal Digital Cellular), PHS (Personal Handyphone System), PDA (Personal Digital Assistant), IMT (International Mobile Telecommunication)-2000, CDMA (Code Division Multiple Access)-2000, W-CDMA (W-Code Division Multiple Access), Wibro (Wireless Broadband Internet) terminal, smartphone, smartpad, tablet PC, etc.

[0029] At least one store terminal (500) may be a terminal that receives eyeglass frames and an eye examination report from a user terminal (100) using a web page, app page, program, or application related to an online fitting eyeglass manufacturing service using a 3D head, and outputs a fitting process to fit eyeglass frames to a sample head (400).

[0030] Here, at least one store terminal (500) may be implemented as a computer capable of connecting to a remote server or terminal via a network. Here, the computer may include, for example, a navigation system, a laptop equipped with a web browser, a desktop, a laptop, etc. At this time, at least one store terminal (500) may be implemented as a terminal capable of connecting to a remote server or terminal via a network. At least one store terminal (500) may include all kinds of handheld-based wireless communication devices, such as navigation, PCS (Personal Communication System), GSM (Global System for Mobile communications), PDC (Personal Digital Cellular), PHS (Personal Handyphone System), PDA (Personal Digital Assistant), IMT (International Mobile Telecommunication)-2000, CDMA (Code Division Multiple Access)-2000, W-CDMA (W-Code Division Multiple Access), Wibro (Wireless Broadband Internet) terminal, smartphone, smartpad, tablet PC, etc.

[0031] FIG. 2 is a block diagram for explaining a production service providing server included in the system of FIG. 1, and FIG. 3 and FIG. 4 are drawings for explaining an embodiment in which an online fitting glasses production service using a 3D head image is implemented according to an embodiment of the present invention.

[0032] Referring to FIG. 2, the production service providing server (300) may include a modeling unit (310), a selection unit (320), an output unit (330), a fitting guide unit (340), an automatic control unit (350), and a shooting guide unit (360).

[0033] When a production service providing server (300) or another server (not shown) operating in conjunction with a 3D head image transmits an online fitting glasses production service application, program, app page, web page, etc., using a 3D head image to at least one user terminal (100), at least one sample head image (400), and at least one store terminal (500), the at least one user terminal (100), at least one sample head image (400), and at least one store terminal (500) may install or open the online fitting glasses production service application, program, app page, web page, etc., using a script executed in a web browser, the service program may be operated on at least one user terminal (100), at least one sample head image (400), and at least one store terminal (500). Here, a web browser refers to a program that enables the use of web (WWW: World Wide Web) services and receives and displays hypertext described in HTML (Hyper Text Mark-up Language), and includes, for example, Chrome, Microsoft Edge, Safari, Firefox, Whale, UC Browser, etc. Additionally, an application refers to an application on a terminal, and includes, for example, an app running on a mobile terminal (smartphone).

[0034] Referring to FIG. 2, the modeling unit (310) can generate a 3D head based on a face and head captured from a user terminal (100). The user terminal (100) can capture the face and head, select eyeglass frames, and upload an eye examination report. At this time, the face may include front, side, top, and bottom views, and the head may refer to images such as top, back, and side views. Here, in order to determine the absolute value more accurately, the user terminal (100) may take a picture with a coin placed next to the face, or take a picture with a banknote or credit card, thereby reducing the error when calculating the absolute value. Even if a monocular camera or stereo camera located on a mobile phone produces an effect similar to a depth camera, or if a real depth camera is used to capture the depth, an error may still occur. Therefore, the error can be reduced by taking a picture with an object that everyone possesses and whose value is known, such as a coin, a banknote, or a credit card.

[0035] In face images, the system can identify each landmark constituting the facial feature points, while in head images, it can identify the shape of the face—that is, the facial contours, overall curves and proportions, and the position or arrangement of the eyes, nose, and ears. Of course, since facial feature points, facial contours, or head shape can be identified from any photograph, the specific data to be extracted from each photograph is not limited to just one.

[0036] In one embodiment of the present invention, although a user may perform a 3D scan directly using a 3D scanner, the description is generally based on capturing the image using a smartphone camera. Accordingly, the description is based on converting a 2D coordinate system into a 3D coordinate system by applying a point cloud to a depth image.

[0037] Point Cloud

[0038] A point cloud is a set of points belonging to a specific coordinate system. In a 3D coordinate system, a point is generally defined by (X, Y, Z), and a 2D coordinate system can be converted to a 3D coordinate system by applying a point cloud to a depth image. First, to convert a 2D depth image into a 3D coordinate system, the focal length, the coordinates of the 2D depth image, and the depth value are utilized from the intrinsic parameter information of the camera that captured the depth image. Here, the focal length is the distance from the camera lens to the image sensor. For example, when a camera photographs a sphere, light reflected from the sphere passes through the lens. The light passing through the lens forms an image on the image sensor, obtaining a 2D depth image of the sphere. The 2D depth image must then be used to convert from the 2D coordinate system to the 3D coordinate system.

[0039] Here, the Similar TriAngles Rule can be used to perform a simple transformation. Assuming that f is the focal length, u is the coordinate where the object is located in the depth image, the origin of the coordinates is the center of the image rather than the top-left corner, z is the depth value corresponding to the 2D coordinate in the 2D depth image, and x is the X-axis coordinate of the object when converted to a 3D coordinate system, the x-coordinate can be estimated by substituting these values ​​into Equation 1 below. Since the value obtained by dividing the distance in the x-axis direction (u) from the center of the 2D image to the corresponding pixel by the focal length (f) is equal to the value obtained by dividing the x-coordinate in the 3D coordinate system by the depth value, the x-coordinate can be estimated. Thus, u in the 2D depth image can be converted to a 3D coordinate with respect to the X-axis, and by repeating the above method for the Y-axis, the 2D coordinate system can be converted to a 3D coordinate system.

[0040] [Mathematical Formula 1]

[0041]

[0042] <Coordinate Transformation>

[0043] Coordinate transformation is the process of changing the position of a point to another position within a coordinate system. In computer vision, rotation, scaling, and translation transformations can be used to model matching relationships between multiple images.

[0044] <3D Coordinate Transformation>

[0045] 3D coordinate transformation is the transformation of coordinate axes in 3D space. 3D coordinate transformation can be utilized by converting the 3D world coordinate system to the camera coordinate system or vice versa. The objective of this process is to obtain the point (X,Y,Z) in the world coordinate system from a point (X1,Y1,Z1) in the camera coordinate system by always rotating the coordinate axes and then performing a translation transformation. In the 3D world coordinate system, the plane of the X and Y axes represents the ground, and the Z axis is defined perpendicular to this plane. In the camera coordinate system, the Zc axis is defined in the direction of the camera's optical axis, the Xc axis is defined perpendicular to the right of the optical axis, and the Yc axis is defined perpendicular to both the Xc and Zc axes.

[0046] The rotation transformation rotates a point (X,Y,Z) in 3D space around the X, Y, and Z axes by an amount θ using a rotation matrix, rotating it counterclockwise relative to the direction of each axis. The rotation matrices for rotating each axis are given by Equations 2, 3, and 4. Equation 2 is a matrix for rotation around the X-axis, Equation 3 is a matrix for rotation around the Y-axis, and Equation 4 is a matrix for rotation around the Z-axis. Any 3D rotation, such as Equation 5, can be expressed by combining the rotation matrices of Equations 2 through 4.

[0047] [Mathematical Formula 2]

[0048]

[0049] [Mathematical Formula 3]

[0050]

[0051] [Mathematical Formula 4]

[0052]

[0053] [Mathematical Formula 5]

[0054]

[0055] [Mathematical Formula 6]

[0056]

[0057] The translation transformation is performed by rotating the point (XC,YC,ZC) in the camera coordinate system to the point (X,Y,Z) in the world coordinate system, and then translating this camera coordinate system into the direction of the world coordinate system using Equation 6.

[0058] <Object Recognition>

[0059] Object detection is the process of analyzing a given image to identify the location and class of a specific object. Object detection algorithms must first locate the object's position (localization) within the image and then classify the object at that location. Various methods have been attempted to achieve this, but recently, the focus has narrowed down to single-stage and two-stage detection methods. Single-stage detection performs location detection and classification simultaneously across all regions, whereas two-stage detection performs approximate location detection and then classifies from a selected group of candidates. Since single-stage detection performs location detection and classification simultaneously across all regions, it is faster but suffers from the disadvantage of slightly lower accuracy compared to two-stage detection. Conversely, because two-stage detection performs location detection and classification sequentially, it offers relatively superior accuracy but has the disadvantage of being slower. In this case, since accuracy is more important than speed in one embodiment of the present invention, a two-stage detection method such as CNN is used rather than a single-stage detection method such as SSD or YOLO that identifies quickly. Of course, the use of a single-stage detection method is not excluded.

[0060] In addition to this, various commercial programs can be used; for example, you can use Get3D (Generate Explicit Textured 3D), released by NVIDIA in 2022. Get3D is a 3D generative model capable of synthesizing any topology and high-quality 3D polygon mesh from a single 2D image input. The polygon mesh consists of triangles, a standard format that can be seamlessly imported into graphics software such as 3D programs, game engines, or movie renderers. Alternatively, you can use DragGAN, a collaboration between Google and MIT (Pan, Xingang, Ayush Kumar Tewari, Thomas Leimkuhler, Lingjie Liu, Abhimitra Meka and Christian Theobalt. “Drag Your GAN: Interactive Point-based Manipulation on the Generative Image Manifold.” ACM SIGGRAPH 2023 Conference Proceedings (2023): n. pag.). Alternatively, 2D images can be converted into 3D images using NeRF (Neural Radiance Fields). However, there is a possibility that the generator in GAN-based AI models may create missing data in the user's face image. Since this can cause problems in generating the head shape, which requires millimeter-level accuracy, AI models that fill in missing data using GANs should be excluded as much as possible, and a tool that constructs 3D data solely from photos taken by the user can be utilized. If the user has insufficient photos, they can request a retake and continue taking photos until a photo from the desired angle is obtained.

[0061] The selection unit (320) can select the sample head most similar to the 3D head among the sample heads (400) that have been produced.

[0062] <Sample Production>

[0063] Sample head shapes (400) can be provided in quantities of approximately 20 to 30 for each race. To this end, in the case of Koreans, a dataset of Korean facial images is constructed as public data (AI Hub) as shown in FIG. 4e, and data on Korean heads (head shapes) is also constructed by age and gender as shown in FIG. 4f to 4i. Therefore, a sample to serve as a standard can be selected using this, and a sample head shape (400) can be generated based on the numerical values ​​to be sampled. At this time, when selecting the sample, clustering can be performed on the Korean facial image and head shape data using a clustering algorithm, and a head shape can be created by selecting a sample for each cluster. For example, the clustering algorithm can perform hierarchical clustering using the K-Means algorithm. In addition to these, various unsupervised learning-based deep learning algorithms such as Mean Shift, Gaussian Mixture Model (GMM), and Density Based Spatial Clustering of Applications with Noise (DBSCAN) can be used, and deep learning algorithms are not limited to those listed and are not excluded for reasons not listed.

[0064] <Exterior of Sample Head>

[0065] The sample head (400) is a head pre-fabricated according to at least one shape of a human head, and may have a device built in to adjust and measure the size of at least one part in millimeters. That is, the sample head (400) means that, as shown in FIG. 3c, a device capable of adjusting the size of each part of the face is built in, and the outer skin of the sample head (400) is formed of polyurethane having elasticity to be adjusted in millimeters. It may be similar to a mannequin face, but with a device located inside that can reduce or increase the size of each part, such as nose height, ear angle, ear size, and face width, and the outside is wrapped in polyurethane so that the size of the sample head (400) itself can be adjusted. The outer skin may be formed like a silicone mask in the shape of a human face as shown in FIG. 4k, but is not limited thereto. FIG. 4k is a drawing to explain what a silicone mask is (Source: SFXKOREA).

[0066] <Inside the sample head>

[0067] Although FIG. 3c is drawn simply for illustrative purposes, the adjustment rods for each part may be more densely embedded in the sphere, and the thickness of each adjustment rod may be formed thinner to allow for detailed representation of the shape of each part. Additionally, at the end of the adjustment rod, that is, the part that meets the outer shell, a plate in the shape of each part may exist. This plate may be formed from silicone material, but is not limited thereto. For example, there may be plates in the shape of ears, noses, cheekbones, etc., and accordingly, the polyurethane exterior shape can be maintained. Of course, if each plate is provided as a single piece, detailed adjustment of the angle or height of the face may not be possible; therefore, each plate may be divided into a mosaic form as shown in FIG. 4j, and each unit module, the mosaic plate, may be attached to the end of a single adjustment rod. Furthermore, gears and motors may be additionally installed to connect to the adjustment rod so that the length of each adjustment rod can be shortened or extended. In this case, Fig. 4j is not an actual picture but an inserted image to aid in understanding each mosaic panel (AI created, source: pikepicture).

[0068] Once both the interior and exterior are completed in this way, each sample head (400) is assigned a unique identification code, and the size of each part can be mapped and stored.

[0069] <Calculation of Feature Vector Similarity>

[0070] The selection unit (320) must find and select the sample head (400) that is most similar to the user's 3D head. This is because the sample head (400) can be made to match the user's 3D head, and the eyeglass frame can be fitted onto the sample head (400). To do this, the sample head (400) that is most similar to the 3D head can be found based on similarity. Although the sample with the most similar size of each part is found, the highest priority may be the face shape. This is because if the face shape is different, even if the position or size of other parts such as the eyes, nose, mouth, and ears is similar, there will be a significant discrepancy when fitting the eyeglass frame. In addition, similarity can be determined by assigning different weights to each part in addition to the face shape. To do this, a method can be used to extract feature vectors of the 3D head and the sample head (400) and determine the similarity between each feature vector, and similarity can be calculated by assigning the aforementioned weights to each feature vector. In this case, the similarity may be, for example, cosine similarity, but is not limited thereto.

[0071] The output unit (330) can compare the selected sample head (400) with the 3D head and output the parts with different sizes to the store terminal (500). In one embodiment of the present invention, the sample head (400) is provided with a device that allows a person to manually adjust the size of each part, and since the length of each adjustment rod can be measured and adjusted in millimeters, a [part-adjustment value] with a different size can be sent in preparation for cases where an optician or dedicated staff member directly fits it. For example, if the user's nose length is 5cm and the nose length of the sample head (400) is 54mm, the adjustment value may be -4mm and may be displayed as [nose length-[-4mm]]. The parts for measuring size may be the 19 parts shown in FIGS. 4a to 4d. However, it is not limited to these 19 parts.

[0072] [Table 1]

[0073]

[0074] [Table 2]

[0075]

[0076] [Table 3]

[0077]

[0078] Accordingly, when comparing the selected sample head (400) with the 3D head, the output unit (330) can output the size of the pupil distance, which is the distance between the pupil focal points; the nose pad distance, which is the distance between the positions where the nose pads of the eyeglass frames are seated; the temporal width, which is the distance between the temporales where the temples of the eyeglass frames are located; the nose pad spacing, which is the distance from the top of the nose to the position of the nose pads; the upper part of both ears (the upper part of both ears), the face width corresponding to the cheekbone width; the cheekbone height, which is the length from the nose to the top of the cheekbone; the face length, which is the length from the forehead to the chin; the chin width; the nose height, which is the length from the tip of the nose to the face; the eyebrow height, which is the length from the top of the eye to the eyebrow; the ear position, which is the distance from the nose to the ear; the nose pad depth; the forehead width; the facial inclination angle tilted toward the eye with the nose as the center; the eyeball position, which is the distance between both eyes; the medial canthal inclination, which is the angle between the inner and outer sides of the eye; the nose tip position within the face; and the temporal bone depth, which is the depth of the unevenness of the face line next to the ear.

[0079] The fitting guide section (340) can output a previously stored eyeglass frame fitting process to a store terminal (500) when the size of a part of the sample head (400) is adjusted to be like a 3D head. The eyeglass frame fitting process may consist of a first step of checking the facial angle and the forward angle, a second step of spreading the temples of the eyeglass frame to adjust the width, a third step of balancing the left and right sides of the nose pads of the eyeglass frame, a fourth step of setting the upper, middle, and lower contact points of the eyeglass frame, and a fifth step of attaching the temples of the eyeglass frame so that the ends of the temples face inward, but is not limited thereto. However, the guide can be transmitted to the store terminal (500) and the optician can determine whether the check items for each step have been properly checked.

[0080] The automatic control unit (350) controls a device that adjusts the size of at least one part inside the sample head (400), thereby automatically controlling the size of the sample head (400) with different sizes, such as the 3D head. As described above, the size of each part of the sample head (400) can be manually adjusted, but if the length of each adjustment rod is automatically controlled, the size of each part can be adjusted to match the size of each part of the 3D head. Since the output unit (330) provides a [part-adjustment value] regarding how much the size of each part differs, the size of each part can be adjusted by shortening or lengthening the length of each adjustment rod based on this. If the adjustment rod itself is designed with a structure that automatically extends and retracts, such as a foldable radio antenna (antenna pole) mounted on a vehicle, the length of the adjustment rod can be automatically extended or shortened. In addition to this, methods that have recently gained popularity as height adjustment devices, such as electric, electromechanical, hydraulic, and pneumatic devices, may also be possible.

[0081] The shooting guide section (360) can guide the shooting so that the face and head are included within the guideline by providing a guideline according to the shooting angle when shooting the face and head on the user terminal (100), and shooting the front, side, back, top, and bottom views respectively.

[0082] Hereinafter, the operation process according to the configuration of the production service providing server of FIG. 2 described above will be explained in detail with reference to FIG. 3 and FIG. 4. However, it is obvious that the embodiment is merely one of the various embodiments of the present invention and is not limited thereto.

[0083] Referring to FIG. 3a, (a) a production service providing server (300) can receive images of the face of a user terminal (100) taken from various angles. Here, images taken from various angles to obtain results similar to a 3D scanner can be received, including front, bottom, side, back, and top views. Then, the production service providing server (300) can convert 2D data into 3D data as in (b) to obtain a 3D head, select a sample head (400) most similar to the 3D head as in (c), and then (d) compare the difference between [3D head - sample head] and provide it to a store terminal (500) as in (a) of FIG. 3b. At this time, it may be made so that adjustments are made automatically based on adjustment values ​​as in (b).

[0084] Preferably, after performing automatic adjustment as in (b), it is desirable for a person to finish as in (a). This is because, even if control is performed well using motors or hydraulic cylinders, it is not easy to control in millimeter units, and errors are bound to occur in control. In this case, when a person readjusts the result of automatic control, it can be updated to gradually reduce control errors by feeding it back as a compensation value to PID control or using it in recent deep learning-based reinforcement learning. Then, after providing a fitting guide to the store terminal (500) as in (c), if it is completed (d), the store terminal (500) prints out a shipping label for delivery, and feedback is received from the user terminal (100) to be applied to the next fitting. FIG. 3c is an example of the internal structure of a sample head (400), and FIG. 4a to 4d show the location, description, and average size of each part in Tables 1 to 3. FIGS. 4e to 4i illustrate a Korean facial and head dataset used when creating a sample head (400), FIG. 4j illustrates an embodiment in which each part of the sample head (400) is mosaicked into unit modules, and FIG. 4k illustrates a silicone mask that can be placed over it. Of course, as previously mentioned, the outer material may be polyurethane rather than silicone.

[0085] Any details regarding the method of providing an online fitting glasses manufacturing service using a 3D head image as shown in FIGS. 2 to 4 that are not described are identical to or can be easily inferred from the description of the method of providing an online fitting glasses manufacturing service using a 3D head image as described in FIG. 1, so further explanation will be omitted.

[0086] FIG. 5 is a diagram illustrating the process of transmitting and receiving data between each component included in the system for providing an online fitting glasses manufacturing service using a 3D head image of FIG. 1 according to an embodiment of the present invention. Hereinafter, an example of the process of transmitting and receiving data between each component will be described through FIG. 5, but the present invention is not to be interpreted as being limited to such an embodiment, and it is obvious to those skilled in the art that the process of transmitting and receiving data illustrated in FIG. 5 may be changed according to various embodiments described above.

[0087] Referring to FIG. 5, the production service providing server generates a 3D head image based on a face and head image captured from a user terminal (S5100).

[0088] Then, the production service provider server selects the head most similar to the 3D head from among the previously produced sample heads (S5200), compares the selected sample head with the 3D head, and outputs the parts with different sizes to the store terminal (S5300).

[0089] Additionally, if the size of a part of the sample head is adjusted to be like a 3D head, the production service provider outputs the previously stored eyeglass frame fitting process to the store terminal (S5400).

[0090] The order of the steps described above (S5100~S5400) is merely an example and is not limited thereto. That is, the order of the steps described above (S5100~S5400) may vary, and some of these steps may be executed simultaneously or deleted.

[0091] Any details regarding the method of providing an online fitting glasses manufacturing service using a 3D head image of Fig. 5 that are not described are identical to or can be easily inferred from the description of the method of providing an online fitting glasses manufacturing service using a 3D head image through Figs. 1 to 4, so further explanation will be omitted.

[0092] A method for providing an online fitting glasses manufacturing service using a 3D head image according to an embodiment described through FIG. 5 may also be implemented in the form of a recording medium containing computer-executable instructions, such as an application or program module executed by a computer. A computer-readable medium may be any available medium accessible by a computer and includes both volatile and non-volatile media, as well as removable and non-removable media. Additionally, a computer-readable medium may include all computer storage media. A computer storage medium includes both volatile and non-volatile, removable and non-removable media implemented by any method or technique for storing information, such as computer-readable instructions, data structures, program modules, or other data.

[0093] The method for providing an online fitting glasses manufacturing service using a 3D head image according to one embodiment of the present invention described above may be executed by an application basically installed on a terminal (which may include a program included in a platform or operating system, etc., basically installed on the terminal), or by an application (i.e., a program) directly installed by a user on a master terminal through an application providing server, such as an application store server, an application, or a web server related to the service. In this sense, the method for providing an online fitting glasses manufacturing service using a 3D head image according to one embodiment of the present invention described above may be implemented as an application (i.e., a program) that is basically installed on a terminal or directly installed by a user, and may be recorded on a computer-readable recording medium such as a terminal.

[0094] The foregoing description of the present invention is for illustrative purposes only, and those skilled in the art will understand that other specific forms can be easily modified without altering the technical spirit or essential features of the present invention. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. For example, each component described as a single unit may be implemented in a distributed manner, and components described as distributed may likewise be implemented in a combined form.

[0095] The scope of the present invention is defined by the claims set forth below rather than by the detailed description above, and all modifications or variations derived from the meaning and scope of the claims and equivalent concepts thereof should be interpreted as being included within the scope of the present invention.

Claims

1. A user terminal that photographs the face and head, selects eyeglass frames, and uploads an eye examination report; and A production service providing server comprising: a modeling unit that generates a 3D head based on a face and head captured from the user terminal; a selection unit that selects a sample head most similar to the 3D head among pre-produced sample heads; an output unit that compares the selected sample head with the 3D head and outputs parts with different sizes to a store terminal; and a fitting guide unit that, when the size of the part in the sample head is adjusted to match the 3D head, outputs a pre-stored eyeglass frame fitting process to the store terminal and determines whether an optician has checked the step-by-step check items within the eyeglass frame fitting process. The above modeling unit is, A 3D head is generated based on a face image including the front, side, top, and bottom views of the face captured from the user terminal to identify each landmark forming the feature points of the face, and top and back and side head images captured from the user terminal to identify the face shape, the overall curvature or proportions of the face, and the position or arrangement of the eyes, nose, and ears. The above user terminal is, In order to reduce error when calculating absolute values, an object with a known value is photographed together with the face image and the head image, The above eyeglass frame fitting process is, Step 1 for checking facial angle and anterior tilt angle, Step 2: Adjusting the width by spreading the temples of the eyeglass frames. Step 3: Balancing the left and right sides of the eyeglass frame nose pads, Step 4, setting the upper, middle, and lower contact points of the above eyeglass frame, A system for providing an online fitting glasses manufacturing service using a 3D head image, characterized by a fifth step of performing ear attachment so that the ends of the temples of the glasses frames face inward.

2. In Paragraph 1, The above sample head is, It is a head that has been pre-made according to at least one form of a human head, and A system for providing an online fitting glasses manufacturing service using a 3D head, characterized by having a built-in device for adjusting and measuring the size of at least one of the above-mentioned parts in millimeters.

3. In Paragraph 2, The above output unit is, A system for providing an online fitting glasses manufacturing service using a 3D head, characterized by comparing and outputting the sizes of the following when comparing a selected sample head with the 3D head: pupil distance, which is the distance between pupil focal points; nose pad distance, which is the distance between positions where the nose pads of the eyeglass frames are seated; temporal width, which is the width between the temporales where the temples of the eyeglass frames are located; nose pad spacing, which is the distance from the top of the nose to the position of the nose pads; the position of the upper part of both ears; face width corresponding to the cheekbone width; cheekbone height, which is the length from the nose to the top of the cheekbone; face length, which is the length from the forehead to the chin; chin width; nose height, which is the length from the tip of the nose to the face; eyebrow height, which is the length from the top of the eye to the eyebrow; ear position, which is the distance from the nose to the ear; nose pad depth; forehead width; facial inclination angle tilted toward the eye centered on the nose; eyeball position, which is the distance between both eyes; medial canthal inclination, which is the angle between the inner and outer sides of the eye; nose tip position within the face; and temporal bone depth, which is the depth of the unevenness of the facial line next to the ear.

4. In Paragraph 3, The above-mentioned production service providing server is, An automatic control unit that controls a device for adjusting the size of at least one part inside the sample head to automatically control the size of the sample head of a part of a different size, such as the 3D head; A system for providing an online fitting glasses manufacturing service using a 3D head image, characterized by further including 5. In Paragraph 1, The outer epidermis of the above-mentioned sample head is, A system for providing an online fitting glasses manufacturing service using a 3D head image, characterized by being formed of polyurethane having elasticity that allows adjustment in millimeter units.

6. In Paragraph 1, The above-mentioned production service providing server is, A shooting guide unit that, when shooting a face and head at the above-mentioned user terminal, allows the front, side, back, top, and bottom views to be shot respectively, and provides guidelines according to the shooting angle to induce shooting such that the face and head are included within the guidelines; A system for providing an online fitting glasses manufacturing service using a 3D head image, characterized by further including

Images