Color measurement method, apparatus, and system using physical medium, and physical medium thereof

Abstract

A color measurement method performed by an apparatus according to an embodiment of the present invention may comprise the operations of: acquiring an image including at least one target object and a physical medium including a plurality of reference color regions disposed to be spaced apart from each other; and displaying, on a display, color information related to the at least one target object.

Description

A method, device, system, and physical medium for color measurement using a physical medium

[0001] The present invention relates to color measurement technology. In particular, it relates to a method for obtaining accurate color values ​​of a target object using a physical medium, an apparatus and system for performing the same, and said physical medium. More specifically, it relates to a technology that enables the provision and utilization of precise color information by improving the accuracy and convenience issues of existing color measurement technologies in various fields, including dentistry.

[0002] Existing color measurement technologies have been widely used in various industrial fields such as medicine, design, and manufacturing; however, their application is limited in areas where minute color differences have a decisive impact on the quality of the final product.

[0003] Depending on the industry, color accuracy is directly linked to customer satisfaction, but the reliability of measurement results is frequently compromised due to ambient lighting conditions, the sensitivity of measuring equipment, and environmental factors. This problem makes it difficult to obtain consistent color data in specialized fields that require accurate colors.

[0004] Color judgment based on the naked eye is subjective and can vary significantly depending on the operator's experience and skill level. Even skilled professionals often found it difficult to accurately distinguish colors due to changes in lighting conditions or visual fatigue. Furthermore, even when using specialized color measurement equipment, it was difficult to obtain consistent data because results varied depending on the color temperature of the lighting, the condition of the equipment, or changes in the measurement environment. This acted as a major cause of unexpected color discrepancies.

[0005] Furthermore, existing color measurement technologies often provide only a single color value, failing to reflect the diversity of the objects being measured. For example, even a single tooth in a dental setting contains various color values; if this is represented by only a single value, fabricated prosthetics may fail to harmonize with the actual tooth in terms of color. Such color discrepancies not only result in wasted time and costs due to the need for additional rework, but can also lead to patient dissatisfaction and reduced work efficiency for dental professionals. Therefore, the development of more precise and reliable color measurement technology is required to meet the diverse demands of the industry.

[0006] The present invention aims to provide a method, system, and physical medium that significantly improve the color measurement accuracy of a target object while minimizing the influence of the external environment.

[0007] The present invention aims to perform accurate color correction from an image obtained using a physical medium and to reliably derive the actual color value of a target object.

[0008] The present invention aims to enhance the usability of result values ​​by providing accurate color values ​​for each area of ​​an object.

[0009] The present invention aims to improve the efficiency and processing speed of the color measurement process by applying marker detection and object detection technologies.

[0010] A color measurement method performed in a device according to an embodiment of the present invention may include the operation of acquiring an image comprising a physical medium including a plurality of reference color regions spaced apart from each other and at least one target object, and the operation of displaying color information related to the at least one target object through a display.

[0011] In one embodiment, each of the plurality of reference color regions may have different color values ​​within a certain color range.

[0012] In one embodiment, the color range is characterized by excluding high-luminosity or low-luminosity regions.

[0013] In one embodiment, the operation of displaying color information related to at least one target object through a display may include the operation of displaying color information for each of a plurality of regions corresponding to a specific target object among the at least one target object.

[0014] In one embodiment, the color information may be displayed as a color block or text.

[0015] In one embodiment, the color range may exclude white and black.

[0016] In one embodiment, color information related to at least one target object can be generated based on the plurality of reference color areas.

[0017] An apparatus for performing a color measurement method according to one embodiment of the present invention may include an image acquisition device for acquiring an image comprising a plurality of reference color regions spaced apart from each other and at least one target object, a communication circuit for communicating with an external server, a display, and a processor connected to the image acquisition device, the communication circuit, and the display.

[0018] In one embodiment, the processor may be configured to display color information related to the at least one target object through the display.

[0019] In one embodiment, each of the plurality of reference color regions may have different color values ​​within a certain color range.

[0020] In one embodiment, the color range may exclude high or low brightness areas.

[0021] A color measurement system according to one embodiment of the present invention may include a physical medium comprising a plurality of reference color regions, an image acquisition device that captures the physical medium and at least one target object to acquire image data, and a computational device that detects the physical medium from the image data and acquires a color value of the at least one target object based on the detected physical medium.

[0022] In one embodiment, each of the plurality of reference color regions may have different color values ​​within a certain color range.

[0023] In one embodiment, the color range may be characterized by excluding high or low brightness areas.

[0024] A physical medium according to one embodiment of the present invention may include a plurality of reference color regions.

[0025] In one embodiment, the plurality of reference color regions are arranged in an N x M matrix structure and are positioned at a constant distance from each other, and may include a background region that is distinct from the plurality of reference color regions.

[0026] In one embodiment, each of the plurality of reference color regions may have different color values ​​within a certain color range.

[0027] In one embodiment, the color range may exclude high or low brightness areas.

[0028] According to the color measurement method, device, system, and physical medium of the present invention, accurate color values ​​of a target object can be obtained despite environmental factors such as external lighting through precise color correction.

[0029] According to the color measurement method, device, system, and physical medium of the present invention, the accuracy of the result can be improved by providing region-specific color values ​​for a target object that can be expressed in multiple colors. This provides accurate information to dentists and dental technicians, thereby reducing the error rate and increasing time efficiency.

[0030] According to the color measurement method, device, system, and physical medium of the present invention, the computational speed and efficiency of image processing and color measurement processes can be improved.

[0031] According to the color measurement method, device, system, and physical medium of the present invention, user convenience can be improved by providing intuitive color information through a user interface.

[0032] FIG. 1 is a schematic block diagram showing the overall system configuration of the present invention.

[0033] FIG. 2 is a schematic plan view showing the physical medium configuration of the present invention.

[0034] FIG. 3 is a schematic block diagram showing an example of an image in which a physical medium and a target object are included together in the present invention.

[0035] FIG. 4 is a flowchart of a color measurement algorithm according to one embodiment of the present invention.

[0036] FIG. 5 is a flowchart of a color measurement algorithm according to one embodiment of the present invention.

[0037] Figure 6 is a diagram showing an example of a region-specific color extraction UI / UX of the present invention.

[0038] FIG. 7 is a flowchart of a color measurement algorithm including object detection according to an embodiment of the present invention.

[0039] FIG. 8 is a flowchart of a color measurement algorithm including object detection according to an embodiment of the present invention.

[0040] FIG. 9 is a flowchart of a color measurement algorithm in which marker detection is performed according to an embodiment of the present invention.

[0041] FIG. 10 is a block diagram showing a system configuration diagram for the application of the present invention in the dental field.

[0042] Figure 11 is a flowchart of the process for measuring tooth color according to the present invention.

[0043] FIG. 12 visually illustrates an image processing process according to an embodiment of the present invention.

[0044] FIG. 13 is a block diagram of a device according to one embodiment of the present invention.

[0045] Hereinafter, various embodiments of the present invention are described with reference to the accompanying drawings. The present invention is not limited to specific embodiments and should be understood to include various modifications, equivalents, and / or alternatives of the embodiments of the present invention. In connection with the description of the drawings, similar reference numerals may be used for similar components.

[0046] In this document, when a part is described as "comprising" a certain component, it refers to the presence of such feature (e.g., a component such as a numerical value, function, operation, or part), and does not exclude the existence of other components but implies that it may include additional components.

[0047] As used in this document, the expression "configured to" may be replaced, depending on the context, with, for example, "suitable for," "having the capacity to," "designed to," "adapted to," "made to," or "capable of." The term "configured to" does not necessarily mean "specifically designed to."

[0048] Terms such as "part," "unit," "module," and "block" described in the specification refer to a unit that processes at least one function or operation, and this may be implemented in hardware, software, or a combination of hardware and software.

[0049] The terms used in this document are used merely to describe specific embodiments and are not intended to limit the scope of other embodiments. Singular expressions may include plural expressions unless the context clearly indicates otherwise. Terms used herein, including technical or scientific terms, may have the same meaning as generally understood by those skilled in the art described in this document. Terms used in this document that are defined in general dictionaries may be interpreted as having the same or similar meaning as they have in the context of the relevant technology, and are not to be interpreted in an ideal or overly formal sense unless explicitly defined in this document. In some cases, even terms defined in this document may not be interpreted to exclude the embodiments of this document.

[0050]

[0051] FIG. 1 is a schematic block diagram showing the overall system configuration of the present invention.

[0052] A color measurement system according to one embodiment of the present invention may largely include a physical medium (100), an image acquisition device (200), a computation execution device (300), and a display (400).

[0053] The physical medium (100) includes a plurality of reference colors (110), which serve as a reference for color correction of a target object (e.g., teeth, facial skin, etc.). The physical medium (100) is described in detail in FIG. 2, which will be described later.

[0054] The image acquisition device (200) serves to acquire image data containing the physical medium (100) and the target object together. The image acquisition device (200) may be a device including various types of image sensors, such as a user device (e.g., a smartphone), a digital camera, or a professional colorimeter camera.

[0055] The computation execution device (300) performs a computation to derive an accurate color value of a target object by performing a color measurement algorithm based on acquired image data. The computation execution device (300) may be included in a user device (e.g., smartphone, tablet, PC, etc.), a server, or a specific measurement device (e.g., a specific medical device), etc.

[0056] The display (400) can visually provide the acquired color value to the user. According to one embodiment, the display (400) can output an image in which the acquired color value is displayed, such that the image includes a target object.

[0057] In addition to the above, a data transmission and reception unit, etc., which are not shown in FIG. 1, may be included in the system according to an embodiment of the present invention. For example, the data transmission and reception unit transmits image data from an image acquisition device (200) to a computation execution device (300) and displays or transmits a color value derived from the computation execution device (300) to the outside.

[0058] In certain embodiments, some of the components shown in FIG. 1 may be integrated into a single device. For example, an image acquisition device (200) and a computation device (300) may be provided as a single integrated device. For example, a smartphone may implement the color measurement system of the present invention as an integrated unit by including both a camera (image acquisition device) and an application (computation device).

[0059]

[0060] FIG. 2 is a schematic plan view showing the physical medium configuration of the present invention.

[0061] According to one embodiment, a physical medium (100) may have a plurality of reference color regions (110). The plurality of reference color regions (110) may be referred to as a first region (110-1), a second region (110-2), and an nth region (110-n). Each of the reference color regions (110) may be filled with different colors used for color measurement. Additionally, the reference color regions (110) may be arranged in an N x M matrix structure, which allows for the efficient arrangement of a variety of color samples.

[0062] For example, the reference color area (110) can be configured in various forms such as 1x5, 5x5, 10x1, etc. The plurality of reference color areas (110) can be arranged such that each area is spaced apart from each other. At least some of the remaining areas in the physical medium (100), excluding the reference color area (110), may be referred to as background areas (120). Accordingly, the spaces between the reference color areas (110) can be filled with colors that constitute the background areas (120). This arrangement contributes to clearly distinguishing each color area during image processing and minimizing interference between adjacent color areas.

[0063] According to one embodiment, the physical medium (100) may have a long rectangular shape. However, it is not limited thereto, and the physical medium may have various shapes that are easy to photograph together with the target object.

[0064] The above physical medium (100) can be manufactured in various physical forms, such as paper, plastic cards, or films. Meanwhile, the material filled in the plurality of reference color areas (110) can be made of a material having optical properties similar to those of a target object.

[0065] For example, in a tooth color measuring device for measuring tooth color, the material filled in the plurality of reference color areas (110) may be made of a material similar to a tooth. For example, the plurality of reference color areas (110) may be made of materials such as zirconia, lithium disilicate, porcelain fused metal (PFM), ceramic, hybrid ceramic, resin, and feldspathic porcelain. However, the scope of the present invention is not limited to the materials mentioned above, and the reference color areas (110) may be made of various materials that exhibit a surface similar to a tooth, in addition to those mentioned above. This contributes to measuring and correcting the actual tooth color more accurately by going beyond simply reproducing color and even mimicking optical properties such as light reflectivity and transmittance.

[0066] In various embodiments of the present invention, the reference color area (110) may be filled with a color different from that of the background area (120) and may also be provided with a different material. Additionally, the reference color area (110) may be referred to by various other names depending on its shape. For example, the reference color area (110) may be referred to as a color chip.

[0067] According to one embodiment, the reference color area (110) may have a color value different from that of the background area (120). Additionally, the color of the reference color area (110) may follow a specific color standard or be configured using a unique color chart optimized for the present invention. The exact color value of each of the colors included in the reference color area (110) may be stored in advance in the memory of the computation device (200).

[0068] Meanwhile, the color range of the plurality of reference color areas (110) used in the present invention may be limited to a partial color range. Due to this limitation of the color range, the physical medium (100) according to the embodiment of the present invention can significantly increase the accuracy of the measurement value even in the same measurement environment compared to existing color guides, etc.

[0069] According to one embodiment, colors in high-luminosity and low-luminosity regions may be excluded from the color range of a plurality of reference color regions (110). In particular, any color space such as RGB, HSV, or Lab may have high uncertainty, such as colors close to white or black, excluded from the plurality of reference color regions, thereby reducing the error rate of the computation device (300).

[0070] Assuming that the entire color gamut has a range of #000000 to #FFFFFF, the above-mentioned specific color gamut can have values ​​within a range of 75% based on RGB standards. More specifically, it can have color values ​​within a color range of 25%. Even more specifically, by limiting it to within 16%, the accuracy of the measurement result value can be improved.

[0071] Assuming the entire color range is within the HSV range of 255*255*255, the above-mentioned certain color range may be limited to a range within 16% of the entire color range with a hue (H) of 100 and a saturation (S) of 100.

[0072] In particular, the color range filled in the reference color area (100) may not include at least one of white or black. More specifically, the color range may not include both white and black. White may represent a color corresponding to a white series color code, and black may represent a color corresponding to a black series code.

[0073] Assuming that RGB(Red, Green, Blue) color codes are applied, the reference color area (100) may include colors excluding RGB(0, 0, 0) and RGB(255, 255, 255). Furthermore, the R, G, B values ​​may not include a range between 0 and 20 or 30, and the R, G, B values ​​may not include a range between 220 or 230 and 255.

[0074] Assuming the entire color space is 255*255*255 based on HSL standards, the above-mentioned color space is limited to a hue (H) of 100 and a saturation (S) of 100, and the color range of the standard color space (110) may have a value within 16% of the entire color space. Here, the entire color space may refer to all color codes defined within a specific color system (e.g., RGB).

[0075] In various embodiments of the present invention, various color codes such as RGB (Red, Green, Blue), HEX (Hexadecimal), HSL (Hue, Saturation, Lightness), and RGBA (Red, Green, Blue, Alpha) may be applied to various embodiments of the present invention.

[0076] Meanwhile, although not shown in FIG. 2, the physical medium (100) may further include a marker (not shown). The marker may be placed at a specific location on the physical medium (100) and utilized to easily and quickly recognize the physical medium on an image. Furthermore, it may be utilized to obtain the location of a reference color area (110). The marker may be placed on the same surface as the one surface on the physical medium (100) where the reference color area (110) is placed. Alternatively, it may be placed on another surface of the physical medium (100).

[0077]

[0078] FIG. 3 is a schematic block diagram showing an example of an image in which a physical medium and a target object are included together in the present invention.

[0079] Referring to FIG. 3, the image acquisition device (200) can acquire or capture an image in which a physical medium (100) and a target object (10) are simultaneously included within one frame. At this time, it is preferable that the physical medium (100) be positioned adjacent to the target object (10). For example, if both upper and lower teeth are the target object (10), the physical medium (100) can be positioned between the teeth corresponding to the upper jaw and the teeth corresponding to the lower jaw to be captured. If only the upper jaw is targeted, the physical medium (100) can be positioned to overlap with the lower jaw to acquire or capture an image.

[0080] When a target object (10) is separated into two or more regions, such as the upper or lower jaw, the present invention can recognize and process each of these separated regions as a separate object. For example, in a tooth image, the upper and lower dentitions can be detected as separate objects to obtain color values.

[0081]

[0082] A color measurement method according to various embodiments of the present invention is disclosed below. The color measurement method according to the present invention can be implemented with various algorithmic flows and is not limited to a specific order. The order of some steps may be changed, and it may be implemented to enable parallel processing.

[0083] FIG. 4 is a flowchart of a color measurement algorithm according to one embodiment of the present invention.

[0084] In operation S410, the operation performing device (200) acquires image data corresponding to an image including a physical medium (100) containing a plurality of reference colors (110) and at least one target object (10). The image acquisition device (200) acquires an image including the physical medium (100) and at least one target object (10), and transmits the image data corresponding to the image to the operation performing device (200).

[0085] In operation S420, the operation performing device (200) extracts or detects at least one target object (10) from the acquired image data. This can be performed using object detection technology. For image data containing multiple objects, the operation performing device (200) can detect multiple target objects. In this operation, the operation performing device (200) can identify the boundaries of the target object (10). Furthermore, the operation performing device (200) can acquire location information of the target object (10).

[0086] In operation S430, the operation performing device (200) can detect a physical medium in the acquired image data. The operation performing device (200) can acquire a color value corresponding to a reference color area included in the physical medium.

[0087] In operation S440, the operation performing device (200) can obtain the color value of each of at least one extracted target object (10) using a reference color area.

[0088] For example, if the target object is a tooth, the computation device (200) can acquire a plurality of tooth images, detect each of the plurality of teeth, and acquire a color value for each of the plurality of teeth. In this process, the computation device (200) can acquire a color value for each of the plurality of teeth by using a color value corresponding to a reference color area included in the physical medium (100).

[0089] The color value corresponding to the above reference color area may include at least one of a measured color value actually measured for each of the reference color areas included in the physical medium or a reference color value stored in advance for each of the above reference color areas.

[0090]

[0091] FIG. 5 is a flowchart of a color measurement algorithm according to one embodiment of the present invention.

[0092] In operation S510, the operation performing device (100) acquires image data in the same way as operation S410 of FIG. 4.

[0093] In operation S520, a physical medium (100) is detected from the acquired image data. This operation may include the operation of the operation performing device (100) detecting or acquiring a reference color area (110).

[0094] In operation S530, the operation performing device (100) can perform color calibration based on the reference color area (110) included in the image data.

[0095] The operation execution device (100) can obtain a measured color value for a reference color area (110) obtained from image data and perform color correction of the image data based on the measured color value. The operation execution device (100) can perform color correction of the image data using the measured color value and a reference color value stored in advance for the reference color area (110).

[0096] According to one embodiment, a computation device (100) can obtain a color-corrected image by performing color correction so that a color corresponding to a reference color area on the acquired image data becomes a true value. The statement that a color corresponding to a reference color area becomes a true value may mean that the reference color area on the acquired image data matches a value of an actual reference color stored in advance or becomes a value within an error range according to the correction. Color correction may be performed on the entire acquired image data or on a part thereof. For example, color correction may include an operation of correcting at least a part of the acquired image data until a color corresponding to the reference color area becomes a true value. Here, at least a part of the image data may include an area corresponding to the reference color area and the target object.

[0097] This correction improves the color accuracy of the image data by correcting color distortion caused by the image acquisition environment (e.g., lighting, camera settings). In this process, the color value result can be significantly improved due to the characteristics of the physical medium (100) itself.

[0098] In operation S540, the operation performing device (200) can detect or acquire at least one target object (10) from color-corrected image data. The operation of detecting or acquiring at least one target object (10) can be performed after color correction, and the target object (10) can be identified in more accurate image data.

[0099] In operation S550, at least some of the color values ​​of the detected target objects (10) can be obtained using color-corrected data.

[0100] According to one embodiment, the operation of acquiring a target object may include the operation of acquiring at least one region for each of at least one target object (10). In this case, the operation of acquiring a color value of at least some of the target objects (10) may include the operation of acquiring a color value for each region included in each of the target objects (10).

[0101] Operations performed according to various embodiments of the present invention may be performed based on an Artificial Intelligence (AI) model. For example, an operation to correct the color of image data so that the color becomes a true value may be performed based on the AI ​​model. For example, a computation device (300) according to an embodiment of the present invention may further include an AI module that performs AI-based computations. According to one embodiment, the AI ​​module may correct the image data by using a previously stored reference color value and image data as input data so that the reference color area of ​​the image data becomes a true value regarding the reference color.

[0102]

[0103] Figure 6 is a diagram showing an example of a region-specific color extraction UI / UX of the present invention.

[0104] The present invention can provide the color value to the user visually by extracting the color value for each specific area within each target object (10), as well as the color value of an individual target object among a plurality of target objects (10). For example, when a tooth is used as a target object (10), a plurality of tooth images are obtained, and a single tooth itself is divided into multiple areas, and the color value of each area is measured and the corresponding color value is provided to the user through a display (400).

[0105] For example, an image of a target object (e.g., a tooth) is displayed in the center of the screen displayed through the display (400), and boundary lines divided into each area can be visually displayed over the image of the tooth.

[0106] In each divided area, the color value measured in that area may be displayed as text, or a small color block filled with that color may be displayed together. When a user touches or clicks a specific area, more detailed color information (e.g., RGB values) of that area may be displayed in a pop-up form or in a separate information area. Alternatively, the measured color value may be provided as text over the small color block. Furthermore, additional information generated based on the color value may be provided through the display (400). For example, in the case of teeth, optimal color matching information required for fabricating a prosthesis may be provided.

[0107] A User Interface (UI) can be provided to store measured color values ​​or color values ​​by region for future use, or, in the case of teeth, to enable easy sharing with dental laboratories or other user devices. Such UI / UX can contribute to dentists or dental technicians identifying a patient's tooth color more accurately and intuitively, and to achieving patient-satisfying results by reflecting even subtle color differences during the fabrication of prosthetics.

[0108]

[0109] FIG. 7 is a flowchart of a color measurement algorithm including object detection according to an embodiment of the present invention.

[0110] In operation S710, the operation performing device (300) acquires image data from the image acquisition device (200).

[0111] In operation S720, the operation performing device (300) performs object detection. The operation performing device (300) detects a target object (10) from an input image through object detection. According to one embodiment, operation S720 can be performed in parallel with operations such as physical media detection and color correction. This process can be performed independently or in parallel with the object detection operation S720, thereby improving the overall processing speed.

[0112] In operation S731, the operation performing device (300) can detect a physical medium (100) from the acquired image data. The operation of detecting the physical medium (100) may include the operation of detecting a reference color area (110).

[0113] Here, detecting a reference color region (110) may include detecting each reference color region (110-1, 110-2, 110-3, 110-4, 110-n). According to one embodiment, operation S731 may include a physical medium segmentation (or reference color segmentation) operation that identifies and separates an accurate region of the reference color region within image data. The region identified here may be the entire or a part of the reference color region.

[0114] According to one embodiment, the operation of performing color correction may apply a color transformation method using a lookup table. For example, precise color correction can be performed by performing color transformation in the Lab color space.

[0115] In operation S733, the operation performing device (300) performs color calibration, and in operation S735, the color-calibrated image can be obtained. The color-calibrated image may be a color-calibrated image for the entire image data obtained. The operation performing device (300) may perform color calibration on the image using a reference color value (or actual color value) that is stored in advance for the reference color area (110). A linear or non-linear method may be applied for color calibration. Additionally, in this operation, color calibration for the reference color area and color calibration for the object or the entire image may be applied together.

[0116] According to one embodiment, the operation performing device (300) can measure a color value for each reference color area (110). The color value obtained in this process may be referred to as the measured color value. The operation performing device (300) can perform color correction on the image using the measured color value and a reference color value (or actual color value) that is stored in advance for the reference color area (110). The operation performing device (300) can perform color correction on the image in a direction such that the color value of the reference color area (110) included in the image obtained in operation S710 approaches the previously stored reference color value.

[0117] In operation S735, the operation performing device (300) may obtain a color-corrected image as a result of the correction performance. In this case, the color value of the reference color area on the color-corrected image may have a color that is the same as or within an error range of the previously stored reference color. Additionally, at least one target object on the color-corrected image may also have a color corrected according to the color correction.

[0118] In operation S740, the operation performing device (300) can obtain a color value of at least one target object (10) using the detected target object (10) and the acquired corrected image. If the acquired image includes multiple target objects, the color value of each target object can be obtained.

[0119]

[0120] FIG. 8 is a flowchart of a color measurement algorithm including object detection according to an embodiment of the present invention.

[0121] In operation S810, the operation performing device (300) acquires image data, and in operation S821, can detect a target object (10) from the acquired image data. If the image data contains a plurality of target objects, each of the plurality of target objects can be detected. The operation of detecting a target object may further include an operation of detecting an empty area within the image data.

[0122] The operation performing device (300) can segment each detected target object in operation S823. The operation performing device (300) performs a process of segmenting color-specific regions for each target object. This operation may be referred to as color segmentation. This may be associated with obtaining color values ​​for various regions of the target object (10). Through the segmentation of the target object, the operation performing device (300) can obtain target segmentation information.

[0123] Target object detection and segmentation operations can be performed in parallel or independently with the physical media detection operations of operations S831 to S835.

[0124] In operation S831, the operation performing device (300) detects a physical medium (100), and in operation S833, can perform color correction using a reference color area (110) included in the detected physical medium (100). The operation performing device (300) can perform color correction of the image data such that the color of the identified reference color area (110) of the acquired image data becomes a true value. Refer to the description in FIG. 7 for this. In operation S835, the operation performing device (300) finally acquires a color-corrected image.

[0125] In operation S840, the color value of a specific region can be obtained using the object (10) detected in operation S820, segmentation information (or region information), and the acquired corrected image. According to one embodiment, each target object may include a plurality of specific regions. Accordingly, the operation performing device (300) can individually obtain the color value of a plurality of regions included in each target object.

[0126] The parallel operation flow presented in FIGS. 7 and FIGS. 8 above is intended to improve processing speed and may be particularly advantageous when multiple images or real-time processing are required. However, each of the above operations does not necessarily need to be performed in parallel and can be sufficiently implemented with a single sequential data operation flow. That is, after physical medium (100) detection and color correction are completed first, object detection and color value acquisition are performed based on the result (corrected image), or the reverse sequential flow is also possible.

[0127]

[0128] FIG. 9 is a flowchart of a color measurement algorithm in which marker detection is performed according to an embodiment of the present invention.

[0129] In operation S910, the operation performing device (300) acquires image data from the image acquisition device (200), and in operation S920, can perform object detection. For operation S920, refer to the descriptions of operation S720 and operation S821 in FIGS. 7 and 8.

[0130] In operation S931, the operation performing device (300) can acquire or detect a marker included in the physical medium (100) from the acquired image data. This operation may be performed immediately after the operation of acquiring image data. Additionally, although it is illustrated in FIG. 9 as a sequential flow with physical medium detection, it may be performed in parallel with other major operations such as object detection or physical medium detection.

[0131] Markers can have specific patterns or take the form of codes such as barcodes or QR codes. Depending on the form of the marker, various image processing techniques (e.g., pattern recognition) can be used for marker detection.

[0132] In operation S933, the computation device (300) can detect the physical medium (100) based on the marker detection result. According to one embodiment, the physical medium (100) can be detected more quickly and accurately by utilizing the marker detection result. The computation device (300) can reduce the amount of computation required to detect the entire physical medium (100) by determining in advance the position, size, or rotation angle of the physical medium (100) or the reference color area (110) included therein through the marker. Subsequently, the computation device (300) performs color calibration based on the information related to the physical medium (100) detected in operation S935, and can obtain the corrected image in operation S937.

[0133] In operation S940, the operation performing device (300) can obtain the color value of the detected target object or the area included in the target object using a corrected image in which the color correction result is reflected. Operations S935, S937, and S940 may refer to operations S733, S735, and S740 of FIG. 7 or operations S833, S835, and S840 of FIG. 8.

[0134] Marker detection operations can significantly improve the speed of subsequent physical media detection operations. This provides a significant advantage, particularly in environments requiring real-time processing or fast response.

[0135] Meanwhile, the detection of a marker may implicitly mean the detection of a physical medium. In this case, in response to the operation of detecting a marker, the operation performing device (300) may detect a reference color area.

[0136]

[0137] FIG. 10 is a block diagram showing a system configuration diagram for the application of the present invention in the dental field.

[0138] According to one embodiment, an image acquisition device (200) acquires an image of a physical medium (100) and a customer's tooth (10). The acquired image data is transmitted to a service server (500) (e.g., a computational device (300) of FIG. 1). The service server (500) can derive an accurate color value of the tooth by performing an algorithm as illustrated in FIG. 4 through FIG. 9.

[0139] The derived color value can be transmitted to an external server (600), such as a dental laboratory, and used to produce dental prosthetics, resins, crowns, etc., which can increase production precision and improve patient satisfaction.

[0140] In some cases, the measured color value may be displayed directly on a user device (700) or a display (400) of a medical device within the dental clinic. In some cases, a function may be provided to divide the area of ​​each tooth into multiple regions and display the color value for each region. This provides useful information to both dentists and patients by clearly showing the color differences between parts of the tooth that the conventional single color value method could not reflect. The color value may be provided in the form of a color code (e.g., HEX code, RGB value, etc.), and in some cases, a color palette corresponding to the color code may be provided together, or color blocks may be displayed by region. Alternatively, the color may be provided in a form applied to a 3D model. This aids in visual understanding and enhances user convenience.

[0141] Although dental clinics are used as examples in FIGS. 10 and 11, the color measurement technology according to the embodiments of the present invention can be applied to various fields. For example, in addition to the field of dentistry, the present invention can be utilized in the beauty field, including skin color measurement, for measuring eye color, lip color, etc. Furthermore, it can be extended to various fields such as measuring the color of plant leaves to measure the growth status of plants in the agricultural field, and measuring the color of clothing and shoes in the fashion field.

[0142]

[0143] Figure 11 is a flowchart of the process for measuring tooth color according to the present invention.

[0144] This process is achieved through interaction between a user device (700), a service server (500), and an external server (600). Here, the user device (700) may be a dentist's device or a medical device used in a dental clinic. Alternatively, it may be a general user device (e.g., a smartphone). The service server (500) acts as a computational execution device (300), and the external server (600) may refer to a server or device of an external service provider, such as a dental laboratory. Meanwhile, although the following examples illustrate the case where the computational execution device (300) of the present invention is implemented as a service server (500), the computational execution device (300) may also be provided integrated into the user device (700).

[0145] In operation S1101, the user device (700) acquires customer data. The customer data may include the patient's basic information (e.g., identification information), medical records, or additional information required for color measurement. This operation may be performed primarily on the user device (700). Alternatively, the customer data may be the user's personal data (e.g., identification information).

[0146] In operation S1103, the user device (700) acquires an image comprising a physical medium (100) and at least one target object (10) (e.g., a tooth). This image is captured through a camera module, etc., of the user device (700). According to one embodiment, the user device (700) may acquire the image while customer data is displayed on a screen. In this case, the customer data may be displayed on a screen displayed through the display of the user device (700) or through a display included in a device separate from the user device (700).

[0147] In operation S1105, the user device (700) transmits the acquired image data and customer data to the service server (500). The user device (700) transmits the acquired image data to the service server (500) via a wired or wireless communication network.

[0148] In operation S1107, the service server (500) obtains a color value. The service server (500) performs a color measurement algorithm according to various embodiments of the present invention on the received image data to derive an accurate color value of the target object. In this process, customer data may be utilized for color measurement and analysis.

[0149] In operation S1109, the service server (500) transmits the acquired color information to the user device (700). The service server (500) sends the analyzed color result back to the user device (700) so that it is displayed through the display (400). The color information may include the color value itself or additional information acquired based on the color value. For example, it may include message information such as "Your tooth color is OOO".

[0150] In operation S1111, the service server (500) transmits the acquired color information to an external server (600). The service server (500) can directly transmit the measured color information and related customer data to an external server (600), such as a dental laboratory or a prosthetic manufacturer, to support subsequent operations for dental-related products (e.g., prosthetics). In this case, the color information or color value may be referred to as a color value for manufacturing prosthetics.

[0151] In some cases, operation S1109 or operation S1111 may be omitted. That is, color information may be transmitted only to the user device (700) or only to the external server (600). Additionally, after sending color information to the user device (700) in operation S1109, the user device (700) may transmit the color information to the external server (600) along with an external service request, in accordance with the user's request. For example, after a dentist checks the measured color, they may transmit the corresponding color information or color value together when ordering the fabrication of a prosthesis from a dental laboratory.

[0152]

[0153] FIG. 12 visually illustrates an image processing process according to an embodiment of the present invention.

[0154] (a) In the initial image acquisition stage, the image acquisition device (200) or the computation device (300) acquires an image containing the target object and the physical medium together, which is an original image in which the actual tooth and the physical medium are captured together. Therefore, there may be some difference from the actual color depending on the ambient lighting or camera settings.

[0155] (b) The operation execution device (300) detects a reference color area included within the physical medium (100) from the acquired image. In this process, the entire reference color area or a part of the area is acquired, and the location, size, etc. of the physical medium (100) or the reference color area can be verified.

[0156] (c) The operation performing device (300) can convert and correct the acquired image color based on the detected reference color area information so that it matches the actual reference color value. In this process, color correction of the image can be performed so that there is no difference between the unique reference color value of the physical medium (100) and the captured color value of the reference color area of ​​the physical medium (100) measured in the image. Accordingly, the color of the entire image is corrected, and the color of the target object can also be adjusted to be close to reality.

[0157] (d) The operation execution device (300) can obtain the color value of the detected target object among the color-corrected images. According to one embodiment, in the case of a tooth, since one target object (tooth) may have multiple regions at the color level, the operation execution device (300) can obtain a precise color value for each region.

[0158] Each device of the present invention (e.g., image acquisition device (200), computation execution device (300), service server (500), user device (700), external server (600), etc.) may be implemented as a general computing device including a processor, memory, communication module, etc., and the algorithms may be implemented in the form of software, stored in memory, and executed by a processor.

[0159]

[0160] FIG. 13 is a block diagram showing the configuration of a device according to one embodiment of the present invention.

[0161] As illustrated in FIG. 13, a device (1300) according to one embodiment of the present invention (e.g., image acquisition device (200), computation execution device (300), service server (500), external server (600), or user device (700) of FIG. 1) may include a bus (1310), a display (1320), a communication circuit (1330), a database (1340), a memory (1350), an I / O interface (1360), and a processor (1370). In other embodiments, the device (1300) may omit at least one of the above components or additionally include other components.

[0162] For reference, the components (1310 to 1370) of the device (1300) illustrated in FIG. 13 are merely exemplary components for describing the device according to one embodiment of the present invention. That is, it is evident that the device (1300) according to one embodiment of the present invention may additionally include other components other than those illustrated.

[0163] The bus (1310) can electrically connect the components (1320 to 1370) to each other. The bus (1310) may include circuits for communication (e.g., control messages and / or data) between the components (1320 to 1370).

[0164] The display (1320) can display text, images, videos, icons, or symbols that constitute various content. The display (1320) may include a touchscreen and can receive touch, gesture, proximity, or hovering input using an electronic pen or a part of the user's body.

[0165] For example, the display (1320) may include a liquid crystal display (LCD), a light-emitting diode (LED) display, an organic light-emitting diode (organic LED) display, a microelectromechanical systems (MEMS) display, or an electronic paper display. The display (1320) may be implemented by being included in the device (1300), or implemented separately from the device (1300) but operatively connected to the device (1300).

[0166] The communication circuit (1330) can establish a communication channel between the device (1300) and external devices. The communication circuit (1330) can communicate with external devices by accessing the network (1380) via wireless communication or wired communication. In one embodiment, the communication circuit (1330) may include a circuit for forming a wide-area network connection or a peer-to-peer connection, such as Wi-Fi, Bluetooth, and / or a cellular communication circuit.

[0167] The database (1340) may be implemented in memory (1350) or on a separate storage medium. The database (1340) may store all contents, history, etc. of data transmitted and received with an external device. Data stored in the database (1340) may be updated regularly according to a predetermined period.

[0168] According to an embodiment of the present invention, various information received from an external device may be stored in the database (1340). For example, various client setting information may be stored in the database (1340).

[0169] According to various embodiments, data stored in the database (1340) is sensitive information of the user, and may be distributed and stored in a blockchain network to enhance security regarding the use of said information. When the database (1340) is distributed and stored in a blockchain network, the history of transmission, modification, deletion, addition, etc. of the information included in the database (1340) can be managed more securely in said blockchain network.

[0170] The memory (1350) may include volatile and / or non-volatile memory. The memory (1350) may store instructions or data related to at least one other component of the device (1300). For example, the memory (1350) may store instructions that cause the processor (1370) to perform various operations described herein at runtime. For example, the instructions may be included in a package file of an application program.

[0171] The I / O interface (1360) can perform the role of transmitting commands or data input from a user or other external device to other components of the device (1300). The I / O interface (1360) can be implemented in hardware or software and can be used as a concept encompassing a user interface (UI) and a terminal for communication with other external devices.

[0172] The processor (1370) may include at least one of a central processing unit (CPU), an application processor (AP), or a communication processor (CP). The processor (1370) is electrically connected to a memory (1350), a display (1320), and a communication circuit (1330) via a bus (1310), and during operation, it may execute operations or data processing regarding the control and / or communication of other components according to instructions, programs, or software stored in the memory (1350). Accordingly, the execution of said instructions, application programs, or software can be understood as the operation of the processor (1370).

[0173] In one embodiment, the processor (1370) acquires an image comprising a physical medium including a plurality of reference color regions spaced apart from each other and at least one target object, and can display color information related to the at least one target object through a display (1320). When the processor (1370) displays color information related to the at least one target object through the display, it can display color information for each of the plurality of regions corresponding to a specific target object among the at least one target object. Color information related to the at least one target object can be generated based on the plurality of reference color regions.

[0174] The network (1380) may include at least one of a telecommunications network, a computer network, the Internet, or a telephone network. A wireless communication protocol for accessing the network (1380) may use, for example, at least one of LTE (Long-Term Evolution), LTE-A (LTE Advanced), CDMA (Code Division Multiple Access), WCDMA (Wideband CDMA), UMTS (Universal Mobile Telecommunications System), WiBro (Wireless Broadband), GSM (Global System for Mobile communications), or 5G standard communication protocols. However, this is exemplary, and various wired and wireless communication technologies applicable in the relevant technical field may be used according to embodiments to which the present invention is applied.

[0175]

[0176] As described above, the best embodiments have been disclosed in the drawings and specification. Specific terms have been used herein, but they are used only for the purpose of describing the invention and are not intended to limit the meaning or the scope of the invention as described in the claims. Therefore, those skilled in the art will understand that various modifications and equivalent alternative embodiments are possible therefrom. Accordingly, the true technical scope of protection of the invention should be determined by the technical spirit of the appended claims.

Claims

1. In a color measurement method performed on a device, The operation of acquiring an image comprising a physical medium including a plurality of reference color regions spaced apart from each other and at least one target object together; and The method includes an operation of displaying color information related to at least one target object through a display, Each of the above plurality of reference color areas has different color values ​​within a certain color range, The above color range is characterized by the exclusion of high-luminosity or low-luminosity regions. method.

2. In Claim 1, The operation of displaying color information related to at least one target object through a display is, The operation of displaying color information for each of a plurality of regions corresponding to a specific target object among the at least one target object, method.

3. In Claim 1, The above color information is displayed as color blocks or text, method.

4. In Claim 1, The above color range excludes white and black, method.

5. In Claim 1, Color information related to at least one target object is generated based on the plurality of reference color areas, method.

6. In a device for performing a color measurement method, An image acquisition device for acquiring an image comprising a physical medium including a plurality of reference color regions spaced apart from each other and at least one target object; Communication circuit for communicating with an external server; Display; and It includes the image acquisition device, the communication circuit, and a processor connected to the display, The above processor is configured to display color information related to the at least one target object through the display, and Each of the above plurality of reference color areas has different color values ​​within a certain color range, The above color range is characterized by the exclusion of high-luminosity or low-luminosity regions. device.

7. A physical medium including multiple reference color spaces; An image acquisition device for acquiring image data by photographing the above physical medium and at least one target object; and A computational device that detects the physical medium from the image data and obtains a color value of at least one target object based on the detected physical medium, Each of the above plurality of reference color areas has different color values ​​within a certain color range, The above color range is characterized by the exclusion of high-luminosity or low-luminosity regions. Color measurement system.

8. Includes multiple reference color spaces, The plurality of reference color regions are arranged in an N x M matrix structure and positioned at a certain distance from one another, and include a background region that is distinct from the plurality of reference color regions, wherein Each of the above plurality of reference color areas has different color values ​​within a certain color range, The above color range is characterized by the exclusion of high-luminosity or low-luminosity regions. Physical media.

Images