Electronic device for generating clothes management information and method of controlling the same

Abstract

An electronic device and a method for controlling the same are provided. According to one embodiment of the present disclosure, an electronic device includes: a camera; a memory storing at least one instruction; a display; and at least one processor obtaining a tag image of the laundry through the camera by executing the at least one instruction, generating care information of the laundry by using the obtained tag image, and causing the display to display the generated care information of the laundry. The at least one processor identifies a plurality of laundry care symbols included in the tag image, identifies an erroneously identified symbol from the plurality of identified laundry care symbols based on a standard type and a care type of each of the plurality of identified laundry care symbols, and correcting the identified wrongly identified symbol based on at least one of a standard type and a care type of another normally identified symbol.

Description

Technical Field

[0001] The present disclosure relates to an electronic device for generating clothing management information and a method of controlling the electronic device. More specifically, the present disclosure relates to an electronic device for generating clothing management information based on a clothing label image and a method of controlling the electronic device. Background Art

[0002] With the development of electronic technology, various electronic devices (e.g., washing machines, dryers, etc.) provide functions for more finely managing (e.g., washing, drying, sterilizing, etc.) user clothing. However, in order to still manage the user's clothing, the user should determine the characteristics of the clothing (e.g., composition, mixing ratio, material, color, etc.) or the method of managing the clothing, and input control commands separately from each electronic device to execute functions or methods of managing the clothing suitable for the characteristics of the clothing. Specifically, the user determines the characteristics of the clothing or the method of managing the clothing based on the clothing information written on a label attached to one side of the clothing. Therefore, when the clothing information written on the clothing label is misunderstood or the clothing label is lost (or missing), it may not be possible to accurately determine the characteristics of the clothing and the method of managing the clothing. Summary of the Invention

[0003] According to an aspect of the present disclosure, an electronic device includes: a camera; at least one memory configured to store at least one instruction; a display; and at least one processor configured to execute the at least one instruction to perform the following operations: obtain a label image of clothing through the camera, and based on identifying a plurality of clothing management symbols in the label image: identify a mis-identified clothing management symbol among the plurality of clothing management symbols based on a standard type and a management type of each clothing management symbol among the plurality of clothing management symbols; correct the mis-identified clothing management symbol based on at least one of a standard type and a management type of a normally-identified symbol among the plurality of clothing management symbols; generate management information related to the clothing using the label image; and control the display to display the generated management information.

[0004] The at least one memory may store a neural network model configured to identify a plurality of clothing management symbols in the label image, and the at least one processor may also be configured to execute the at least one instruction to identify the plurality of clothing management symbols included in the label image using the neural network model.

[0005] The at least one processor may also be configured to execute the at least one instruction to perform the following operations: identify candidate symbol information of each clothing management symbol among the plurality of clothing management symbols and reliability of the identified candidate symbol information, and identify a mis-identified clothing management symbol by comparing a standard type of the identified candidate symbol information having the highest reliability value with a standard type of another identified candidate symbol information.

[0006] The at least one processor may also be configured to execute at least one instruction to perform the following operations: identify candidate symbol information of each clothing management symbol among a plurality of clothing management symbols and the reliability of the identified candidate symbol information, identify a plurality of first clothing management symbols corresponding to the identified candidate symbol information having the same management type among the plurality of clothing management symbols, and identify, as a misidentified clothing management symbol, the first clothing management symbol corresponding to the identified candidate symbol information having the lowest reliability value among the plurality of first clothing management symbols.

[0007] The at least one processor may also be configured to execute at least one instruction to perform the following operations: identify a plurality of semantic information corresponding to each of the plurality of first clothing management symbols, and based on identifying that the plurality of semantic information do not match, identify, as a misidentified clothing management symbol, the first clothing management symbol corresponding to the identified candidate symbol information having the lowest reliability value among the plurality of first clothing management symbols.

[0008] The at least one processor may also be configured to execute at least one instruction to perform the following operations: based on the plurality of clothing management symbols being arranged in a plurality of rows in the label image, repeat the operations described in claim 1 row by row.

[0009] The at least one processor may also be configured to execute at least one instruction to perform the following operations: based on identifying that the label image does not include a clothing management symbol, identify the material of the clothing based on the label image, and generate management information about the clothing based on the identified material.

[0010] The at least one memory may store a plurality of candidate symbol images corresponding to each of the plurality of candidate symbol information, and the at least one processor may also be configured to execute at least one instruction to perform the following operations: identify candidate symbol information of each clothing management symbol among a plurality of clothing management symbols and the reliability of the identified candidate symbol information; based on the identified candidate symbol information having the highest reliability value being a first type of clothing management symbol, identify the identified clothing management symbol corresponding to the identified candidate symbol information having the highest reliability value as an unrecognizable symbol; obtain a clothing management symbol image corresponding to the unrecognizable symbol based on the label image; and identify the standard type and management type of the unrecognizable symbol based on the similarity between the obtained clothing management symbol image and each of the plurality of candidate symbol images.

[0011] The at least one processor may also be configured to execute at least one instruction to perform the following operations: based on identifying that the label image does not include a clothing management symbol associated with a first management type, obtain at least one first management processor information including management information corresponding to the first management type based on the generated management information, and generate management information corresponding to the first management type based on the obtained first management process information.

[0012] The electronic device may further include: a communication interface configured to communicate with a clothing processing device, wherein the at least one processor is further configured to execute at least one instruction to perform the following operation: control the communication interface to send management information corresponding to the management type to the clothing processing device.

[0013] According to an aspect of the present disclosure, a method for controlling an electronic device includes: obtaining a label image of clothing through a camera; identifying a plurality of clothing management symbols in the label image; identifying mis-identified clothing management symbols among the plurality of clothing management symbols based on the standard type and management type of each clothing management symbol in the plurality of clothing management symbols; correcting the mis-identified clothing management symbols based on at least one of the standard type and management type of the normally identified symbols among the plurality of clothing management symbols; generating management information about the clothing using the label image; and displaying the generated management information on a display.

[0014] Identifying the plurality of clothing management symbols may include using a neural network model configured to identify the plurality of clothing management symbols in the label image.

[0015] Identifying the mis-identified clothing management symbols may further include: identifying candidate symbol information of each clothing management symbol among the plurality of clothing management symbols and the reliability of the identified candidate symbol information; and identifying the mis-identified clothing management symbol by comparing the standard type of the candidate symbol information with the highest reliability value with the standard type of another identified candidate symbol information.

[0016] Identifying the mis-identified clothing management symbols may further include: identifying candidate symbol information of each clothing management symbol among the plurality of clothing management symbols and the reliability of the identified candidate symbol information; identifying a plurality of first clothing management symbols corresponding to the candidate symbol information with the same management type among the plurality of clothing management symbols; and identifying the first clothing management symbol corresponding to the candidate symbol information with the lowest reliability value among the plurality of first clothing management symbols as the mis-identified clothing management symbol.

[0017] The misidentified clothing management symbols that can be recognized may further include: recognizing multiple semantic information items corresponding to each of the multiple first clothing management symbols; and based on recognizing that the recognized multiple semantic information items do not match, identifying as a misidentified clothing management symbol the first clothing management symbol corresponding to the candidate symbol information with the lowest reliability value among the multiple first clothing management symbols.

[0018] According to one aspect of the present disclosure, a non-transitory computer-readable medium includes instructions stored therein that, when executed by at least one processor, cause the at least one processor to perform a method of controlling an electronic device, the method including: acquiring a label image of clothing through a camera; recognizing multiple clothing management symbols in the label image; identifying misidentified clothing management symbols among the multiple clothing management symbols based on the standard type and management type of each clothing management symbol among the multiple clothing management symbols; correcting the misidentified clothing management symbols based on at least one of the standard type and management type of the correctly recognized symbols among the multiple clothing management symbols; generating management information about the clothing using the label image; and displaying the generated management information on a display.

[0019] Recognizing the multiple clothing management symbols may include using a neural network model configured to recognize the multiple clothing management symbols in the label image.

[0020] The misidentified clothing management symbols that can be recognized may further include: recognizing the candidate symbol information of each clothing management symbol among the multiple clothing management symbols and the reliability of the recognized candidate symbol information; and identifying the misidentified clothing management symbol by comparing the standard type of the recognized candidate symbol information with the highest reliability value with the standard type of another recognized candidate symbol information.

[0021] The misidentified clothing management symbols that can be recognized may further include: recognizing the candidate symbol information of each clothing management symbol among the multiple clothing management symbols and the reliability of the recognized candidate symbol information; identifying multiple first clothing management symbols corresponding to the recognized candidate symbol information with the same management type among the multiple clothing management symbols; and identifying as a misidentified clothing management symbol the first clothing management symbol corresponding to the recognized candidate symbol information with the lowest reliability among the multiple first clothing management symbols.

[0022] The misrecognized clothing management symbols that can be recognized may further include: recognizing multiple pieces of semantic information corresponding to each of the multiple first clothing management symbols; and based on recognizing that the recognized multiple pieces of semantic information do not match, recognizing the first clothing management symbol corresponding to the candidate symbol information with the lowest reliability among the multiple first clothing management symbols as a misrecognized clothing management symbol. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In the following description with reference to the accompanying drawings, the above and other aspects, features, and advantages of certain embodiments of the present disclosure will become clearer. In the drawings:

[0024] Figure 1 is an exemplary diagram of an electronic device for generating management information according to an embodiment of the present disclosure;

[0025] Figure 2 is a schematic configuration diagram of an electronic device according to an embodiment of the present disclosure;

[0026] Figure 3 is a flowchart of a method for controlling an electronic device according to an embodiment of the present disclosure;

[0027] Figure 4 is an exemplary diagram showing the management type, washing type, and semantic information of clothing management symbols;

[0028] Figure 5 is an exemplary diagram of displaying clothing management information according to an embodiment of the present disclosure;

[0029] Figure 6 is an exemplary diagram showing a method for obtaining the reliability of candidate symbol information of a clothing management symbol according to an embodiment of the present disclosure;

[0030] Figure 7 is a flowchart schematically showing a method for controlling an electronic device to verify the recognition result of a clothing management symbol according to an embodiment of the present disclosure;

[0031] Figure 8 is an exemplary diagram showing the recognition of misrecognized symbols based on a standard type according to an embodiment of the present disclosure;

[0032] Figure 9 is an exemplary diagram showing the recognition of misrecognized symbols based on a management type according to an embodiment of the present disclosure;

[0033] Figure 10 is an exemplary diagram showing a method for correcting misrecognized symbols according to an embodiment of the present disclosure;

[0034] Figure 11is an exemplary diagram showing the symbol of line-by-line recognition error recognition according to an embodiment of the present disclosure;

[0035] Figure 12 is an exemplary diagram showing generation of management information on a management type not recognized in a tag based on a management type of a clothing management symbol included in the tag according to an embodiment of the present disclosure;

[0036] Figure 13 is a sequence diagram of an electronic device implemented as a server device according to an embodiment of the present disclosure; and

[0037] Figure 14 is a detailed configuration diagram of an electronic device according to an embodiment of the present disclosure. Detailed Description of the Invention

[0038] After briefly describing the terms used in this specification, the present disclosure will be described in detail.

[0039] Considering the functions in the present disclosure, general terms currently widely used are selected as the terms used in the exemplary embodiments of the present disclosure, but may be changed according to the intention of those skilled in the art or judicial precedents, the emergence of new technologies, etc. Additionally, in certain cases, there may be terms arbitrarily selected by the applicant. In such cases, the meanings of these terms will be detailed in the corresponding description parts of the present disclosure. Therefore, the terms used in the present disclosure should be defined based on the meanings of the terms and content throughout the present disclosure rather than the simple names of the terms.

[0040] In the present disclosure, expressions such as "has", "may have", "includes", "may include", etc. indicate the existence of corresponding features (e.g., numerical values, functions, operations, components such as parts, etc.), and do not exclude the existence of additional features.

[0041] In the present disclosure, expressions such as "A or B", "at least one of A and / or B", "one or more of A or B", etc. may include all possible combinations of the items listed together. For example, "A or B", "at least one of A and B", or "at least one of A or B" may indicate all of the following cases: 1) a case including at least one A, 2) a case including at least one B, or 3) a case including both at least one A and at least one B.

[0042] Expressions such as "first", "second", "1st", or "2nd" used in the present disclosure may indicate that various components are only used to distinguish one component from other components, regardless of the order and / or importance of the components, and do not limit the corresponding components.

[0043] When it is mentioned that any component (e.g., a first component) is coupled (operatively or communicatively) to, coupled to, or connected to another component (e.g., a second component), it should be understood that any component is directly coupled to another component or may be coupled to another component through other components (e.g., a third component).

[0044] The expression “~ is configured (or set) to” used in the present disclosure may be replaced, as the case may be, with expressions (e.g., “is adapted to,” “has the ability to,” “is designed to,” “is adapted for,” “is fabricated as,” or “is capable of”). The term “~ is configured (or set) to” may not necessarily mean “specially designed” in terms of hardware.

[0045] In any case, the expression “a device configured to...” may mean that the device “is capable of” being used with other devices or components. For example, “a processor configured (or set) to execute A, B, and C” may mean a dedicated processor (e.g., an embedded processor) for executing the corresponding operations or a general-purpose processor (e.g., a central processing unit (CPU) or an application processor) that can execute the corresponding operations by executing one or more software programs stored in a storage device.

[0046] Unless the context clearly indicates otherwise, the singular forms are also intended to include the plural forms. It should be understood that the terms “comprises” or “has” used in this specification specify the presence of the features, numbers, steps, operations, components, parts, or combinations thereof mentioned in this specification, but do not preclude the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0047] In an embodiment, a “module” or “device” may perform at least one function or operation and may be implemented by hardware or software, or by a combination of hardware and software. In addition, except for a “module” or “device” that needs to be implemented by specific hardware, multiple “modules” or multiple “devices” may be integrated into at least one module and may be implemented by at least one processor.

[0048] Various elements and regions in the drawings are schematically illustrated. Accordingly, the spirit of the present disclosure is not limited to the relative sizes or intervals shown in the drawings.

[0049] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings.

[0050] Figure 1 is an exemplary diagram of an electronic device 100 for generating management information according to an embodiment of the present disclosure.

[0051] Refer to Figure 1, according to an embodiment of the present disclosure, the electronic device 100 generates management information regarding the clothing 10. Here, the management information may be information regarding washing, drying, sterilizing, deodorizing, etc. of the clothing 10. For example, the electronic device 100 may generate information such as the washing method, washing time, washing intensity, etc. of the clothing 10 as management information.

[0052] In this case, the electronic device 100 may generate management information according to the management type of the clothing 10. That is, the electronic device 100 may generate management information regarding the clothing 10 according to the management type, such as washing management information, drying management information, sterilizing management information, etc.

[0053] The electronic device 100 may generate management information regarding the clothing 10, and then send the management information to the external electronic device 200, or send a control signal corresponding to the management information. Here, the external electronic device 200 is an electronic device 100 that executes the function of managing the clothing 10, and may include a washing machine 210, a dryer 220, a clothing manager 230, etc. However, the present disclosure is not limited thereto, and the external electronic device 200 may further include a server device linked to the electronic device 100 to send information regarding the clothing 10 to the external electronic device 200 and the electronic device 100, and receive information regarding the clothing 10 from the external electronic device 200 and the electronic device 100.

[0054] The electronic device 100 according to an embodiment of the present disclosure may generate appropriate management information according to the characteristics of each piece of clothing 10 (such as the color and composition of the clothing 10). To this end, the electronic device 100 may generate management information based on the clothing management symbols 500 included in the label 11 of the clothing 10. The label 11 of the clothing 10 may include information regarding the management of the clothing 10 in the form of clothing management symbols 500-1, 500-2, and 500-3 (hereinafter referred to as 500). The clothing management symbol 500 is a symbol that reflects the characteristics such as the composition of the clothing 10 and represents a management method suitable for the clothing 10. Accordingly, the electronic device 100 acquires a label image 420 of the label 11 attached to the clothing 10, and identifies the characteristics of the clothing 10 based on the label 11' in the acquired image 420 and the clothing management symbol 500 included in the label 11', and then generates appropriate management information suitable for the clothing 10.

[0055] However, when the electronic device 100 misidentifies the clothing management symbol 500, it may misidentify the characteristics of the clothing 10. Accordingly, the electronic device 100 may generate inappropriate management information for the clothing 10, which may damage the clothing 10. Therefore, the electronic device 100 according to an embodiment of the present disclosure performs a process of verifying whether the recognition result of the clothing management symbol 500 recognized by the electronic device 100 is accurate. In other words, by verifying whether the recognition result of the clothing management symbol 500 is accurate, the electronic device 100 generates more accurate management information.

[0056] Specifically, the electronic device 100 may recognize the clothing management symbol 500 included in the label 11 of the clothing 10, and then verify the recognition result of the clothing management symbol 500 based on the recognition result (more specifically, the information about the clothing management symbol 500 included in the recognition result) to determine whether there is a misrecognized clothing management symbol 500. In addition, the electronic device 100 may obtain a more accurate recognition result by selecting the misrecognized clothing management symbol 500 and correcting the recognition result for the misrecognized clothing management symbol 500.

[0057] In addition, even for a management type not included in the label 11 of the clothing 10, the electronic device 100 may generate management information based on the clothing management symbol 500 included in the label 11 of the clothing 10. For example, when the label 11 of the clothing 10 does not include a clothing management symbol 500 for drying but includes a clothing management symbol 500 for washing and sterilizing, the electronic device 100 may determine a suitable drying method for the clothing 10 based on the clothing management symbol 500 for washing and sterilizing.

[0058] Specifically, the electronic device 100 may generate washing and sterilizing information about the clothing 10 based on the clothing management symbol 500 for washing and sterilizing, and then identify the characteristics of the clothing 10 based on the generated washing and sterilizing information, and infer and determine a drying method suitable for the characteristics of the clothing 10. In other words, the electronic device 100 may identify the material, type, etc. of the clothing 10 based on the washing method, washing time, washing temperature, etc., and infer a drying method suitable for the identified material, type, etc. of the clothing 10. The electronic device 100 may generate drying information (i.e., drying time and drying intensity) corresponding to the inferred drying method.

[0059] In this way, even for a drying method that the user may not be able to recognize from the label 11 of the clothing 10, the electronic device 100 may generate and provide drying information (i.e., drying time, drying intensity, etc.) to the user.

[0060] In Figure 1In [the figure], the electronic device 100 is shown as a smart phone, but the electronic device 100 according to the present disclosure is a device that generates management information about clothing 10 and may include at least one of a TV, a tablet PC, a desktop PC, or a laptop PC. In addition, the electronic device 100 may include a washing machine 210, a dryer 220, a clothing manager 230, etc. that can perform the function of managing clothing 10. For example, when the electronic device 100 is implemented as the washing machine 210, the washing machine 210 can directly recognize the clothing management symbol 500 included in the tag 11 of the clothing 10, generate washing information of the washing machine 210, and then directly perform an operation corresponding to the generated washing information.

[0061] In addition, the electronic device 100 may be implemented as a server device including an application server, a database server, etc. For example, when the electronic device 100 is implemented as a server device, the electronic device 100 can generate management information about clothing 10 based on the information about the clothing management symbol 500 included in the tag 11 obtained through a user terminal. The electronic device 100 can send the generated management information about clothing 10 to the user terminal. However, hereinafter, for the convenience of describing the present disclosure, it will be assumed that the electronic device 100 is a smart phone for description.

[0062] Hereinafter, with reference to Figures 2 to 14 Embodiments of the present disclosure will be described in more detail.

[0063] Figure 2 is a schematic configuration diagram of the electronic device 100 according to an embodiment of the present disclosure. Referring to Figure 2 , the electronic device 100 may include a camera 110, a memory 120, a display 130, and at least one processor 140.

[0064] The camera 110 captures an object (or subject) around the electronic device 100 included in the capture area to obtain an image of the object. For example, the electronic device 100 can obtain a clothing image by capturing clothing using the camera 110, or obtain a tag image by capturing a tag on the clothing. To this end, the camera 110 may be implemented as an imaging device, such as a CMOS image sensor (CIS) having a CMOS structure, a charge-coupled device (CCD) having a CCD structure, etc.

[0065] However, the camera 110 is not limited thereto, and the camera 110 may be implemented as a camera 110 module capable of capturing an object (or subject) with various resolutions. The camera 110 may be implemented as a depth camera 110 (e.g., an IR depth camera 110, etc.), a stereo camera 110, an RGB camera 110, etc.

[0066] The memory 120 may store at least one instruction. In addition, the memory 120 may store an operating system (O / S) for driving the electronic device 100. In addition, the memory 120 may store software programs or applications for operating the electronic device 100 according to various embodiments of the present disclosure. In addition, the memory 120 may store various types of information, such as various types of data input, set, or generated during the execution of a program or application.

[0067] The display 130 may display various types of visual information. For example, the processor may display clothing management information through the display 130. In addition, the display 130 may display various information related to clothing, such as the acquired tag image.

[0068] To this end, the display 130 may be implemented by various types of displays 130, such as a liquid crystal display (LCD), an organic light emitting diode (OLED) display 130, a quantum dot light emitting diode (QLED) display 130, a plasma display panel (PDP), etc. A driving circuit, a backlight unit, etc. may be included in the display 130, which may be implemented in forms such as a thin film transistor (TFT), a low temperature polycrystalline silicon (LTPS) TFT, an organic TFT (OTFT), etc. The display 130 may be implemented as a flexible display, a 3D display, etc.

[0069] The display 130 may be implemented together with a touch screen and a touch panel. In this case, the display 130 may be used as an output unit for outputting information between the electronic device 100 and the user, and at the same time, may also be used as an input unit for providing an input interface between the electronic device 100 and the user.

[0070] One or more processors 140 are electrically connected to the camera 110, the memory 120, and the display 130, and execute at least one instruction stored in the memory 120 to control the overall operation and functions of the electronic device 100.

[0071] One or more processors 140 may include one or more of a central processing unit (CPU), a graphics processing unit (GPU), an accelerated processing unit (APU), a many integrated core (MIC), a digital signal processor (DSP), a neural processing unit (NPU), a hardware accelerator, or a machine learning accelerator. One or more processors 140 may control one or any combination of other components of the electronic device 100, and may perform operations related to communication or data processing. One or more processors 140 may execute one or more programs or instructions stored in the memory 120. For example, one or more processors 140 may execute a method according to an embodiment of the present disclosure by executing one or more instructions stored in the memory 120.

[0072] When the method according to an embodiment of the present disclosure includes multiple operations, the multiple operations may be executed by one processor or by multiple processors. For example, when a first operation, a second operation, and a third operation are executed by the method according to an embodiment, the first operation, the second operation, and the third operation may all be executed by a first processor, the first operation and the second operation may be executed by a first processor (e.g., a general-purpose processor), and the third operation may be executed by a second processor (e.g., a processor dedicated to artificial intelligence).

[0073] One or more processors 140 may be implemented as a single-core processor including one core or one or more multi-core processors including multiple cores (e.g., homogeneous multi-core or heterogeneous multi-core). When one or more processors 140 are implemented as a multi-core processor, each of the multiple cores included in the multi-core processor may include an internal memory 120 of the processor 140 (such as a cache memory 120 and an on-chip memory 120) and a common cache shared by the multiple cores that may be included in the multi-core processor. In addition, each of the multiple cores (or some of the multiple cores) included in the multi-core processor may read and execute program instructions for implementing the method according to an embodiment of the present disclosure, and all (or part) of the multiple cores may be linked to read and execute program instructions for implementing the method according to an embodiment of the present disclosure.

[0074] When the method according to an embodiment of the present disclosure includes multiple operations, the multiple operations may be executed by one of the multiple cores included in the multi-core processor or may be executed by multiple cores. For example, when a first operation, a second operation, and a third operation are executed by the method according to an embodiment, the first operation, the second operation, and the third operation may all be executed by a first processor in the multi-core processor, the first operation and the second operation may be executed by a first core included in the multi-core processor, and the third operation may be executed by a second core included in the multi-core processor.

[0075] In an embodiment of the present disclosure, the processor 140 may be a system-on-chip (SoC) integrating one or more processors and other electronic components, a single-core processor, a multi-core processor, or a core included in a single-core processor or a multi-core processor. Here, the core may be implemented as a CPU, GPU, APU, MIC, DSP, NPU, hardware accelerator, machine learning accelerator, etc., but the embodiments of the present disclosure are not limited thereto.

[0076] Hereinafter, for convenience of description, one or more processors 140 are referred to as the processor 140.

[0077] Figure 3 is a flowchart of a method for controlling the electronic device 100 according to an embodiment of the present disclosure.

[0078] Refer to Figure 3, the processor 140 obtains a label image 420 of the clothing 10 through the camera 110 (S310). Here, the label 11 can be paper, cloth, etc., which includes the price of the product 10, the brand name and product name, the product number of the clothing 10, the characteristics of the clothing 10 (e.g., composition, color, material, mixing ratio, etc.), the management information of the clothing 10, and other information of the clothing 10.

[0079] The processor 140 can control the camera 110 to obtain an image of the label 11 (i.e., the label image 420). In this case, the processor 140 can display a UI for guiding the capture of the label on the display 130 so that the user can obtain an accurate label image 420. Here, the position of the UI can be set on the display 130 based on the size of the label image 420 to be obtained. In addition, the processor 140 can identify the resolution, size, etc. of the obtained label image 420, and request the user to re-obtain the label image 420. Specifically, the processor 140 can display a message requesting the re-obtaining of the label image 420 on the display 130, or output the message in the form of voice through the output interface.

[0080] The processor 140 can obtain the label image 420 (S310), and then generate management information about the clothing 10 using the obtained label image 420 (S320).

[0081] Specifically, the processor 140 can obtain the information about the clothing 10 included in the label 11 based on the label image 420. For example, based on the label image 420, the processor 140 can obtain the information about the clothing 10 written on the label 11, such as the composition, material, color, and mixing ratio of the clothing 10. In addition, the processor 140 can generate appropriate management information suitable for the clothing 10 based on the obtained information.

[0082] For example, the processor 140 can identify the characteristics of the clothing 10 based on the label image 420, or identify the washing method of the clothing 10 described in the label image 420 to determine the washing method of the clothing 10. In addition, the processor 140 can generate washing information about the clothing 10 based on the determined washing method. Here, the washing information can include text, images, etc. indicating the washing method of the clothing 10. In addition, the processor 140 can generate various management information related to the clothing 10, such as drying, deodorizing, and sterilizing, based on the obtained information about the clothing 10.

[0083] In this case, the processor 140 may generate management information regarding the clothing 10 based on the clothing management symbol 500 included in the tag 11 of the clothing 10. Specifically, the processor 140 may identify the clothing management symbol 500 in the tag image 420, and then identify the method of managing the clothing 10 described on the tag 11 based on the identified clothing management symbol 500, such as the washing method and drying method of the clothing 10. In addition, the processor 140 may generate management information regarding the clothing 10 based on the method of managing the clothing 10 identified according to the clothing management symbol 500.

[0084] To this end, the processor 140 may first identify the clothing management symbol 500 within the tag image 420. Specifically, the processor 140 may perform an object recognition process on each clothing management symbol 500 included in the tag image 420. Here, the object recognition process for each clothing management symbol 500 may be a process of identifying the clothing management symbol 500 included in the tag image 420 to identify the meaning of the clothing management symbol 500 (e.g., context information) and the management type and standard type of the clothing management symbol 500. For example, the processor 140 may identify the clothing management symbol 500 within the tag image 420 based on an object recognition algorithm.

[0085] The management type may be a type indicating the method of managing the clothing 10, such as the washing method or drying method of the clothing 10. For example, washing, drying, dry cleaning, ironing, etc. of the clothing 10 may be included in the management type. That is, the processor 140 may identify the management type of each clothing management symbol 500 by determining whether each clothing management symbol 500 is related to water washing or dry cleaning. However, it is not limited thereto, and the management type may be further subdivided or divided into various forms. For example, the management type may be further subdivided or divided according to the type of information (such as whether washing is related to hand washing or the temperature of the washing water).

[0086] In addition, the standard type may be an international standard type including the clothing management symbol 500, or an organizational type that defines and is responsible for the international standard type including the washing symbol. For example, each clothing management symbol 500 may be classified into types such as ASTM standard type, ISO standard type, KS standard type, etc. according to the international standard type. In this case, since each country uses a clothing management symbol 500 of one international standard type, the standard type may be the country type using the clothing management symbol 500. The processor 140 may identify the standard type of each clothing management symbol 500 by determining the international standard type symbol corresponding to each clothing management symbol 500.

[0087] The semantic information may be the information represented by each clothing management symbol 500. Even if there are multiple clothing management symbols 500 belonging to the same management type, the information represented by each clothing management symbol 500 may be different. For example, even in the case where multiple clothing management symbols 500 are included in the same washing type, a specific clothing management symbol 500 may be a symbol indicating hand washing, and another specific clothing management symbol 500 may be a symbol indicating the washing temperature. The semantic information may be referred to as context information, content information, etc. The semantic information may be identified based on at least one context information corresponding to each clothing management symbol 500. For example, the semantic information may be identified as a combination of multiple contexts corresponding to (or matching) the clothing management symbol 500.

[0088] Figure 4 is an exemplary diagram showing the management type, washing type, and semantic information of the clothing management symbol 500.

[0089] Refer to Figure 4 , the management types of the first clothing management symbol 511', the second clothing management symbol 512', the third clothing management symbol 513', and the fourth clothing management symbol 514' are washing, while the management types of the fifth clothing management symbol 515' and the sixth clothing management symbol 516' are drying, and thus, the management types are different from each other. In addition, the first clothing management symbol 511', the second clothing management symbol 512', the third clothing management symbol 513', and the fourth clothing management symbol 514' have the same management type (i.e., washing). However, the standard types of the first clothing management symbol 511', the second clothing management symbol 512', and the third clothing management symbol 513' may be the KS standard, while the standard type of the fourth clothing management symbol 514' may be the ASTM standard, and thus, the standard types are different from each other. In addition, although corresponding to the same management type (i.e., washing) and the same standard type (i.e., KS standard), the first clothing management symbol 511' indicates that it is recommended to wash in water at 40 °C, the second clothing management symbol 512' indicates that it is recommended to wash in water at 60 °C, and the third clothing management symbol 513' indicates that it is recommended to wash by hand in water at 30 °C, and thus, the semantic information of each of the first clothing management symbol 511', the second clothing management symbol 512', and the third clothing management symbol 513' may be different.

[0090] Even clothing management symbols of different standard types may have the same semantic information. Refer to Figure 4 , the first clothing management symbol 511' and the fourth clothing management symbol 514' correspond to different standard types (specifically, the first clothing management symbol 511' is of the KS standard type, and the fourth clothing management symbol 514' is of the ASTM standard type), but have the same semantic information, that is, it is recommended to wash in water at 40 °C.

[0091] The processor 140 can identify the management type, standard type, and semantic information of each clothing management symbol 500, and then identify the washing method corresponding to each clothing management symbol 500 (or corresponding to each management type). In addition, the processor 140 can generate management information corresponding to each management type based on the identified washing method.

[0092] According to an embodiment of the present disclosure, the processor 140 can input the label image 420 obtained by the camera 110 into a neural network model to obtain information on a plurality of clothing management symbols 500 included in the label image 420.

[0093] To this end, the memory 120 can store a neural network model trained to identify a plurality of clothing management symbols 500 included in an input image. For example, the neural network model can be a neural network model trained to identify a plurality of clothing management symbols 500 included in an input image (i.e., the label image 420) and to identify the standard type and management type of the plurality of identified clothing management symbols 500. In addition, the neural network model can be pre-trained based on training data including a plurality of label images 420 and standard type information and management type information of each clothing management symbol among the plurality of clothing management symbols 500 included in each label image 420, and then stored in the memory 120. For example, the neural network model trained to identify a plurality of clothing management symbols 500 included in the input image 420 can be implemented as a convolutional neural network (CNN) model, a fully convolutional network (FCN) model, a region with convolutional neural network features (RCNN) model, a YOLO model, etc.

[0094] The processor 140 can input the label image 420 obtained by the camera 110 into the neural network model to obtain information on a plurality of clothing management symbols 500 included in the label image 420 as a result value. In this case, the information obtained by the processor 140 can include the standard type information and management type information of each clothing management symbol 500, and can include the position information (e.g., coordinate values of each clothing management symbol 500) of each clothing management symbol 500 within the label image 420.

[0095] The processor 140 can generate management information (S320), and then control the display 130 to display the generated management information (S330). That is, the processor 140 can display the generated management information on the display 130. In this way, the user can identify the method of managing the clothing 10 with the attached label image 420 (or corresponding to the label image 420).

[0096] Figure 5It is an exemplary diagram showing the management information of the display clothing 10 according to an embodiment of the present disclosure.

[0097] Referring to Figure 5 , the processor 140 can display the management information of the clothing 10 generated based on the clothing management symbol 500 in the label image 420. Specifically, the processor 140 can display the management information on the display 130, and the management information includes the label image 420, each clothing management symbol image 510”, 520” and 530”, and the text indicating the management method corresponding to each clothing management symbol 500.

[0098] Referring to Figure 5 , the processor 140 displays the label image 420 and the clothing image 410 with the attached label image 420 on the display 130 together, so that the user can identify the accurate management information about each clothing 10.

[0099] To this end, the processor 140 can execute a process of registering information about the user's clothing 10. Specifically, the processor 140 can simultaneously obtain the user's clothing image 410 and the label image 420 corresponding to the clothing 10 through the camera 110. In this case, the processor 140 can match the clothing image 410 and the label image 420, and store the matched clothing image 410 and label image 420 in the memory 120. In addition, the processor 140 can obtain clothing 10 information (type of clothing 10, brand name, product name, etc.) through the UI displayed on the display 130. Alternatively, the processor 140 can identify the clothing 10 information based on the obtained clothing image 410 and label image 420. In this case, the processor 140 can match the clothing image 410, the label image 420, and the obtained clothing 10 information, and store the matched clothing image 410, label image 420, and clothing 10 information in the memory 120. In addition, the processor 140 can match the clothing 10 management information generated based on the label image 420 with the user's clothing image 410 and label image 420, and store the matched clothing 10 management information, the user's clothing image 410, and the label image 420 in the memory 120. In this way, the user can effectively manage the user's clothing 10 by storing the user's clothing 10 information in the electronic device 100 and receiving appropriate management information about the user's clothing 10.

[0100] Figure 6 It is an exemplary diagram showing a method for obtaining the reliability of the candidate symbol information of the clothing management symbol 500 according to an embodiment of the present disclosure.

[0101] According to an embodiment of the present disclosure, the processor 140 may identify multiple candidate symbols for each clothing management symbol 500 and identify any one of the multiple candidate symbols as the clothing management symbol 500. Here, the candidate symbol may be a symbol that can be identified (or estimated) as the clothing management symbol 500, and information about the multiple candidate symbols may be stored in the memory 120. In this case, the processor 140 may calculate the reliability for the multiple candidate symbols corresponding to the clothing management symbol 500 and identify, based on the calculated reliability, a candidate symbol that matches the clothing management symbol 500 from among the multiple candidate symbols. In addition, the processor 140 may identify the clothing management symbol 500 based on the information of the candidate symbol that matches the clothing management symbol 500. Hereinafter, the candidate symbol that is identified as matching the clothing management symbol 500 among the multiple candidate symbols will be referred to as the target symbol.

[0102] Specifically, the processor 140 may obtain multiple pieces of candidate symbol information for each clothing management symbol 500. The processor 140 may also calculate the respective reliabilities of the multiple candidate symbols corresponding to each clothing management symbol 500. Here, the reliability may be a probability value that the clothing management symbol 500 can correspond to the multiple candidate symbols.

[0103] Specifically, when the processor 140 uses a neural network model to identify each clothing management symbol 500, the reliability may be a probability value obtained based on the activation function applied in the output layer of the neural network model. In this case, the candidate symbol may be referred to as a class. The processor 140 may obtain multiple probability values corresponding to the multiple candidate symbols that the clothing management symbol 500 may correspond to and identify each obtained probability value as the reliability of each candidate symbol. In addition, the processor 140 may identify the candidate symbol corresponding to the highest reliability among the multiple reliabilities as the clothing management symbol 500. Hereinafter, for the convenience of describing the present disclosure, it will be assumed that the candidate symbol with the highest reliability among the multiple candidate symbols is the target symbol for description.

[0104] Refer to Figure 6, the processor 140 can obtain the reliabilities of multiple candidate symbols 611, 612, 613, and 614 of the first clothing management symbol 511. In this case, the processor 140 can identify the third candidate symbol 613 corresponding to the highest reliability value among the multiple reliabilities as the target symbol. In addition, the processor 140 can identify the first clothing management symbol 511 as the third candidate symbol 613, which is the target symbol. The third candidate symbol 613 matches the first clothing management symbol 511 in that its management type is washing, its standard type is the KS standard, and its semantic information is to recommend washing at 40°C. That is, the processor 140 obtains an accurate recognition result for the first clothing management symbol 511.

[0105] On the other hand, the processor 140 obtains an inaccurate recognition result for the fifth clothing management symbol 515. Specifically, for the fifth clothing management symbol 515, the processor 140 can identify the seventh candidate symbol 617 corresponding to the highest reliability among the multiple candidate symbols 615, 616, and 617 as the target symbol. In this case, for the fifth clothing management symbol 515, the management type is drying, while the management type of the seventh candidate symbol 617 is washing. That is, the management type of the fifth clothing management symbol 515 included in the clothing label 11 is different from that of the seventh candidate management symbol 617, which is the recognition result of the fifth clothing management symbol 515 identified by the processor.

[0106] In addition, the semantic information of the seventh candidate symbol 617 involves recommending washing in water at 60°C, which is different from the semantic information of the fifth clothing management symbol 515, which is to recommend drying in the sun. That is, the processor 140 obtains an inaccurate recognition result for the fifth clothing management symbol 515. Therefore, when the processor 140 generates management information based on the fifth clothing management symbol 515 that is misidentified as the seventh candidate symbol 617, this will result in providing incorrect management information to the user. Therefore, the processor 140 performs a process of verifying the recognition result of the identified clothing management symbol 500. Hereinafter, embodiments of the present disclosure related thereto will be described in detail.

[0107] Figure 7 is a flowchart schematically showing a method for a control electronic device 100 to verify the recognition result of a clothing management symbol 500 according to an embodiment of the present disclosure.

[0108] Figure 7 The operations S710, S750, and S760 shown in Figure 3 can respectively correspond to the operations S310, S320, and S330 shown in

[0109] According to an embodiment of the present disclosure, the processor 140 may identify a plurality of clothing management symbols 500 included in the label image 420 (S720). The processor 140 may extract the plurality of clothing management symbols 500 included in the label image 420 and identify the meaning of each clothing management symbol 500, etc. Specifically, the processor 140 may identify the management type, standard type, context type, etc. of each clothing management symbol 500, identify a target symbol from a plurality of candidate symbols 611 to 617 (hereinafter referred to as 610) corresponding to each clothing management symbol 500 to identify each clothing management symbol 500, and obtain information about each clothing management symbol 500. In this regard, the embodiments of the object recognition process for each clothing management symbol 500 included in the above label image 420 may be applied in the same manner.

[0110] In addition, the processor 140 may identify mis-identified symbols from the plurality of identified clothing management symbols 500 based on the standard type and management type of each of the plurality of identified clothing management symbols 500 (S730).

[0111] Here, the mis-identified symbol may be a clothing management symbol 500 having a mis-identification result among the plurality of clothing management symbols 500 identified by the processor 140 (specifically, the plurality of clothing management symbols 500 included in the label image 420). For example, when the processor 140 identifies the clothing management symbol 500 based on the above reliability, the target symbol may be a clothing management symbol 500 that is not correctly identified.

[0112] The processor 140 may identify mis-identified symbols from the plurality of clothing management symbols 500 using the standard type and management type of each clothing management symbol among the plurality of clothing management symbols 500. As an example, the processor 140 may identify a clothing management symbol 500 having a different standard type as a mis-identified symbol based on the standard type of each clothing management symbol among the plurality of clothing management symbols 500, or may identify any clothing management symbol corresponding to the same type among the plurality of clothing management symbols 500 as a mis-identified symbol based on the management type of each clothing management symbol among the plurality of clothing management symbols 500.

[0113] Figure 8 is an exemplary diagram showing the identification of mis-identified symbols based on the standard type according to an embodiment of the present disclosure. Figure 9 is an exemplary diagram showing the identification of mis-identified symbols based on the management type according to an embodiment of the present disclosure.

[0114] In this regard, according to an embodiment of the present disclosure, the processor 140 may identify a mis-identified symbol by comparing the standard type of the candidate symbol information having the highest value among the calculated reliabilities with the standard types of other candidate symbol information. That is, the processor 140 may identify a management symbol having a different standard type among the plurality of clothing management symbols 500 as a mis-identified symbol.

[0115] To this end, the processor 140 may identify the standard types of the plurality of clothing management symbols 500. Specifically, the processor 140 may identify the standard type of each management symbol based on the reliability. Specifically, the processor 140 may identify the candidate symbol having the highest reliability value among the plurality of candidate symbols 610 corresponding to each management symbol as the target symbol. In addition, the processor 140 may identify the standard type of the identified target symbol as the standard type of the clothing management symbol 500 corresponding to the target symbol. In this case, the processor 140 may identify at least one clothing management symbol 500 having a different standard type from among the plurality of clothing management symbols 500, and identify the at least one identified clothing management symbol 500 as a mis-identified symbol.

[0116] For example, referring to Figure 8 , when the processor 140 includes four clothing management symbols 511, 512, 513, and 514 in the label image 420, the target symbols 621, 622, 623, and 624 corresponding to the four clothing management symbols 511, 512, 513, and 514 may be identified based on the reliability. In this case, when the standard types of the target symbols 622, 623, and 624 of three of the four clothing management symbols 512, 513, and 514 are KS standards, and the standard type of the target symbol 621 of the remaining one clothing management symbol 511 is ASTM standard, the processor 140 may identify the clothing management symbol 511 corresponding to the target symbol 621 of the ASTM standard as a mis-identified symbol.

[0117] In addition, according to an embodiment of the present disclosure, the processor 140 may identify a plurality of clothing management symbols 500 having candidate symbol information of the same management type, which has the highest reliability for each of the plurality of clothing management symbols 500, and identify the clothing management symbol 500 corresponding to the candidate symbol information having low reliability among the plurality of clothing management symbols 500 as a mis-identified symbol.

[0118] Specifically, when a plurality of clothing management symbols 500 corresponding to the same management type are identified from among the plurality of clothing management symbols 500, the processor 140 may identify any one of the plurality of clothing management symbols 500 as a mis-identified symbol.

[0119] To this end, the processor 140 may identify the management types of a plurality of clothing management symbols 500. In this case, the processor 140 may identify the management type of each management symbol based on reliability. Specifically, the processor 140 may identify, as the target symbol, the candidate symbol having the highest reliability value among a plurality of candidate symbols 610 corresponding to each management symbol. Then, the processor 140 may identify the management type of the identified target symbol 620 as the management type of the clothing management symbol 500 corresponding to the target symbol.

[0120] In this case, the processor 140 may identify whether there is any management symbol 500 having the same management type among the plurality of clothing management symbols 500. Then, when it is identified that there is a management symbol 500 corresponding to the same management type among the plurality of clothing management symbols 500, the processor 140 may identify any one of the plurality of clothing management symbols 500 corresponding to the same management type as the misidentified symbol.

[0121] The processor 140 may identify the reliability of each target symbol of a plurality of clothing management symbols corresponding to the same management type. In this case, the processor 140 may identify, as the misidentified symbol, the clothing management symbol corresponding to the target symbol having a low reliability value among the plurality of clothing management symbols corresponding to the same management type.

[0122] For example, referring to Figure 9 , when the processor 140 includes four clothing management symbols 511, 512, 513, and 514 in the label image 420, the target symbols 621, 622, 623, and 624 corresponding to the four clothing management symbols 511, 512, 513, and 514 may be identified based on reliability. In this case, when, among the four clothing management symbols 511, 512, 513, and 514, the management types of the target symbols of two clothing management symbols 511 and 513 correspond to drying, and the management types of the remaining two clothing management symbols 512 and 514 correspond to ironing and dry cleaning, respectively, the processor 140 may identify the two clothing management symbols 511 and 513 corresponding to drying, where drying is the same management type. Then, the processor 140 may compare the reliabilities of the target symbols 621 and 623 corresponding to the two clothing management symbols (i.e., 511 and 513) corresponding to the drying type. Then, the processor 140 may identify, as the misidentified clothing symbol, the clothing management symbol corresponding to the target symbol having a low reliability among the two clothing management symbols (i.e., 511 and 513) corresponding to the drying type.

[0123] The processor 140 may also identify multiple semantic information corresponding to multiple clothing management symbols corresponding to the same management type, and when it is identified that the multiple identified semantic information does not match, the processor 140 may identify the clothing management symbol corresponding to the candidate symbol information with low reliability among the multiple clothing management symbols corresponding to the same management type as the misidentified symbol.

[0124] Specifically, the processor 140 may determine the meanings of multiple clothing management symbols corresponding to the same management type. To this end, the processor 140 may identify the semantic information corresponding to each clothing management symbol corresponding to the same management type. Then, the processor 140 may compare each identified semantic information to identify whether each semantic information matches. That is, the processor 140 may identify whether the meanings of the multiple clothing management symbols identified as the same management type conflict based on the semantic information. In addition, when it is identified that the multiple semantic information does not match, the processor 140 may identify the meanings between the multiple clothing management symbols corresponding to the same management type as conflicting, and identify whether there is a misidentified symbol among the multiple clothing management symbols. Therefore, the processor 140 may identify the clothing management symbol corresponding to the candidate symbol with a low reliability value among the multiple clothing management symbols corresponding to the same management type as the misidentified symbol (or misidentified symbol or first type symbol).

[0125] On the other hand, when the multiple semantic information is identified as matching, the processor 140 may identify that the meanings between the multiple clothing management symbols corresponding to the same management type do not conflict. Accordingly, the processor 140 may identify all the clothing management symbols among the multiple clothing management symbols corresponding to the same management type as the correctly identified symbols (or correctly identified symbols or second type symbols).

[0126] If there are three or more clothing management symbols corresponding to the same management type, the processor 140 may compare the semantic information of the multiple clothing management symbols corresponding to the same management type, and identify the clothing management symbol with different semantic information as the misidentified symbol. That is, when the semantic information of two clothing management symbols matches and the semantic information of one clothing management symbol is different, the clothing management symbol corresponding to the different semantic information may be identified as the misidentified symbol. When the semantic information of the multiple clothing management symbols corresponding to the same management type is different, only the clothing management symbol with the highest reliability value may be identified as the correctly identified symbol, and the remaining two clothing management symbols may be identified as the misidentified symbols.

[0127] Figure 10 FIG. is an exemplary diagram showing a method of correcting a misidentified symbol according to an embodiment of the present disclosure.

[0128] Return reference Figure 7 , the processor 140 can correct the mis-recognized symbol that has been recognized (S740). For example, with reference to Figure 10 , the processor 140 can use the candidate symbol 618 with the second highest reliability value among the multiple candidate symbols 617, 618, and 619 of the clothing management symbol 511 that is recognized as a mis-recognized symbol to correct the target symbol 620 of the recognized clothing management symbol 511 that is recognized as a mis-recognized symbol. Then, the processor 140 can correct the recognition result of the clothing management symbol 511 that is recognized as a mis-recognized symbol based on the corrected target symbol 620.

[0129] The processor 140 corrects the mis-recognized symbol (S740), generates management information about the clothing 10 using the label image 420 (S750), and controls the display 130 to display the generated management information (S760). In this regard, the descriptions of S320 and S330 can be applied in the same way, and thus, their detailed descriptions will be omitted.

[0130] The processor 140 can correct the recognized mis-recognized symbol based on at least one of the standard type and management type of another normally recognized symbol. Specifically, the processor 140 can correct the recognition result of the mis-recognized clothing management symbol 500 based on at least one of the standard type and management type of the remaining normally recognized clothing management symbols 500, and exclude the mis-recognized clothing management symbol 500 from the multiple clothing management symbols 500.

[0131] For example, the processor 140 can correct the standard type of the mis-recognized clothing management symbol 500 to match the remaining normally recognized clothing management symbols 500. Specifically, when a mis-recognized symbol is recognized due to different standard types, the processor 140 can recognize multiple candidate management symbols corresponding to the clothing management symbol 500 that is recognized as a mis-recognized symbol, sort the multiple candidate management symbols in descending order of reliability value, and then, among the candidate symbols that follow the same standard type as the remaining normally recognized clothing management symbols 500, re-recognize the candidate symbol with the highest reliability value as the target symbol 620. Then, the processor 140 can correct the recognition result of the clothing management symbol 500 that is recognized as a mis-recognized symbol using the re-recognized target symbol 620.

[0132] In addition, for example, the processor 140 may correct the management type of the mis-recognized clothing management symbol 500 based on the management type of the remaining normally recognized clothing management symbols 500. Specifically, the processor 140 may recognize the management type of the normally recognized clothing management symbol 500, and then recognize the missing management type within the label. Then, the processor 140 may recognize a plurality of candidate management symbols corresponding to the clothing management symbol 500 recognized as a mis-recognized symbol, sort the plurality of candidate management symbols in descending order of reliability value, and then re-recognize the candidate symbol with the highest reliability value from the candidate symbols with the missing management type as the target symbol 620.

[0133] The processor 140 may re-apply the above verification method to the recognition result of the clothing management symbol 500 corresponding to the mis-recognized symbol obtained after correcting the mis-recognized symbol, and repeatedly verify whether the corrected recognition result is appropriate.

[0134] Figure 11 is an exemplary diagram showing line-by-line confirmation of mis-recognized symbols according to an embodiment of the present disclosure.

[0135] In addition, according to an embodiment of the present disclosure, when a plurality of recognized clothing management symbols 500 are arranged in multiple lines, the processor 140 may recognize mis-recognized symbols line by line.

[0136] Specifically, the processor 140 may recognize the positions of the plurality of clothing management symbols 500 within the label based on the label image 420. As an example, when the processor 140 inputs the label image 420 into the neural network model, the processor 140 may obtain information about the neural network model as a result value. The information about the neural network model may include not only the management type, standard type, and context type of each clothing management symbol 500, but also the position (e.g., coordinate value) of each clothing management symbol 500 within the label image 420. In this case, the processor 140 may recognize the position of each clothing management symbol 500 within the label based on the position information of each clothing management symbol 500, and may recognize the arrangement of the plurality of clothing management symbols 500 within the label image 420. Then, the processor 140 may recognize the plurality of clothing management symbols 500 arranged in multiple lines. That is, the processor 140 may recognize the rows and the number of rows formed by the plurality of clothing management symbols 500 according to the arrangement form of the plurality of clothing management symbols 500 within the label. In this case, when the number of rows is multiple, the processor 140 may recognize mis-recognized symbols line by line. That is, the processor 140 may recognize whether there is a mis-recognized symbol among the plurality of clothing management symbols 500 included in the same row line by line. Specifically, referring to Figure 11, the processor 140 can identify whether there is a mis-identified symbol among the four clothing management symbols 511, 512, 513, and 514 included in the first row 81, and can identify whether there is a mis-identified symbol among the four clothing management symbols 515, 516, 517, and 518 included in the second row 82.

[0137] In the label 11, a plurality of management symbols of a standard type corresponding to each row 80 can be written in a plurality of rows 81 and 82 (hereinafter referred to as 80) divided according to the standard type. Accordingly, the processor 140 can determine that a plurality of management symbols of the same standard type are written in each row 80, and identify a mis-identified symbol from the plurality of management symbols of the same standard type. That is, it is possible to identify different standard type clothing management symbols 500 within the same row 80 or a plurality of clothing management symbols 500 corresponding to the same management type based on each row 80.

[0138] The processor 140 can pre-determine the row 80 among the plurality of rows 80 in which a mis-identified symbol may be identified. Specifically, the processor 140 can identify the number of identified standard types and the number of management types corresponding to each row 80 based on the management type and standard type of the plurality of clothing management symbols 500 identified in each row 80. In this case, the processor 140 can determine the row 80 among the plurality of rows 80 in which a mis-identified symbol may be identified by comparing the number of identified standard types and the number of identified management types corresponding to each row 80. For example, when four standard types of KS standards are identified in the first row, four standard types of ASTM standards are identified in the second row, and three standard types of ISO standards and one standard type of KS standard are identified in the third row, the processor 140 can determine that the third row is the row in which a mis-identified symbol may be identified. Then, the processor 140 can determine and correct the mis-identified symbol in the third row.

[0139] Alternatively, when two management types of washing types, one management type of drying type, and one management type of dry cleaning type are respectively identified in the first row and the third row, and two management types of washing types and two management types of drying types are identified in the second row, the processor 140 can determine that the second row can be determined as the row in which a mis-identified symbol may be identified. That is, even if a plurality of first clothing management symbols corresponding to the same type are identified in the first row and the third row, the processor 140 can identify both the first row and the third row as including normally identified symbols. Accordingly, the processor 140 can only perform the process of identifying and correcting mis-identified symbols on the second row.

[0140] In addition, according to an embodiment of the present disclosure, when it is determined that the label image 420 does not include a plurality of clothing management symbols 500, the processor 140 may identify the material of the clothing 10 based on the label image 420 and generate management information about the clothing 10 based on the identified material.

[0141] Specifically, when the label image 420 does not include a plurality of clothing management symbols 500, the material of the clothing 10 may be used to generate management information about the clothing 10. However, the present disclosure is not limited thereto, and when the number of the plurality of clothing management symbols 500 identified in the label image 420 is less than a preset number, the processor 140 may generate management information about the clothing 10 based on the material of the clothing 10.

[0142] When the label image 420 does not include the clothing management symbol 500, the processor 140 may identify the material of the clothing 10. Here, the material of the clothing 10 may include the texture of the clothing 10, the mixing ratio of the clothing 10, and the color of the clothing 10. To this end, the processor 140 may obtain an image including the material information of the clothing 10 (i.e., an image including a part of the clothing 10) through the camera 110. Specifically, when it is determined that the label image 420 does not include the clothing management symbol 500, the processor 140 may display on the display 130 a UI that guides the user to obtain an image including the material information of the clothing 10 (i.e., guides the user to capture a part of the clothing 10). In addition, the processor 140 may identify the material of the clothing 10 by inputting the obtained image into a neural network model. Here, the neural network model may be a neural network model trained to identify at least one of the color and texture of the clothing 10 in the input image.

[0143] In addition, the processor 140 may identify the material of the clothing 10 based on information about the clothing 10 included in the label image 420 other than the clothing management symbol 500. For example, the processor 140 may extract the material information of the clothing 10 written on the label 11 or extract the mixing ratio information of the clothing 10 written on the label 11 to identify the material of the clothing 10. To this end, the processor 140 may use a trained neural network model to identify the text indicating the material information or the mixing ratio information written on the label 11 and output the material information. That is, the processor 140 may input the label image 420 into the neural network model to obtain the material information corresponding to the material information or the mixing ratio information of the clothing 10 included in the label image 420.

[0144] The neural network model trained to recognize the texture of the clothing 10 in the input image and the neural network model trained to output the material information by recognizing the text written in the label 11 indicating the material information or the mixing ratio information may be the same model as the neural network model that recognizes the clothing management symbol 500 in the label image 420. In this case, the neural network model may output the recognition results of the material information or the clothing management symbol 500 as the output values of different layers.

[0145] The processor 140 may recognize the material of the clothing 10 and then generate management information suitable for the clothing 10 based on the material of the clothing 10. For example, when it is recognized that the material of the clothing 10 is wool, the processor 140 may generate information suggesting not to wash the clothing 10 with water as the washing information of the clothing 10 and generate information suggesting dry cleaning as the dry cleaning information.

[0146] When the clothing management symbol 500 is recognized as a preset type, the processor 140 may perform additional recognition processing on the clothing management symbol 500. Specifically, when the clothing management symbol 500 is recognized as a rare type of symbol, the processor 140 may perform additional recognition processing to more accurately recognize the clothing management symbol 500.

[0147] In this regard, according to an embodiment of the present disclosure, candidate symbol information and the reliability of the candidate symbol information for each of the plurality of clothing management symbols 500 may be calculated, and when the candidate symbol information having the highest value among the calculated reliabilities is recognized as the first type of clothing management symbol 500, the recognized clothing management symbol 500 may be recognized as an unrecognizable symbol.

[0148] Here, the first type may be the rare type as described above. Specifically, when the processor 140 inputs the label image 420 into the neural network model and recognizes the plurality of clothing management symbols 500 included in the label image 420, the accuracy of the neural network model may vary according to the quantity (or amount) of the training data corresponding to each clothing management symbol 500. For clothing management symbols (500) that are used equally in many countries, the quantity of the training data may be large. Therefore, since the neural network model is trained for the clothing management symbol 500 based on a large amount of training data, the accuracy of the recognition result obtained through the neural network model may be high. On the other hand, for rare clothing management symbols 500, the quantity of the training data may be small. Therefore, since the neural network model is trained for the clothing management symbol 500 based on a small amount of training data, the accuracy of the recognition result obtained through the neural network model may be low. Accordingly, the processor 140 may perform a separate additional recognition process to increase the accuracy of the recognition result for rare clothing management symbols 500.

[0149] The processor 140 can identify whether the clothing management symbol 500 corresponds to a rare type. In this case, the clothing management symbol 500 with less than a preset number of training data may become a rare clothing management symbol 500. Then, when it is recognized that the clothing management symbol 500 is identified as a rare type, the processor 140 can identify the clothing management symbol 500 as an unrecognizable symbol.

[0150] For an unrecognizable symbol, due to the small amount of training data, even if the target symbol 620 of the clothing management symbol 500 is corrected based on reliability (more specifically, even if the candidate symbol with the second highest reliability value is used for correction), the recognition result of the clothing management symbol 500 may still be inaccurate. Accordingly, since the target symbol 620 is misrecognized, the unrecognizable symbol can be distinguished from the misrecognized symbol obtained by correcting the target symbol 620 based on reliability.

[0151] The processor 140 can identify the clothing management symbol 500 corresponding to the target symbol 620 as an unrecognizable symbol when the candidate symbol (i.e., the target symbol 620) with the highest reliability value among multiple candidate symbols is of a preset type (or preset category).

[0152] Alternatively, the processor 140 can identify the clothing management symbol 500 corresponding to the target symbol 620 as an unrecognizable symbol when the reliability value of the candidate symbol (i.e., the target symbol 620) with the highest reliability value among multiple candidate symbols is less than a preset value.

[0153] When the clothing management symbol 500 is identified as an unrecognizable symbol, the processor 140 can obtain a clothing management symbol image corresponding to the unrecognizable symbol based on the label image 420.

[0154] Specifically, the processor 140 can identify the clothing management symbol 500 recognized as an unrecognizable symbol in the label image 420 based on the position information of each clothing management symbol 500. Then, the processor 140 reduces, enlarges, or crops the label image 420 to obtain an image corresponding to the clothing management symbol 500 recognized as an unrecognizable symbol (hereinafter referred to as the clothing management symbol image).

[0155] Then, the processor 140 can identify the standard type and management type of the unrecognizable symbol based on the similarity between the obtained clothing management symbol image and multiple candidate symbol images.

[0156] To this end, a plurality of candidate symbol images may be stored in the memory 120. Specifically, the plurality of candidate symbol images stored in the memory 120 may be images of candidate symbols corresponding to rare types. That is, images of a plurality of clothing management symbols 500 corresponding to rare types may be stored in the memory 120 as the plurality of candidate symbol images.

[0157] The processor 140 may obtain the similarity between the acquired clothing management symbol image and each of the plurality of candidate symbol images stored in the memory 120 by matching the acquired clothing management symbol image with the plurality of candidate symbol images stored in the memory 120. Specifically, the processor 140 may extract the features of the clothing management symbol image and each candidate symbol image, respectively, and embed each of the extracted features into a three-dimensional vector. In this case, the processor 140 may obtain three-dimensional vectors corresponding to the features of the clothing management symbol image and each candidate symbol image based on an algorithm such as t-stochastic neighbor embedding (t-SNE). Then, the processor 140 may identify each three-dimensional vector in a preset three-dimensional space and identify the Euclidean distance between the vector corresponding to the clothing management symbol image and each vector corresponding to each candidate symbol image. Then, the processor 140 may identify each identified Euclidean distance using the similarity between the clothing management symbol image and each candidate symbol image. In this case, the processor 140 may identify that the smaller the Euclidean distance, the higher the similarity between the clothing management symbol image and the candidate symbol image. In addition to the Euclidean distance, the processor 140 may also identify the similarity between the clothing management symbol image and the candidate management symbol image based on various methods such as the Mahalanobis distance and the cosine distance between vectors.

[0158] The processor 140 may identify the candidate symbol image with the highest similarity from the plurality of candidate symbol images and identify the clothing management symbol (i.e., the unrecognizable symbol) corresponding to the clothing management symbol image based on the identified candidate symbol image. That is, the processor 140 may identify the candidate symbol corresponding to the candidate symbol image with the highest similarity as the target symbol 620 and identify the clothing management symbol 500 corresponding to the unrecognizable symbol based on the target symbol 620.

[0159] Figure 12 FIG. is an exemplary diagram showing generation of management information about an unrecognized management type based on the management type of the clothing management symbol 500 included in the label 11 according to an embodiment of the present disclosure.

[0160] According to an embodiment of the present disclosure, when a management type not included in a plurality of management types corresponding to a plurality of clothing management symbols 500 is identified, the processor 140 may obtain at least one management processor 140 information including management information corresponding to the management type from multiple pieces of management process information based on the generated management information. In this case, the multiple pieces of management process information may be a combination of multiple pieces of management information corresponding to multiple management types.

[0161] Specifically, the processor 140 may identify a plurality of clothing management symbols 500 included in the label image 420, and when a management type missing (or not included) in the label image 420 is identified based on the management type information of each clothing management symbol 500 (specifically, a management type not identified using the plurality of clothing management symbols 500 included in the label image 420), the processor 140 may generate management information about the missing (or not included) management type based on the management information generated according to the plurality of clothing management symbols 500 included in the label image 420. Hereinafter, for the convenience of describing the present disclosure, the management type missing (or not included) in the label image 420 is referred to as the first management type.

[0162] Refer to Figure 12 , when the label image 420 includes a plurality of clothing management symbols 511, 512, and 513 with management types of washing, dry cleaning, and ironing, the processor 140 may generate management information for each of washing, dry cleaning, and ironing. Specifically, the processor 140 may generate management information for each management type based on the semantic information of each clothing management symbol 511, 512, and 513. Specifically, the processor 140 may generate management information 911 suggesting washing in 60°C water for washing, management information 912 suggesting prohibition of dry cleaning for dry cleaning, and management information 913 suggesting ironing at a temperature of 140°C to 160°C for ironing. In this case, the processor 140 may generate management information about the first management type (e.g., drying) not included in the label image 420 based on the generated management information 911, 912, and 913.

[0163] For example, the processor 140 may obtain management information about drying by using at least one of the generated management information (washing, dry cleaning, and ironing) as a query. In this case, the memory 120 may store multiple pieces of management process information, in which multiple pieces of management information corresponding to multiple management types are combined. The processor 140 may identify the management information about washing and the management information about dry cleaning as queries (e.g., SQL queries) from the multiple pieces of generated management information, and obtain at least one of the multiple pieces of management process information based on the identified queries.

[0164] Specifically, the processor 140 may obtain the generated management process information including management information regarding washing (management information recommending washing in 60°C water) and management information regarding dry cleaning (management information recommending prohibition of dry cleaning) from multiple pieces of management process information.

[0165] Alternatively, the processor 140 may send the identified query information to the server through the communication interface to obtain management information not included in the label image 420.

[0166] Hereinafter, for the convenience of describing the present disclosure, the management process information obtained based on the generated management information among multiple pieces of management process information is referred to as first management process information.

[0167] The processor 140 may generate management information corresponding to the first management type not included in the label 11 based on the obtained first management process information. Specifically, the processor 140 may extract management information regarding drying from the obtained management process information and generate management information regarding drying based on the extracted management information regarding drying.

[0168] In this case, the processor 140 may only select, among the multiple obtained first management process information, the first management process information including the management information of the first management type to be generated (i.e., the management type not identified in the label 11). That is, to describe the above example again, the processor 140 may select, among the multiple selected first management process information, only the first management process information including the generated information regarding drying, and may generate management information regarding drying based on the selected first management process information.

[0169] In addition, when the number of the multiple obtained first management process information is more than a preset number, the processor 140 may generate management information corresponding to the first management type not included in the label (or the label image 420) based on the multiple obtained first management process information. In this case, the processor 140 may extract management information regarding the first management type from the obtained first management process information, and then generate management information regarding the first management type based on the management information with the weakest management intensity among the extracted management information regarding the first management type. Here, the management information with the weakest management intensity may be the management information with the lowest management temperature or the shortest management time. When the number of the multiple obtained first management process information is less than the preset number, the processor 140 may display a UI requesting input of management information regarding the management type not included on the display 130, match the input management information with the clothing 10 (more specifically, the clothing image 410 and the label image 420), and store the input management information in the memory 120.

[0170] According to an embodiment of the present disclosure, the processor 140 may send management information corresponding to a management type to a device for processing the clothing 10 corresponding to the management type through a communication interface included in the electronic device 100. In addition, the processor 140 may generate control information corresponding to the management information and send the generated control information to the device for processing the clothing 10. For example, when the processor 140 generates washing information about the clothing 10 based on the clothing management symbol 500 of the washing type among the plurality of clothing management symbols 500 included in the label image 420, the processor 140 may send the washing information to the washing machine through the communication interface, or may send control information corresponding to the washing information.

[0171] Figure 13 FIG. is a sequence diagram of the electronic device 100 implemented as a server device according to an embodiment of the present disclosure.

[0172] Refer to Figure 13 , when the electronic device 100 is implemented as a server device, the electronic device 100 may receive the label image 420 from a user terminal included in the system 1000 for managing the clothing 10. Here, the user terminal may be a user terminal registered in the server device and linked to the server device.

[0173] In this case, the electronic device 100 may use the received label image 420 to generate management information about the clothing 10. Specifically, the electronic device 100 may identify the plurality of clothing management symbols 500 included in the label image 420, confirm misidentified symbols among the plurality of identified clothing management symbols 500 based on the standard type and management type of each of the plurality of identified clothing management symbols 500, and correct the identified misidentified symbols based on at least one of the standard type and management type of another normally identified symbol. In this regard, the above description of the present disclosure may be similarly applied.

[0174] The electronic device 100 may send the generated management information to the user terminal, and the user terminal may display the received management information on the display 130.

[0175] In addition, the electronic device 100 may send control information to at least one device for managing the clothing 10 included in the system 1000 for managing the clothing 10 based on the generated management information.

[0176] The electronic device 100 may generate multiple pieces of management information for each user for multiple pieces of clothing 10 registered by the user, store the generated management information in the memory 120 (or database), and specifically, may cluster the clothing 10 of the registered user based on the management information or clothing management symbols 500 included in the label images 420 of each piece of clothing 10. For example, the electronic device 100 may cluster the clothing 10 among the multiple pieces of registered clothing 10 that have the same clothing management symbols 500 for washing and drying, and provide information to the user suggesting to wash or dry the multiple pieces of clustered clothing 10. Of course, even if the electronic device 100 is implemented as a smart phone, the above description can be similarly applied.

[0177] Figure 14 is a detailed configuration diagram of the electronic device 100 according to an embodiment of the present disclosure. Refer to Figure 14 , the electronic device 100 includes a camera 110, a memory 120, a display 130, a communication interface 150, a speaker 160, a microphone 170, an input interface 180, a sensor 190, and one or more processors 140. The detailed description of the components overlapping with the components shown in Figure 13 among the components shown in Figure 2 will be omitted.

[0178] The communication interface 150 may communicate with external devices and external servers through various communication methods. For example, the electronic device 100 may send a control signal corresponding to the generated management information to a device for managing the clothing 10 through the communication interface 150. Alternatively, when the electronic device 100 is implemented as a server device, the electronic device 100 may obtain the label image 420 from the user terminal through the communication interface 150, and send the generated management information to the user terminal. Alternatively, when the electronic device 100 is implemented as a smart phone, the electronic device 100 may send the generated query to a server device linked to the electronic device 100 through the communication interface 150, and receive management information about the first management type.

[0179] The communication connection between the communication interface 150 and external devices and external servers may include communication through a third device (e.g., a repeater, a hub, an access point, a gateway, etc.). For example, the external device may be implemented as another external electronic device 200, a server, a cloud storage device, a network, etc.

[0180] In addition, the communication interface 150 may include various communication modules to communicate with external devices. For example, the communication interface 150 may include a wireless communication module and a cellular communication module, and the cellular communication module uses at least one of, for example, the third generation (3G), the 3rd Generation Partnership Project (3GPP), Long Term Evolution (LTE), and LTE-Advanced (LTE-A), Code Division Multiple Access (CDMA), Wideband CDMA (WCDMA), Universal Mobile Telecommunication System (UMTS), Wireless Broadband (WiBro), or Global System for Mobile Communications (GSM). As another example, the wireless communication module may include at least one of, for example, Wireless Fidelity (WiFi), Bluetooth, Bluetooth Low Energy (BLE), and Zigbee.

[0181] The speaker 160 is a component that outputs various audio data, and the audio data has been subjected to various processing tasks such as decoding, amplification, and noise filtering by the audio processing unit. The speaker 160 may output various notification sounds or voice messages. According to an embodiment of the present disclosure, the processor 140 may convert an electrical signal received from an external device into the user's voice and output the converted user's voice through the speaker 160.

[0182] The microphone 170 may receive an acoustic signal around the electronic device 100. For example, the microphone 170 may receive the voice of a user who controls the electronic device 100. When the voice of the user for performing a specific function is received through the microphone 170, the processor 140 may convert the user's voice into a digital signal through a Speech-to-Text (STT) algorithm and provide response information corresponding to the user's voice.

[0183] The input interface 180 is a configuration that allows the electronic device 100 to interact with the user. For example, the user interface may include at least one of a touch sensor, a motion sensor, a button, a dial, a switch, and the microphone 170, but is not limited thereto.

[0184] The sensor 190 may acquire information about the external environment or information about a user command. To this end, the sensor 190 may be implemented in the form of a circuit as at least one of a gyro sensor, an infrared sensor, a ToF sensor, a LiDAR sensor, an acceleration sensor, a touch sensor, and a motion sensor. In addition, the sensor 190 may further include at least one of a Global Positioning System (GPS) sensor and a geomagnetic sensor.

[0185] The methods according to various embodiments of the present disclosure described above may be implemented in the form of an application that can be installed in an existing electronic device. Alternatively, the methods according to various embodiments of the present disclosure described above may also be executed using a deep learning-based artificial neural network (or deep artificial network) (i.e., a learning network model).

[0186] In addition, the methods according to various embodiments of the present disclosure can be implemented only through software upgrades or hardware upgrades of existing electronic devices.

[0187] In addition, various embodiments of the present disclosure above can be executed by an embedded server provided in an electronic device or a server external to the electronic device.

[0188] According to an embodiment of the present disclosure, the various embodiments above can be implemented by software including instructions stored in a machine-readable storage medium (e.g., a computer-readable storage medium). The machine can be a device that calls the stored instructions from the storage medium and operates according to the called instructions, and can include an electronic device (e.g., Electronic Device A) according to the disclosed embodiment. In the case where the command is executed by a processor, the processor can directly or under the control of the processor use other components to execute the function corresponding to the command. The command can include code created or executed by a compiler or an interpreter. The machine-readable storage medium can be provided in the form of a non-transitory storage medium. Here, the term "non-transitory" means that the storage medium is tangible and does not include signals, and does not distinguish whether the data is stored semi-permanently or temporarily in the storage medium.

[0189] In addition, according to an embodiment of the present disclosure, the above methods according to different embodiments can be included and provided in a computer program product. The computer program product can be traded as a product between a seller and a buyer. The computer program product can be distributed in the form of a storage medium (e.g., a compact disc read-only memory (CD-ROM)), which can be read by a machine or read online through an application store (e.g., PlayStore TM ). For the case of online distribution, at least a part of the computer program product can be stored at least temporarily in a storage medium such as the memory of a manufacturer's server, an application store's server, or a relay server, or generated temporarily.

[0190] In addition, each component (e.g., a module or a program) according to the various embodiments above can include a single entity or multiple entities, and some of the corresponding sub-components above can be omitted, or other sub-components can be further included in different embodiments. Alternatively or additionally, some components (e.g., a module or a program) can be integrated into a single entity to perform the same or similar functions as those performed by each corresponding component before integration. The operations performed by the modules, programs, or other components according to the various embodiments can be executed in a sequential manner, a parallel manner, an iterative manner, or a heuristic manner, at least some operations can be executed in a different order or omitted, or other operations can be added.

[0191] Although the embodiments of the present disclosure have been shown and described above, the present disclosure is not limited to the above specific embodiments, but those skilled in the art to which the present disclosure pertains can make various modifications without departing from the gist of the present disclosure disclosed in the appended claims. These modifications should also be understood to fall within the scope and spirit of the present disclosure.

Claims

1. An electronic device, comprising: a camera; at least one memory configured to store at least one instruction; a display; and at least one processor configured to execute the at least one instruction to perform the following operations: obtain a label image of clothing through the camera, based on identifying a plurality of clothing management symbols in the label image: identify mis-identified clothing management symbols among the plurality of clothing management symbols based on the standard type and management type of each clothing management symbol in the plurality of clothing management symbols, correct the mis-identified clothing management symbols based on at least one of the standard type and management type of the correctly identified symbols among the plurality of clothing management symbols, generate management information related to the clothing using the label image, and control the display to display the generated management information.

2. The electronic device according to claim 1, wherein, the at least one memory stores a neural network model configured to identify the plurality of clothing management symbols in the label image, and wherein the at least one processor is further configured to: execute the at least one instruction to identify the plurality of clothing management symbols included in the label image using the neural network model.

3. The electronic device according to claim 1, wherein, the at least one processor is further configured to execute the at least one instruction to perform the following operations: identify candidate symbol information of each clothing management symbol among the plurality of clothing management symbols and the reliability of the identified candidate symbol information, and identify the mis-identified clothing management symbols by comparing the standard type of the identified candidate symbol information with the highest reliability value with the standard type of another identified candidate symbol information.

4. The electronic device according to claim 1, wherein, the at least one processor is further configured to execute the at least one instruction to perform the following operations: identify candidate symbol information of each clothing management symbol among the plurality of clothing management symbols and the reliability of the identified candidate symbol information, identify a plurality of first clothing management symbols corresponding to the identified candidate symbol information with the same management type among the plurality of clothing management symbols, and identify the first clothing management symbol corresponding to the identified candidate symbol information with the lowest reliability value among the plurality of first clothing management symbols as the mis-identified clothing management symbol.

5. The electronic device according to claim 4, wherein, the at least one processor is further configured to execute the at least one instruction to perform the following operations: identify multiple semantic information corresponding to each first clothing management symbol among the plurality of first clothing management symbols, and based on identifying that the multiple semantic information does not match, identify the first clothing management symbol corresponding to the identified candidate symbol information with the lowest reliability value among the plurality of first clothing management symbols as the mis-identified clothing management symbol.

6. The electronic device according to claim 1, wherein, The at least one processor is further configured to execute the at least one instruction to perform the following operations: Based on the plurality of clothing management symbols being arranged in a plurality of rows in the label image, repeat the operations of claim 1 row by row.

7. The electronic device according to claim 1, wherein, The at least one processor is further configured to execute the at least one instruction to perform the following operations: Based on identifying that the label image does not include clothing management symbols, identify the material of the clothing based on the label image, and generate management information about the clothing based on the identified material.

8. The electronic device according to claim 1, wherein, The at least one memory stores a plurality of candidate symbol images corresponding to each of the plurality of candidate symbol information, and wherein the at least one processor is further configured to execute the at least one instruction to perform the following operations: Identify the candidate symbol information of each clothing management symbol among the plurality of clothing management symbols and the reliability of the identified candidate symbol information, Based on the identified candidate symbol information with the highest reliability value having a first type of clothing management symbol, identify the identified clothing management symbol corresponding to the identified candidate symbol information with the highest reliability value as an unrecognizable symbol, Obtain a clothing management symbol image corresponding to the unrecognizable symbol based on the label image, and Identify the standard type and management type of the unrecognizable symbol based on the similarity between the obtained clothing management symbol image and each of the plurality of candidate symbol images.

9. The electronic device according to claim 1, wherein, The at least one processor is further configured to execute the at least one instruction to perform the following operations: Based on identifying that the label image does not include clothing management symbols associated with a first management type, obtain at least one first management processor information including management information corresponding to the first management type based on the generated management information, and Generate management information corresponding to the first management type based on the obtained first management process information.

10. The electronic device according to claim 1, further comprises: A communication interface configured to communicate with a clothing processing device, wherein the at least one processor is further configured to execute the at least one instruction to perform the following operations: Control the communication interface to send management information corresponding to the management type to the clothing processing device.

11. A method for controlling an electronic device, the method comprises: Obtain a label image of clothing through a camera; Identify a plurality of clothing management symbols in the label image; Based on the standard type and management type of each clothing management symbol among the plurality of clothing management symbols, identify mis-identified clothing management symbols among the plurality of clothing management symbols; Correct the mis-identified clothing management symbols based on at least one of the standard type and management type of the correctly identified symbols among the plurality of clothing management symbols; Generate management information about the clothing using the label image; and Display the generated management information on a display.

12. The method according to claim 11, wherein, identifying the plurality of clothing management symbols includes using a neural network model configured to identify the plurality of clothing management symbols in the label image.

13. The method according to claim 11, wherein, identifying the mis-identified clothing management symbol further includes: identifying candidate symbol information of each clothing management symbol among the plurality of clothing management symbols and the reliability of the identified candidate symbol information; and identifying the mis-identified clothing management symbol by comparing the standard type of the candidate symbol information with the highest reliability value identified with the standard type of another identified candidate symbol information.

14. The method according to claim 11, wherein, identifying the mis-identified clothing management symbol further includes: identifying candidate symbol information of each clothing management symbol among the plurality of clothing management symbols and the reliability of the identified candidate symbol information; identifying a plurality of first clothing management symbols corresponding to the candidate symbol information of the same management type identified among the plurality of clothing management symbols; and identifying the first clothing management symbol corresponding to the candidate symbol information with the lowest reliability value identified among the plurality of first clothing management symbols as the mis-identified clothing management symbol.

15. The method according to claim 14, wherein, identifying the mis-identified clothing management symbol further includes: identifying a plurality of semantic information corresponding to each first clothing management symbol among the plurality of first clothing management symbols; and identifying the first clothing management symbol corresponding to the candidate symbol information with the lowest reliability value identified among the plurality of first clothing management symbols as the mis-identified clothing management symbol based on identifying that the identified plurality of semantic information does not match.

Images