Method and device for recognizing pattern coded visually

Abstract

To provide a method for recognizing a pattern that a user terminal executes.SOLUTION: The method for recognizing a pattern includes the steps of: receiving an image including a first pattern and a second pattern; determining the pattern type of the first pattern; determining the pattern type of the second pattern; comparing the priorities of the pattern type of the first pattern and the pattern type of the second pattern; and automatically executing processing of the first pattern if the pattern type of the first pattern is determined to have a higher priority than that of the second pattern.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to methods and apparatus for recognizing visually coded patterns, and more particularly to methods and apparatus for recognizing and automatically executing multiple visually coded patterns and displaying previews of multiple visually coded patterns. [Background technology]

[0002] Devices with built-in cameras, such as smartphones, tablets, PCs, laptops, and desktop PCs, are becoming increasingly common. In this environment, the use of built-in cameras in devices is on the rise. A typical example is the proliferation of services that utilize barcodes captured with built-in cameras in devices.

[0003] In this case, if multiple barcodes are included in an image captured by a built-in camera of a terminal, the barcodes may not be recognized or a barcode that the user does not want may be recognized. In such a case, the user must take another photograph of the barcode that the user wants to recognize in order to receive the desired service, which is inconvenient. This increases the time it takes for the user to receive the desired service, making it difficult to provide a smooth user experience. Summary of the Invention [Problem to be solved by the invention]

[0004] The present disclosure provides a pattern recognition method, an apparatus, and a computer program recorded on a recording medium to solve the above-mentioned problems.

[0005] Provided are a method, an apparatus, and a computer program recorded on a recording medium for receiving an image including a plurality of visually coded patterns at a user terminal, comparing the priorities of the pattern types by determining the pattern types of the patterns detected from the received image, and automatically processing the patterns with higher priorities based on the comparison.

[0006] Provided are a method, an apparatus, and a computer program recorded on a recording medium for receiving an image including a plurality of visually coded patterns at a user terminal, generating a preview of one or more patterns detected from the received image, and displaying the preview in a preview area on the display of the user terminal. [Means for solving the problem]

[0007] The present disclosure can be implemented in numerous ways, including as a method, an apparatus, or a computer program stored on a computer-readable recording medium.

[0008] A pattern recognition method performed by a user terminal according to an embodiment of the present disclosure includes receiving an image including a first visually coded pattern and a second visually coded pattern, determining a pattern type of the first visually coded pattern, determining a pattern type of the second visually coded pattern, comparing priorities of the pattern type of the first visually coded pattern and the pattern type of the second visually coded pattern, and automatically performing processing of the first visually coded pattern if it is determined that the pattern type of the first visually coded pattern has a higher priority than the pattern type of the second visually coded pattern.

[0009] According to one embodiment, the step of determining the pattern type of the first visually coded pattern includes a step of determining whether the first visually coded pattern is a pattern associated with a specific service, and if the first visually coded pattern is determined to be a pattern associated with the specific service, a step of determining the pattern type of the first visually coded pattern as the first pattern type.

[0010] According to one embodiment, the step of determining the pattern type of the first visually coded pattern further includes a step of determining whether the first visually coded pattern is a pattern associated with a service of a specific company, and if the first visually coded pattern is determined to be a pattern associated with a service of a specific company, a step of determining the pattern type of the first visually coded pattern as a second pattern type.

[0011] According to one embodiment, when the first visually coded pattern is determined to be a pattern not related to a specific company's service, the step of determining the pattern type of the first visually coded pattern further includes a step of determining the pattern type of the first visually coded pattern as a third pattern type if the first visually coded pattern is a two-dimensional pattern, and a step of recognizing the pattern type of the first visually coded pattern as a fourth pattern type if the first visually coded pattern is a one-dimensional pattern.

[0012] According to one embodiment, the first pattern type has the highest priority, followed by the second pattern type, the third pattern type, and the fourth pattern type.

[0013] According to one embodiment, the method further includes calculating a reliability score of the first visually coded pattern based on at least one of a position or a size of the first visually coded pattern in the received image if it is determined that the pattern type of the first visually coded pattern has the same priority as the pattern type of the second visually coded pattern; calculating a reliability score of the second visually coded pattern based on at least one of a position or a size of the second visually coded pattern in the received image; and displaying a preview of the first visually coded pattern on a display of the user terminal if it is determined that the reliability score of the first visually coded pattern is higher than the reliability score of the second visually coded pattern.

[0014] In one embodiment, calculating a confidence score for the first visually coded pattern based on at least one of the position or size of the first visually coded pattern in the received image comprises calculating a distance between a center point of the received image and a center point of the first visually coded pattern.

[0015] In one embodiment, calculating a confidence score for the first visually coded pattern based on at least one of the position or size of the first visually coded pattern in the received image comprises calculating a percentage of the area of ​​the received image that is occupied by an area corresponding to the first visually coded pattern.

[0016] In one embodiment, the method further includes calculating a confidence score for the first visually coded pattern based on at least one of a position or a size of the first visually coded pattern in the received image if it is determined that the pattern type of the first visually coded pattern has the same priority as the pattern type of the second visually coded pattern, calculating a confidence score for the second visually coded pattern based on at least one of a position or a size of the second visually coded pattern in the received image, and automatically performing processing of the first visually coded pattern if it is determined that the confidence score of the first visually coded pattern is higher than the confidence score of the second visually coded pattern.

[0017] In one embodiment, the method further includes a step of notifying a user that processing of the first visually coded pattern will be automatically performed by displaying an indicator on a first area corresponding to the first visually coded pattern.

[0018] According to one embodiment, the method further includes detecting a first region including a first visually coded pattern, detecting a second region including a second visually coded pattern, and highlighting the first region and the second region in different colors.

[0019] According to one embodiment, the method further includes receiving a plurality of images in sequence in an image capture mode, and, when an image including a first visually coded pattern and a second visually coded pattern is received, displaying a notification on a display of the user terminal to confirm whether to enter a pattern scan mode.

[0020] A computer program is provided that is recorded on a computer-readable recording medium for executing the pattern recognition method described in one embodiment of the present disclosure on a computer.

[0021] A pattern recognition device according to one embodiment of the present disclosure includes an image sensor configured to receive an image including a first visually coded pattern and a second visually coded pattern, a pattern type determination unit configured to determine a pattern type of the first visually coded pattern and a pattern type of the second visually coded pattern, and a pattern processing unit configured to compare the priorities of the pattern type of the first visually coded pattern and the pattern type of the second visually coded pattern, wherein the pattern processing unit is further configured to automatically perform processing of the first visually coded pattern when it determines that the pattern type of the first visually coded pattern has a higher priority than the pattern type of the second visually coded pattern.

[0022] A pattern recognition method performed by a user terminal according to one embodiment of the present disclosure includes receiving an image including a first visually coded pattern and a second visually coded pattern, generating a preview of the first visually coded pattern, generating a preview of the second visually coded pattern, displaying the preview of the first pattern and the preview of the second pattern in a preview area on a display of the user terminal, receiving a selection for the preview of the first pattern from a user, and performing processing on the first visually coded pattern in response to the selection.

[0023] According to one embodiment, the step of displaying a preview of the first pattern and a preview of the second pattern in a preview area on a display of the user terminal includes the steps of: calculating a reliability score of the first visually coded pattern based on at least one of a position or a size of the first visually coded pattern in the received image; calculating a reliability score of the second visually coded pattern based on at least one of a position or a size of the second visually coded pattern in the received image; and, if it is determined that the reliability score of the first visually coded pattern is higher than the reliability score of the second visually coded pattern, displaying a preview of the first visually coded pattern at an upper end of the preview area on the display of the user terminal.

[0024] According to one embodiment, the method further includes detecting a first region including a first visually coded pattern, detecting a second region including a second visually coded pattern, and highlighting the first region and the second region in different colors.

[0025] According to one embodiment, generating a preview of the visually coded first pattern further includes, if the visually coded first pattern corresponds to encoding of a Uniform Resource Locator (URL), transmitting URL information coded by the visually coded first pattern to an external device; receiving at least one of a thumbnail, a title, or a description associated with the URL information from the external device; and displaying the received thumbnail, title, or description in a preview area on a display.

[0026] A computer program is provided that is recorded on a computer-readable recording medium for executing the pattern recognition method described in one embodiment of the present disclosure on a computer.

[0027] A pattern recognition device according to one embodiment of the present disclosure includes an input interface configured to receive input from a user; an image sensor configured to receive an image including a first visually coded pattern and a second visually coded pattern; a pattern preview generator configured to generate a preview of the first visually coded pattern and a preview of the second visually coded pattern; a display configured to output the generated previews of the first visually coded pattern and the second visually coded pattern to a preview area; and a pattern processor configured to perform processing on the first visually coded pattern in response to receiving a selection for the preview of the first pattern through the input interface. [Effects of the Invention]

[0028] According to an embodiment of the present disclosure, for multiple visually coded patterns, automatic pattern processing is performed based on the priority of the pattern type and / or the reliability score of the pattern, thereby improving the accuracy of pattern recognition and facilitating the provision of services desired by the user, thereby providing a smooth user experience.

[0029] According to an embodiment of the present disclosure, previews of multiple patterns can be displayed in a preview area on the display of the user terminal, allowing the user to select a pattern associated with a desired service.

[0030] The effects of the present disclosure should not be limited to those described above, and other effects not mentioned will be clearly understood by those skilled in the art from the following description. [Brief explanation of the drawings]

[0031] Embodiments of the present disclosure will now be described with reference to the accompanying drawings, where like reference numerals indicate like elements, without limitation, in which:

[0032] [Figure 1] FIG. 10 is a diagram illustrating an example of a pattern recognition method executed by a user terminal in one embodiment of the present disclosure. [Figure 2] FIG. 1 is a diagram illustrating a configuration in which multiple user terminals are communicatively connected to a server to perform pattern recognition in one embodiment of the present disclosure. [Figure 3] FIG. 2 is a block diagram showing the configuration of a user terminal in one embodiment of the present invention. [Figure 4] FIG. 10 is a diagram illustrating a lookup table that defines a priority for each pattern type in one embodiment of the present invention. [Figure 5] FIG. 10 illustrates an example of automatically processing a pattern without user input in accordance with an embodiment of the present invention. [Figure 6] 1 is a flowchart illustrating a visually coded pattern recognition method according to one embodiment of the present disclosure. [Figure 7] 10 is a flowchart illustrating a method for performing processing on one of patterns having the same priority in an embodiment of the present disclosure. [Figure 8] 10 is a flowchart illustrating a method for performing processing on one of patterns having the same priority in an embodiment of the present disclosure. [Figure 9] 1 is a flowchart illustrating a method for determining a pattern type of a visually coded pattern according to one embodiment of the present disclosure. [Figure 10] FIG. 10 is a diagram illustrating an example of displaying a notification to confirm whether to enter pattern scanning mode when a user terminal receives multiple visually coded patterns in one embodiment of the present disclosure. [Figure 11] FIG. 10 is a diagram illustrating an example in which a preview of a pattern selected by a user is displayed on a display according to an embodiment of the present disclosure. [Figure 12] FIG. 10 is a diagram showing an example of displaying previews of multiple patterns included in an image on a display according to an embodiment of the present disclosure. [Figure 13] FIG. 10 is a diagram illustrating an example in which a preview of a pattern selected by a user is displayed on a display according to an embodiment of the present disclosure. [Figure 14] 1 is a flowchart illustrating a visually coded pattern recognition method according to one embodiment of the present disclosure. [Figure 15] 1 is a flowchart illustrating a visually coded pattern recognition method according to one embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0033] Hereinafter, specific details for implementing the present disclosure will be described in detail with reference to the accompanying drawings. However, in the following description, detailed descriptions of well-known functions and configurations will be omitted if they may unnecessarily obscure the gist of the present disclosure.

[0034] In the accompanying drawings, identical or corresponding components are denoted by the same reference numerals. In addition, in the following description of the embodiments, duplicate descriptions of identical or corresponding components may be omitted. However, even if the description of a component is omitted, it is not intended that the component is not included in one embodiment.

[0035] The advantages and features of the disclosed embodiments, and methods for achieving them, will become clearer with reference to the embodiments described below in conjunction with the accompanying drawings. However, the present disclosure should not be limited to the embodiments disclosed below, and may be realized in various different forms. The present embodiments are provided merely to ensure a complete understanding of the present disclosure and to fully convey the scope of the invention to those skilled in the art to which the present disclosure pertains.

[0036] First, the terms used in this specification will be briefly explained, and then the disclosed embodiments will be specifically described.

[0037] The terms used in this specification have been selected from currently widely used general terms that are possible while taking into consideration the functionality of the present disclosure, but these may differ depending on the intentions of those skilled in the art, precedents, the emergence of new technology, etc. In addition, in certain cases, terms arbitrarily selected by the applicant are also included, and in such cases, their meanings will be described in detail in the relevant description of the invention. Therefore, the terms used in this disclosure should be defined based on the meanings implied by the terms and the overall content of the present disclosure, rather than simply by the name of the terms.

[0038] As used herein, the singular includes the plural unless the context clearly dictates otherwise, and the plural includes the singular unless the context clearly dictates otherwise.

[0039] Throughout this specification, when a part "comprises" an element, it means that it further includes other elements, not that it excludes other elements, unless stated to the contrary.

[0040] Additionally, the terms "module" or "unit" used in this specification refer to a software or hardware component, and the "module" or "unit" performs a certain function. However, a "module" or "unit" should not be limited to software or hardware. A "module" or "unit" may be configured to reside on an addressable storage medium or to execute on one or more processors. Thus, as an example, a "module" or "unit" may include at least one of a software component, an object-oriented software component, a component such as a class component or task component, a process, a function, an attribute, a procedure, a subroutine, a segment of program code, a driver, firmware, microcode, a circuit, data, a database, a data structure, a table, an array, or a variable. The components and functions provided within a "module" or "unit" may be combined into fewer components and "modules" or "units," or may be further separated into additional components and "modules" or "units."

[0041] According to one embodiment of the present disclosure, a "module" or "unit" may be implemented with a processor and memory. "Processor" should be broadly interpreted to include a general-purpose processor, a central processing unit (CPU), a microprocessor, a digital signal processor (DSP), a controller, a microcontroller, a state machine, etc. In some environments, "processor" may refer to an application-specific integrated circuit (ASIC), a programmable logic device (PLD), a field-programmable gate array (FPGA), etc. "Processor" may also refer to a combination of processing devices, such as, for example, a combination of a DSP and a microprocessor, a combination of multiple microprocessors, a combination of one or more microprocessors in conjunction with a DSP core, or any other such configuration. Additionally, "memory" should be broadly interpreted to include any electronic component capable of storing electronic information. "Memory" may refer to various types of processor-readable media, such as random access memory (RAM), read-only memory (ROM), nonvolatile random access memory (NVRAM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory, magnetic or optical data storage devices, registers, etc. A memory and a processor are in electronic communication if the processor can read information from or write information to the memory. Memory integrated into a processor is in electronic communication with the processor.

[0042] In this disclosure, an "image" may include an image captured by an image sensor built into a user terminal, an image captured by an image sensor configured to be able to communicate with a user terminal, or a partial image extracted from captured video footage.

[0043] In this disclosure, the term "visually coded pattern" refers to a code that displays data or information in a machine-readable visual form. A visually coded pattern is scanned / read using a pattern reader or an application program on a device with a built-in camera, such as a smartphone. Visually coded patterns include one-dimensional visually coded patterns and two-dimensional visually coded patterns. One-dimensional visually coded patterns are composed of parallel lines of different widths and spacing, and include Universal Product Codes (UPCs). Two-dimensional visually coded patterns are composed of geometric patterns such as squares, dots, and hexagons, and include Data Matrix, Maxi Codes, and Quick Response (QR) codes. In this specification, the term "visually coded pattern" may also be referred to as "pattern."

[0044] 1 is a diagram illustrating an example of a pattern recognition method performed by a user terminal 120 according to an embodiment of the present disclosure. As illustrated in FIG. 1, a user 110 may hold the user terminal 120 so that an image sensor of the user terminal 120 is directed toward an object 130 to scan one of a first pattern 132, a second pattern 134, and a third pattern 136 included in the object 130. For example, the object 130 may be a document on which multiple visually coded patterns are printed. As illustrated, the first pattern 132 and the second pattern 134 may be two-dimensional patterns, and the third pattern 136 may be a one-dimensional pattern.

[0045] When the user terminal 120 receives an image including the first pattern 132, the second pattern 134, and the third pattern 136, the user terminal 120 may detect an area including the visually coded patterns 132, 134, and 136 from the image and determine the pattern type of each pattern. The user terminal 120 may compare the priority of the pattern type of each pattern and automatically execute processing for the pattern with the highest priority. For example, if the user terminal 120 determines that the priority of the first pattern 132 is higher than the priority of the second pattern 134 and the third pattern 136, the user terminal 120 may automatically execute processing for the first pattern 132 without user input.

[0046] To this end, the user terminal 120 may record a priority for each predefined pattern type. In one embodiment, the user terminal 120 may determine a pattern related to a financial service of a specific company (e.g., the LINE Pay service) as a first pattern type, a pattern related to a non-financial service of a specific company (e.g., the LINE Messenger service) as a second pattern type, a two-dimensional pattern related to services of other companies as a third pattern type, and other one-dimensional patterns as a fourth pattern type. In this case, the user terminal 120 may predefine the first pattern type, the second pattern type, the third pattern type, and the fourth pattern type as having the highest priority in this order.

[0047] If multiple patterns of the same pattern type exist in an image and no patterns with a higher priority than the pattern in question exist, the user terminal 120 may provide the user with previews of the patterns in question and allow the user to select one pattern to run (or scan). Alternatively, the user terminal may calculate a reliability score for the patterns in question and automatically process the pattern with the highest reliability score. In this case, the reliability score for each pattern may be calculated based on at least one of the position or size of the pattern in the image.

[0048] In another embodiment, the user terminal 120 may allow the user 110 to select one of the patterns 132, 134, 136 contained in the image, rather than automatically performing an operation on one of the patterns in the image. In this case, the user terminal 120 may display the patterns 132, 134, 146 detected from the image as selectable options on the display. Additionally or alternatively, the user terminal 120 may generate previews of the patterns 132, 134, 146 detected from the image and display them on the display. In this case, the user 110 may view the previews of the patterns displayed on the display, select a pattern, and perform an operation on that pattern.

[0049] 2 illustrates a configuration in which multiple user terminals 210, 220, and 230 are communicatively connected to a server 250 to perform pattern recognition in accordance with an embodiment of the present disclosure. While FIG. 2 illustrates a mobile phone terminal 210, a tablet terminal 220, and a laptop PC 230 as examples of user terminals, the user terminals are not limited thereto. The user terminals may be any computing device capable of wired and / or wireless communication, having an image sensor (e.g., a camera) built-in or capable of communicating with an image sensor, and having an application program installed and executable for scanning visually coded patterns. For example, the user terminals may include smartphones, mobile phones, navigation systems, PCs, laptop PCs, digital broadcasting terminals, personal digital assistants (PDAs), portable multimedia players (PMPs), tablets, game consoles, wearable devices, internet of things (IoT) devices, virtual reality (VR) devices, and augmented reality (AR) devices.

[0050] 2 illustrates three user terminals 210, 220, and 230 communicating with server 250 via network 240, but this is not a limitation and other numbers of user terminals may be configured to communicate with server 250 via network 240. The communication method via network 240 is not limited and may include a mobile communication network, a wired Internet, a wireless Internet, a broadcast network, a satellite network, short-range wireless communication between user terminals, etc. For example, network 240 may include one or more of a personal area network (PAN), a local area network (LAN), a campus area network (CAN), a metropolitan area network (MAN), a wide area network (WAN), a broadband network (BBN), the Internet, etc. Furthermore, network 240 may include one or more of a network topology including, but not limited to, a bus network, a star network, a ring network, a mesh network, a star-bus network, a tree network, or a hierarchical network.

[0051] In one embodiment, server 250 may include one or more server devices and / or databases capable of storing, providing, and executing computer-executable programs and data, or one or more distributed computing devices and / or distributed databases of a cloud computing service infrastructure. In one embodiment, server 250 may be a web server configured to store at least one of a thumbnail, a title, or a description of a uniform resource locator (URL).

[0052] In one embodiment, if the pattern corresponds to encoding of a uniform resource locator (URL), the user terminal 210, 220, 230 may transmit the pattern to the server 250 or transmit URL information coded by the pattern to the server 250 to generate a preview of the pattern. In this case, the server 250 may provide at least one of a thumbnail, a title, or a description associated with the URL to the user terminal 210, 220, 230. The user terminal 210, 220, 230 may display the received thumbnail, title, or description on a display as a preview of the pattern.

[0053] 3 is a block diagram illustrating the configuration of a user terminal 300 according to an embodiment of the present invention. The user terminal 300 may be any device equipped with capture and image processing capabilities. As shown in the figure, the user terminal 300 may include an image sensor 310, a display 320, an input interface 330, a communication module 340, a processor 350, and a storage unit 360.

[0054] Image sensor 310 may be configured to receive one or more images. For example, image sensor 310 may be a camera module. Images received by image sensor 310 may be temporarily or permanently stored in the form of electronic files in storage 360 ​​of user terminal 300. In one embodiment, images received by image sensor 310 may be provided to processor 350 to determine whether the images include a visually coded pattern. In other embodiments, image sensor 310 may be configured to store images in storage 360 ​​or provide images to processor 350 in response to receiving an input command from a user via input interface 330.

[0055] The display 320 may be configured to output information and / or data generated by the user terminal 300. According to one embodiment, images received by the image sensor 310 may be displayed on the display 320. In addition to the display 320, devices such as speakers, haptic feedback devices, etc. may be provided as output devices in addition to the display 320.

[0056] The input interface 330 may be configured to allow a user to input information and / or data into the user terminal 300. For example, the input interface 330 may include devices such as a touch screen, buttons, a keypad, a keyboard, a microphone, a mouse, etc. While the display 320 and the input interface 330 are shown as separate components in FIG. 3, this is not limiting, and the display 320 and the input interface 330 may be integrated and provided as an input / output device.

[0057] 3, the user terminal 300 may be configured to communicate information and / or data with other devices via a network 370 using a communication module 340. For example, a request generated by the processor 350 of the user terminal 300 using a program code recorded in the storage unit 360 may be transmitted to another device (e.g., a server) via the network 370 under the control of the communication module 340. Conversely, control signals, instructions, content, files, etc. provided by other devices may be received by the user terminal 300 via the communication module 340. For example, the control signals, instructions, content, files, etc. from other devices received via the communication module 340 may be transmitted to the processor 350 or the storage unit 360.

[0058] The processor 350 may be configured to process computer program instructions by performing basic arithmetic, logic, and input / output operations. The instructions may be provided to the processor 350 by the storage unit 360 or the communication module 340. For example, the processor 350 may be configured to execute instructions received according to program code recorded in the storage unit 360. In one embodiment, the processor 350 may include a pattern region detection unit 352, a pattern type determination unit 354, a pattern preview generation unit 356, and a pattern processing unit 358.

[0059] Storage unit 360 may include any computer-readable recording medium. According to an embodiment, storage unit 360 may include a recording device such as a random access memory (RAM), a read-only memory (ROM), a disk drive, a solid state drive (SSD), a flash memory, etc. Storage unit 360 may store at least one program code, such as an operating system or an application program.

[0060] In one embodiment, image sensor 310 may receive an image including multiple patterns. In this case, the image received by image sensor 310 may be output on display 320 and provided to processor 350. Processor 350 may provide the received image to pattern region detector 352, which may detect regions within the image that include visually coded patterns and determine whether the image includes any visually coded patterns.

[0061] If the pattern region detection unit 352 determines that the received image does not contain a visually coded pattern, the processor 350 may cause the image sensor 310 to receive a subsequent image. If the pattern region detection unit 352 determines that the received image contains one visually coded pattern, the pattern region detection unit 352 may provide the pattern to the pattern processing unit 358, which may then process the pattern. If the pattern region detection unit 352 determines that the received image contains multiple patterns, the pattern region detection unit 352 may provide the detected multiple patterns (or an image of the region in which the pattern was detected) to the pattern type determination unit 354.

[0062] The pattern type determination unit 354 may determine the pattern type of each of the received patterns. For example, a pattern associated with a financial service of a specific company (e.g., the LINE Pay service) may be determined as a first pattern type, a pattern associated with a non-financial service of a specific company (e.g., the LINE Messenger service) may be determined as a second pattern type, a two-dimensional pattern associated with services of other companies may be determined as a third pattern type, and a one-dimensional pattern may be determined as a fourth pattern type. In one embodiment, the pattern type determination unit 354 may transmit the pattern or information coded by the pattern to another module (e.g., the LINE Pay application) to determine whether each pattern is associated with a financial service of a specific company (e.g., the LINE Pay service). In another embodiment, the pattern type determination unit 354 may include a module (e.g., the LINE Pay application) for determining whether each pattern is associated with a financial service of a specific company (e.g., the LINE Pay service).

[0063] The pattern processing unit 358 may receive the pattern type of each of the multiple patterns from the pattern type determination unit 354, and compare the priorities of the patterns based on the priorities of each pattern type pre-recorded in the storage unit 360. For example, the storage unit 360 may pre-define the first pattern type, the second pattern type, the third pattern type, and the fourth pattern type in descending order of priority. If it is determined that there is one pattern with the highest priority, the pattern processing unit 358 may automatically process the corresponding pattern without user input.

[0064] On the other hand, if it is determined that there are two or more patterns with the highest priority, the pattern processing unit 358 may calculate a confidence score for each pattern. In one embodiment, the confidence score for a visually coded pattern may be calculated based on at least one of the position or size of the pattern. For example, the confidence score for a visually coded pattern may be calculated based on the distance between the center of the image and the center of the pattern, or based on the percentage of the area that the pattern occupies in the image. The pattern processing unit 358 may then automatically process the pattern with the highest confidence score without user input.

[0065] In another embodiment, if it is determined that there are two or more patterns with the highest priority, the pattern processing unit 358 may provide the pattern with the highest priority to the pattern preview generation unit 356. In this case, the pattern preview generation unit 356 may generate a preview of each pattern and display it on the display 320 to allow the user to select one pattern. In this case, the pattern preview generation unit 356 may calculate a reliability score for each pattern and display a preview of the pattern with the higher reliability score at the top of the preview area on the display 320. If the user selects one of the previews displayed on the display 320 using the input interface 330, the pattern processing unit 358 may process the pattern associated with the preview selected by the user. Alternatively, the preview generation unit 356 may display only the preview of the pattern with the highest reliability score in the preview area on the display 320.

[0066] In another embodiment, when the pattern region detection unit 352 detects multiple patterns within the received image, the detected multiple patterns may be provided to the pattern preview generation unit 356 instead of the pattern type determination unit 354. In this case, the pattern preview generation unit 356 may generate a preview of each pattern and display it on the display 320. For example, the preview of each pattern may be displayed in a preview area on the display 320. In this case, the pattern preview generation unit 356 may determine the display order of the pattern previews based on the priority of the pattern and / or the reliability score of the pattern. When the user selects one of the multiple previews displayed on the display 320, the pattern processing unit 358 may process the pattern associated with the preview selected by the user.

[0067] For example, if the visually coded pattern corresponds to encoding of a Uniform Resource Locator (URL), the pattern preview generator 356 may transmit the corresponding pattern or URL information coded by the corresponding pattern to an external device (e.g., a web server) through the communication module 340. In this case, the user terminal 300 may receive at least one of a thumbnail, a title, or a description associated with the corresponding URL from the external device. The pattern preview generator 356 may generate a preview of the corresponding pattern based on at least one of the thumbnail, the title, or the description received from the external device.

[0068] As another example, if the visually coded pattern corresponds to encoding of text information, the pattern preview generator 356 may generate a visually coded pattern preview based on the corresponding text information. If the visually coded pattern corresponds to encoding of information (e.g., URL information) related to a financial service or a non-financial service of a specific company, the pattern preview generator 356 may generate a preview of a user interface that can execute the financial service or non-financial service of the specific company coded by the corresponding pattern.

[0069] The user terminal 300 may include more components than those shown in FIG. 3. According to an embodiment, the user terminal 300 may further include other components such as a transceiver, a global positioning system (GPS) module, various sensors, a database, etc. For example, if the user terminal 300 is a smartphone, it may include components typically included in a smartphone, and may be implemented to further include various components such as an acceleration sensor, a gyro sensor, various physical buttons, buttons using a touch panel, input / output ports, a vibrator for vibration, etc. In this case, the user terminal 300 may be configured to provide feedback (e.g., haptic feedback) to the user when a pattern is detected from the received image.

[0070] 4 is a diagram illustrating a lookup table 400 that defines a priority for each pattern type according to one embodiment of the present invention. In one embodiment, visually coded patterns may be classified into patterns related to financial services of a specific company (e.g., the LINE Pay service), patterns related to non-financial services of a specific company (e.g., the LINE Messenger service), two-dimensional patterns related to services of other companies, and one-dimensional patterns. According to lookup table 400, the priority is highest in the following order: patterns related to financial services of a specific company, patterns related to non-financial services of a specific company, two-dimensional patterns related to services of other companies, and one-dimensional patterns.

[0071] For example, patterns related to financial services of a particular company may include a pattern for paying expenses (e.g., a store QR code deployed in a store, a QR code used by a user to pay expenses at a store, etc.), a pattern for splitting bills (e.g., a QR code for multiple users to pay expenses in installments), a pattern for transferring expenses, etc. Patterns related to non-financial services of a particular company may include a pattern for adding friends (e.g., a QR code for adding a particular user as a friend), a chat room invitation pattern, an open chat invitation pattern, a pattern for performing logins (e.g., QR code login), etc. Two-dimensional patterns related to other company services may include a two-dimensional pattern for encoding a URL, a two-dimensional pattern for encoding text information, etc.

[0072] FIG. 5 illustrates an example of automatic pattern processing without user input in an embodiment of the present invention. FIG. 5 illustrates a first operational stage 510 and a second operational stage 520 of a user terminal. The first operational stage 510 illustrates automatic selection of a first pattern 512 from multiple patterns 512, 514 included in an image received by an image sensor of the user terminal without user input. The first pattern 512 may be a two-dimensional pattern associated with financial services of a specific company, and the second pattern 514 may be a two-dimensional pattern associated with non-financial services of a specific company. In this case, the first pattern 512 may be determined to have a higher priority than the second pattern 514.

[0073] In the first operation step 510, the user terminal may highlight an area including the first pattern 512 and an area including the second pattern 514 in different colors on the display. Although FIG. 5 illustrates the detected first pattern 512 and second pattern 514 being highlighted in different colors, the present invention is not limited to this, and the first pattern 512 and the second pattern 514 may be displayed in various ways, such as shading, so that the user can easily check each pattern area on the display. In addition, the user terminal may display an indicator 516 on the first pattern 512 to notify the user that processing of the first pattern 512, which has a higher priority, will be automatically executed.

[0074] Second operation stage 520 shows an example screen in which the processing of first pattern 512 is automatically executed without user input. Because first pattern 512 is a pattern related to financial services of a specific company, a screen for providing financial services of the specific company associated with the pattern may be displayed on the display of the user terminal. In this case, the user may proceed with payment by selecting payment button 522 displayed on the display.

[0075] FIG. 6 is a flowchart illustrating a visually coded pattern recognition method 600 according to one embodiment of the present disclosure. Method 600 may begin in step 610 when a user terminal receives an image including a first pattern and a second pattern. For example, suppose the user terminal receives an image including multiple patterns while a pattern scan mode for recognizing visually coded patterns is active. The user terminal may then detect areas in the image that include the visually coded patterns (the first and second patterns) and determine a pattern type for each pattern (step 620). In one embodiment, the user terminal may determine patterns related to financial services of a specific company (e.g., the LINE Pay service) as a first pattern type, patterns related to non-financial services of a specific company (e.g., the LINE Messenger service) as a second pattern type, two-dimensional patterns related to services of other companies as a third pattern type, and one-dimensional patterns as a fourth pattern type.

[0076] In step 630, the user terminal may compare the priority of the pattern type of the first pattern with that of the second pattern. For example, the user terminal may predefine the first pattern type as having the highest priority, followed by the second pattern type, the third pattern type, and the fourth pattern type. Based on the determination result in step 630, the user terminal may determine whether the pattern type of the first pattern and the pattern type of the second pattern have the same priority (step 640).

[0077] If it is determined in step 640 that the pattern type of the first pattern and the pattern type of the second pattern do not have the same priority (i.e., NO in step 640), the user terminal may automatically execute the process of the pattern determined to have the higher priority without user input (step 650). With this configuration, the process of the pattern with the highest priority is automatically executed without the user having to press the camera capture button or select one of the multiple patterns, thereby providing a smooth user experience. On the other hand, if it is determined in step 640 that the pattern type of the first pattern and the pattern type of the second pattern have the same priority (i.e., YES in step 640), method 600 may proceed to method 700 of FIG. 7 or method 800 of FIG. 8.

[0078] FIG. 7 is a flowchart illustrating a method 700 for processing one of patterns having the same priority according to an embodiment of the present disclosure. If it is determined in step 640 of FIG. 6 that the pattern type of the first pattern and the pattern type of the second pattern have the same priority (i.e., YES in step 640), the user terminal may calculate a reliability score for the first pattern (step 710). For example, the reliability score for the first pattern may be calculated based on the result of calculating the distance between the center point of the first pattern and the center point of the received image. In this case, the reliability score for the first pattern may be inversely proportional to the distance between the center point of the first pattern and the center point of the received image. That is, the closer the center point of the visually coded pattern is to the center point of the received image, the higher the reliability score for the pattern may be calculated. This is because a user is likely to hold the user terminal so that a pattern of interest is located closer to the center point of the image.

[0079] Alternatively, the reliability score of the first pattern may be calculated based on the percentage of the area of ​​the area corresponding to the first pattern in the received image. In this case, the reliability score of the pattern may be calculated to be higher as the percentage of the area of ​​the area corresponding to the pattern in the received image increases. This is because a user is more likely to hold the user terminal so that a pattern of interest occupies a higher percentage of the area of ​​the image.

[0080] In step 720, the user terminal may calculate a reliability score for the second pattern. The reliability score for the second pattern may be calculated using the same method as when calculating the reliability score for the first pattern. That is, the reliability score for the second pattern may be calculated based on the result of calculating the distance between the center point of the second pattern and the center point of the received image, or the proportion of the area of ​​the received image that corresponds to the second pattern. In FIG. 7, step 710 of calculating the reliability score for the first pattern is shown to be performed before step 720 of calculating the reliability score for the second pattern, but this is not limited thereto, and the reliability scores of any pattern may be calculated first, or the reliability scores may be calculated simultaneously.

[0081] The user terminal may then compare the calculated confidence score of the first pattern with the confidence score of the second pattern and display a preview of the pattern with the higher confidence score on the display (step 730). For example, if the first pattern and the second pattern have the same priority and the first pattern is located closer to the center of the image than the second pattern, the user terminal may determine that the confidence score of the first pattern is higher than the confidence score of the second pattern and display a preview of the first pattern on the display.

[0082] In one embodiment, if the pattern is related to a financial service of a specific company or a non-financial service of a specific company, the user terminal may display a user interface capable of executing the corresponding service (financial service of a specific company or non-financial service of a specific company) on the display as a preview. If the pattern is a URL type pattern related to services of other companies, the user terminal may receive at least one of a thumbnail, a title, or a description of the URL coded by the corresponding pattern from an external server and display it on the display as a preview. If the pattern codes text information, the user terminal may generate a preview based on the corresponding text information and display it on the display.

[0083] In step 740, the user terminal may receive a user selection for a pattern displayed as a preview. For example, the user selection may be an input for executing processing of a pattern associated with the preview displayed on the display. In this case, the user terminal may execute processing of the pattern selected by the user (step 750). While FIG. 7 illustrates previews of patterns with higher confidence scores being displayed on the display, this is not limiting, and all previews of patterns with the same priority may be displayed on the display. In this case, previews of patterns with higher confidence scores may be displayed at the top of the preview area on the display.

[0084] FIG. 8 is a flowchart illustrating a method 800 for processing one of patterns having the same priority according to an embodiment of the present disclosure. If it is determined in step 640 of FIG. 6 that the pattern type of the first pattern and the pattern type of the second pattern have the same priority (i.e., YES in step 640), the user terminal may calculate a reliability score for the first pattern (step 810). For example, the reliability score for the first pattern may be calculated based on the result of calculating the distance between the center point of the first pattern and the center point of the received image. In this case, the reliability score for the first pattern may be inversely proportional to the distance between the center point of the first pattern and the center point of the received image. That is, the closer the center point of the visually coded pattern is to the center point of the received image, the higher the reliability score for the pattern may be calculated. This is because a user is likely to hold the user terminal so that a pattern of interest is located closer to the center point of the image.

[0085] Alternatively, the reliability score of the first pattern may be calculated based on the percentage of the area of ​​the area corresponding to the first pattern in the received image. In this case, the reliability score of the pattern may be calculated to be higher as the percentage of the area of ​​the area corresponding to the pattern in the received image increases. This is because a user is more likely to hold the user terminal so that a pattern of interest occupies a higher percentage of the area of ​​the image.

[0086] In step 820, the user terminal may calculate a reliability score for the second pattern. The reliability score for the second pattern may be calculated using the same method as when calculating the reliability score for the first pattern. That is, the reliability score for the second pattern may be calculated based on the result of calculating the distance between the center point of the second pattern and the center point of the received image, or the proportion of the area of ​​the received image that corresponds to the second pattern. In FIG. 8, step 810 of calculating the reliability score for the first pattern is shown to be performed before step 820 of calculating the reliability score for the second pattern, but this is not limited thereto, and the reliability scores of any pattern may be calculated first, or the reliability scores may be calculated simultaneously.

[0087] Next, the user terminal may compare the calculated reliability score of the first pattern with the reliability score of the second pattern and automatically process the pattern with the higher reliability score without user input (step 830). For example, if the first pattern and the second pattern have the same priority and the first pattern is located closer to the center of the image than the second pattern, the user terminal may determine that the reliability score of the first pattern is higher than the reliability score of the second pattern and automatically process the first pattern without user input. With this configuration, the pattern with the highest priority and the highest reliability score is automatically processed without the user having to press the camera capture button or select one from multiple patterns, thereby providing a smooth user experience.

[0088] 9 is a flowchart illustrating a method 900 for determining a pattern type of a visually coded pattern according to one embodiment of the present disclosure. For example, when multiple patterns are detected within an image while the pattern scan mode is active, the user terminal may begin determining the pattern type for each pattern (step 910). For one pattern, the user terminal may first determine whether the pattern is associated with a financial service of a particular company (e.g., the LINE Pay service) (step 920).

[0089] If the corresponding pattern is determined to be a pattern related to financial services of a specific company, the user terminal may determine that the corresponding pattern is a first pattern type (step 930). On the other hand, if the corresponding pattern is determined to be a pattern not related to financial services of a specific company, the user terminal may proceed to step 940 and determine whether the corresponding pattern is a pattern related to non-financial services of a specific company (e.g., LINE messenger service). Here, if the corresponding pattern is determined to be a pattern related to non-financial services of a specific company, the user terminal may determine that the corresponding pattern is a second pattern type (step 950). On the other hand, if the corresponding pattern is determined to be a pattern not related to non-financial services of a specific company, the user terminal may proceed to step 960 and determine whether the corresponding pattern is a two-dimensional pattern.

[0090] If the corresponding pattern is determined to be a two-dimensional pattern, the user terminal may determine that the corresponding pattern is a third pattern type (step 970). Conversely, if the corresponding pattern is determined to be neither a two-dimensional pattern nor a one-dimensional pattern (e.g., a one-dimensional pattern), the user terminal may determine that the corresponding pattern is a fourth pattern type (step 980).

[0091] 10 is a diagram illustrating an example of displaying a notification for confirming whether to enter pattern scan mode when a user terminal receives multiple visually coded patterns according to an embodiment of the present disclosure. In the image capture mode, the user terminal may be configured to capture a photo by the user selecting an image capture button 1016, for example, by touch input, while the image sensor is sequentially receiving multiple images. Furthermore, in the pattern scan mode, if a pattern is detected in a received image while the image sensor is sequentially receiving multiple images, the user terminal may be configured to automatically execute the detected pattern without user input or display a preview of the detected pattern on the display. Additionally, if a pattern is detected in a received image while in the image capture mode, the user terminal may confirm with the user whether to enter pattern scan mode.

[0092] A first operational step 1010 shows the user terminal receiving an image including multiple patterns 1012, 1014 via an image sensor in an image capture mode. In this case, the user terminal may detect the multiple patterns 1012, 1014 from the received image. A second operational step 1020 shows the user terminal detecting the multiple patterns 1012, 1014 from the received image in image capture mode and displaying a notification 1022 on the display of the user terminal to confirm whether to enter pattern scan mode. If the user selects the notification 1022 by touch input or the like, the user terminal may enter pattern scan mode and process the multiple patterns 1012, 1014.

[0093] 11 is a diagram illustrating an example of displaying a preview of a pattern selected by a user on a display according to an embodiment of the present disclosure. FIG. 11 illustrates a first operational stage 1110, a second operational stage 1120, and a third operational stage 1130 of a user terminal. In the first operational stage 1110, when the user terminal receives an image including a first pattern 1112 and a second pattern 1114 visually coded by an image sensor while the pattern scan mode is active, the user terminal detects the first pattern 1112 and the second pattern 1114 and displays the detected patterns 1112 and 1114 as selectable options on the display. In this case, the area including the first pattern 1112 and the area including the second pattern 1114 may be highlighted in different colors on the display.

[0094] A second operational step 1120 shows the user terminal receiving a user selection of a first pattern 1112 from the user and displaying a preview 1124 of the first pattern 1112 in a preview area on the display. When the user selects the first pattern 1112, such as by touch input, the user terminal may display an indicator 1122 on the first pattern 1112 indicating that the first pattern 1112 has been selected, and may display a corresponding preview 1124 in the preview area of ​​the display. As shown in the figure, the first pattern 1112 may be a pattern for adding "James" as a friend in a non-financial service (i.e., a LINE messenger service) of a particular company.

[0095] The first pattern preview 1124 may include a LINE Messenger logo, information of the user to be added as a friend (e.g., "James"), an add friend button 1126, and a share button 1128. When the user selects the add friend button 1126 by touch input or the like, the user terminal may execute an add friend function in LINE Messenger. When the user selects the share button 1128 by touch input or the like, the user terminal allows the user to share the first pattern 1112 or the information coded by the first pattern 1112 with other users. In the second operation phase 1120, the first pattern preview 1124 is shown to include, but is not limited to, the LINE Messenger logo, information of the user to be added as a friend (e.g., "James"), the add friend button 1126, and the share button 1128, and may further include other information.

[0096] A third operational step 1130 shows the user terminal receiving a user selection of a second pattern 1114 from the user and displaying a preview 1134 of the second pattern in a preview area on the display. When the user selects the second pattern 1114 by touch input or the like, the user terminal may display an indicator 1132 on the second pattern 1114 indicating that the second pattern 1114 has been selected, and may display a corresponding preview 1134 in the preview area of ​​the display. As shown in the figure, the second pattern 1114 may be a pattern that encodes URL information associated with a service of a company other than a specific company.

[0097] To generate a preview of the second pattern 1114 encoding the URL information, the user terminal may send the pattern to an external device (e.g., a web server) or send the URL information coded by the pattern to an external device. In this case, the user terminal may receive at least one of a thumbnail, a title, or a description associated with the URL from the external device. The preview 1134 of the second pattern may include the thumbnail, the title, the description, and a share button 1136 received from the external device.

[0098] When a user selects the second pattern preview 1134 by touch input or the like, the user terminal may process the URL information coded by the second pattern 1114 in a web browser application or the like. When a user selects the share button 1136 by touch input or the like, the user terminal allows the user to share the second pattern 1114 or the URL information coded by the second pattern 1114 with other users. In the third operation stage 1130, the second pattern preview 1134 is shown to include, but is not limited to, a thumbnail, a title, a description, and the share button 1136, and may further include other information.

[0099] While Figure 11 illustrates a first pattern 1112 and a second pattern 1114 being displayed on the display, this is not intended to be limiting, and if a received image contains three or more patterns, three or more patterns may be displayed as selectable options on the display. Also, while Figure 11 illustrates a user selecting one of the selectable options (patterns) displayed on the display to display a preview of the selected pattern, this is not intended to be limiting, and alternatively, a preview of a pattern with a higher priority may be automatically displayed without user input based on the priority of the patterns in the image. In this case, if the patterns in the image have the same priority, a reliability score may be calculated for each pattern, and a preview of a pattern with a higher reliability score may be automatically displayed without user input.

[0100] 12 is a diagram illustrating an example of displaying previews of multiple patterns included in an image on a display according to an embodiment of the present disclosure. FIG. 12 illustrates a first operational stage 1210 and a second operational stage 1220 of a user terminal. In the first operational stage 1210, when the user terminal is in a pattern scan mode and receives an image including a first pattern 1212 and a second pattern 1214 visually coded by an image sensor, the first pattern 1212 and the second pattern 1214 are detected, and the area including the detected first pattern 1112 and the area including the detected second pattern 1114 are highlighted in different colors on the display.

[0101] A second operation step 1220 shows that, in response to the user terminal detecting the first pattern 1212 and the second pattern 1214 from the received image, the user terminal automatically displays a first pattern preview 1222 and a second pattern preview 1228 in a preview area on the display. As shown in FIG. 12 , the first pattern 1212 may be a two-dimensional pattern for adding “James” as a friend in a non-financial service of a specific company (i.e., the LINE Messenger service), and the second pattern 1214 may be a two-dimensional pattern coded with text. The first pattern preview 1222 may include the LINE Messenger logo, information about the user to be added as a friend (e.g., “James”), an add friend button 1224, and a share button 1226. When the user selects the add friend button 1246 in the first pattern preview 1222 by touch input or the like, the user terminal may execute an add friend function in the LINE Messenger service. When a user selects the share button 1224 in the first pattern preview 1222, such as by touch input, the user terminal allows the user to share the first pattern 1212 or information coded by the first pattern 1212 with other users.

[0102] The second pattern preview 1228 may include text information coded by the pattern, a text copy button 1230, and a share button 1232. When the user selects the text copy button 1230 by touch input or the like, the text information coded by the pattern may be copied to a memory area of ​​the user terminal. When the user selects the share button 1232 by touch input or the like, the user terminal allows the user to share the second pattern 1214 or the text information coded by the second pattern 1214 with other users.

[0103] 12 shows a first pattern preview 1222 and a second pattern preview 1228 displayed on the display, but is not limited to this. If a received image contains three or more patterns, previews for each of the three or more patterns may be displayed on the display. Also, while FIG. 12 shows previews displayed in the order of the patterns in the image, this is not limited to this. Alternatively, a preview of a pattern with a higher priority may be displayed at the top of the preview area based on the priority of the patterns in the image. In this case, if patterns in the image have the same priority, a reliability score may be calculated for each pattern, and the preview of the pattern with the higher reliability score may be displayed at the top of the preview area.

[0104] 13 is a diagram illustrating an example of displaying a preview of a pattern selected by a user on a display according to an embodiment of the present disclosure. FIG. 13 illustrates a first operational stage 1310, a second operational stage 1320, and a third operational stage 1330 of a user terminal. The first operational stage 1310 illustrates detecting the first pattern 1312 and the second pattern 1314 when the user terminal receives an image including visually coded first and second patterns 1312 and 1314 via an image sensor while the user terminal is in a pattern scan mode. In this case, the area including the first pattern 1312 and the area including the second pattern 1314 may be highlighted in different colors on the display.

[0105] A second operational step 1320 depicts the user terminal displaying the detected patterns 1312, 1314 as selectable options 1324, 1326 in an options area 1322 on the display. For example, the selectable options 1324, 1326 may be icons generated by resizing the detected patterns 1312, 1314. The options area 1322 may display all of the detected patterns.

[0106] When the user selects first option 1324, for example, by touch input, a preview 1328 of first pattern 1312 associated with first option 1324 may be displayed in a preview area of ​​the display. As shown in the figure, first pattern 1312 may be a two-dimensional pattern for adding "James" as a friend in a non-financial service of a particular company (i.e., the LINE messenger service). In this case, the selected option (i.e., second option 1326) may be shaded to indicate that first option 1324 has been selected. When the user selects an add friend button in preview 1328 of first pattern 1312 associated with first option 1324, for example, by touch input, the user terminal may execute an add friend function in LINE messenger.

[0107] A third operational step 1330 shows a user selecting a second option 1326, such as by touch input, and a preview 1332 of a second pattern 1314 associated with the second option 1326 is displayed in a preview area of ​​the display. As shown, the second pattern 1314 may be a two-dimensional pattern encoding URL information associated with other company services. In this case, the user terminal may receive at least one of a thumbnail, a title, or a description associated with the URL from an external device and display the at least one of the thumbnail, the title, or the description received from the external device as a preview 1332 of the second pattern 1314 in a preview area of ​​the display. When the user selects the preview 1332 of the second pattern 1314, the user terminal may process the URL information encoded by the second pattern 1314, such as in a web browser application.

[0108] 14 is a flowchart illustrating a visually coded pattern recognition method 1400 according to one embodiment of the present disclosure. Step 1400 may begin with a user terminal receiving an image including a first pattern and a second pattern at step 1410. For example, the image including multiple patterns may be received while the user terminal is in a pattern scan mode for recognizing visually coded patterns.

[0109] The user terminal may then detect an area including the visually coded patterns (first and second patterns) from the image and display the first and second patterns as selectable options on the display (step 1420). In one embodiment, the area including the first pattern and the area including the second pattern may be highlighted in different colors on the display and used as selectable options. In another embodiment, an icon generated by resizing the detected pattern may be displayed in an option area on the display as a selectable option.

[0110] In step 1430, the user terminal may receive a selection for a first pattern from the user. In this case, a preview of the selected first pattern may be displayed in a preview area on the display (step 1440). If a selection for the preview of the first pattern is received from the user in step 1450, the user terminal may execute processing of the first pattern (step 1460).

[0111] Although FIG. 14 shows the first and second patterns being displayed as selectable options in an option area on the display, this is not limiting, and if the received image contains three or more patterns, three or more patterns may be displayed as selectable options in an option area on the display.

[0112] 15 is a flowchart illustrating a visually coded pattern recognition method 1500 according to one embodiment of the present disclosure. Step 1500 may begin with a user terminal receiving an image including a first pattern and a second pattern at step 1510. For example, the image including multiple patterns may be received while the user terminal is in a pattern scan mode for recognizing visually coded patterns.

[0113] The user terminal may then detect the first and second patterns from the received image and display a preview of the first and second patterns in a preview area on the display (step 1520). In one embodiment, the arrangement of the previews of the first and second patterns may be determined by comparing the priority of the pattern type of the first pattern with the pattern type of the second pattern. Alternatively or additionally, the arrangement of the previews of the first and second patterns may be determined by comparing the reliability score of the first pattern with the reliability score of the second pattern.

[0114] The user terminal may receive from the user a selection for a preview of the first pattern displayed on the display (step 1530). In this case, the user terminal may execute processing of the selected first pattern (step 1540). While Figure 15 illustrates previews of the first and second patterns being displayed in a preview area on the display, this is not limiting, and if three or more patterns are included in the received image, previews of three or more patterns may be displayed in the preview area on the display.

[0115] The above-described visually coded pattern recognition method may be realized as computer-readable code (e.g., a computer program) on a computer-readable recording medium. The computer-readable recording medium includes all types of recording devices in which data readable by a computer system is recorded. Examples of the computer-readable recording medium include ROM, RAM, CD-ROM, magnetic tape, floppy disk, and optical data recording device. The computer-readable recording medium may also be distributed among computer systems connected via a network, and the computer-readable code may be recorded and executed in a distributed manner. Furthermore, functional programs, codes, and code segments for implementing the above-described embodiments should be easily construed by a programmer skilled in the art to which the present invention pertains.

[0116] The methods, operations, or techniques of the present disclosure may be implemented by various means. For example, such techniques may be implemented by hardware, firmware, software, or a combination thereof. Those skilled in the art will appreciate that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the present disclosure may also be implemented by electronic hardware, computer software, or a combination thereof. To clearly illustrate this interchangeability between hardware and software, the various illustrative components, blocks, modules, circuits, and steps have been described generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the particular application and design requirements assigned to the overall system. Those skilled in the art may implement the described functionality in various ways for each particular application, but such implementation should not be interpreted as departing from the scope of the present disclosure.

[0117] In a hardware implementation, the processing units utilized to perform the techniques may be implemented within one or more ASICs, DSPs, digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, electronic devices, other electronic units designed to perform the functions described in this disclosure, computers, or combinations thereof.

[0118] Accordingly, the various illustrative logic blocks, modules, and circuits described in connection with this disclosure may be implemented or performed by any combination of general purpose processors, DSPs, ASICs, FPGAs or other programmable logic devices, discrete gate and transistor logic, discrete hardware components, or the like designed to perform the functions described herein. A general purpose processor may be a microprocessor, but alternatively, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented by a combination of computing devices, such as a DSP and a microprocessor, multiple microprocessors, one or more microprocessors in conjunction with a DSP core, or any other configuration.

[0119] In a firmware and / or software implementation, the techniques may be implemented as instructions (e.g., a computer program) recorded on a computer-readable medium such as Random Access Memory (RAM), Read Only Memory (ROM), Non-Volatile Random Access Memory (NVRAM), Programmable Read Only Memory (PROM), Erasable Programmable Read Only Memory (EPROM), Electrically Erasable PROM (EEPROM), flash memory, compact disc (CD), magnetic or optical data storage device, etc. The instructions may be executable by one or more processors and may cause the processor(s) to perform certain aspects of the functions described in this disclosure.

[0120] If implemented in software, the techniques described above may be stored on or transmitted by a computer-readable medium as one or more instructions or code. Computer-readable media includes any medium that facilitates the transfer of a computer program from one place to another, including all computer storage media and communication media. Storage media may be any available medium that can be accessed by a computer. By way of non-limiting example, such computer-readable media may include RAM, ROM, EEPROM, CD-ROM, or other optical disk storage, magnetic disk storage, or other magnetic storage devices, or any other medium that can be used to transport or store desired program code in the form of instructions or data structures and that can be accessed by a computer. Also, any connection is properly termed a computer-readable medium.

[0121] For example, if software is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair wire, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then the coaxial cable, fiber optic cable, twisted pair wire, digital subscriber line, or technologies such as infrared, radio, and microwave are included within the definition of media. As used herein, disk and disc include CDs, laser discs, optical discs, digital versatile discs (DVDs), floppies, and Blu-ray discs, where disks typically reproduce data magnetically and discs reproduce data optically using a laser. Such combinations should also be included within the scope of computer-readable media.

[0122] A software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium may be coupled to the processor such that the processor can read information from, and record information on, the storage medium. Alternatively, the storage medium may be integrated into the processor. The processor and the storage medium may reside in an ASIC. The ASIC may reside in a user terminal. Alternatively, the processor and the storage medium may reside as discrete components in a user terminal.

[0123] While the embodiments described above have been described as utilizing aspects of the presently disclosed subject matter on one or more stand-alone computer systems, the present disclosure should not be limited thereto and may be implemented in connection with any computing environment, such as a network or distributed computing environment. Furthermore, aspects of the subject matter in the present disclosure may be implemented on multiple processing chips or devices, and storage may be affected in the same way across multiple devices. Such devices may include PCs, network servers, and mobile devices.

[0124] Although the present disclosure has been described in connection with some embodiments herein, various modifications and changes that would be readily apparent to those skilled in the art to which the present disclosure pertains can be made without departing from the scope of the present invention, and such modifications and changes must be construed as falling within the scope of the claims appended hereto.

[0125] The following items are further disclosed regarding embodiments of the present invention.

[0126] (1) To the computer, receiving an image including a first visually coded pattern and a second visually coded pattern; determining a pattern type of the visually coded first pattern; determining a pattern type of the visually coded second pattern; comparing the priority of the pattern type of the first visually coded pattern with the priority of the pattern type of the second visually coded pattern; and automatically performing processing of the first visually coded pattern when it is determined that the pattern type of the first visually coded pattern has a higher priority than the pattern type of the second visually coded pattern. A program that executes the following.

[0127] (2) determining the pattern type of the visually coded first pattern includes: determining whether the first visually coded pattern is a pattern associated with a particular service; and determining a pattern type of the visually coded first pattern as a first pattern type when the visually coded first pattern is determined to be a pattern associated with a specific service; The program according to (1), comprising:

[0128] (3) determining the pattern type of the visually coded first pattern includes: determining whether the first visually coded pattern is a pattern associated with a particular company's services; and determining a pattern type of the first visually coded pattern as a second pattern type when the first visually coded pattern is determined to be a pattern related to a service of a specific company; The program according to (2), further comprising:

[0129] (4) determining the pattern type of the visually coded first pattern, If it is determined that the first visually coded pattern is not associated with a particular company's service, determining a pattern type of the first visually coded pattern as a third pattern type if the first visually coded pattern is a two-dimensional pattern; and recognizing a pattern type of the first visually coded pattern as a fourth pattern type when the first visually coded pattern is a one-dimensional pattern; The program according to (3), further comprising:

[0130] (5) The first pattern type, the second pattern type, the third pattern type, and the fourth pattern type have a higher priority in this order. (4) The program described in (4).

[0131] (6) the computer; If it is determined that the pattern type of the first visually coded pattern has the same priority as the pattern type of the second visually coded pattern, calculating a confidence score for the first visually coded pattern based on at least one of a position or a size of the first visually coded pattern within the received image; calculating a confidence score for the second visually coded pattern based on at least one of a position or a size of the second visually coded pattern within the received image; and displaying a preview of the first visually coded pattern on a display of the computer if the confidence score of the first visually coded pattern is determined to be higher than the confidence score of the second visually coded pattern. The program according to any one of (1) to (5), further comprising:

[0132] (7) calculating a confidence score for the first visually coded pattern based on at least one of a position or a size of the first visually coded pattern within the received image, calculating a distance between a center point of the received image and a center point of the first visually coded pattern; (6) The program described in (6).

[0133] (8) calculating a confidence score for the first visually coded pattern based on at least one of a position or a size of the first visually coded pattern in the received image, calculating a percentage of the area of ​​the received image that corresponds to the first visually coded pattern; (6) The program described in (6).

[0134] (9) the computer, If it is determined that the pattern type of the first visually coded pattern has the same priority as the pattern type of the second visually coded pattern, calculating a confidence score for the first visually coded pattern based on at least one of a position or a size of the first visually coded pattern within the received image; calculating a confidence score for the second visually coded pattern based on at least one of a position or a size of the second visually coded pattern within the received image; and automatically performing processing of the first visually coded pattern if it is determined that the reliability score of the first visually coded pattern is higher than the reliability score of the second visually coded pattern. The program according to any one of (1) to (5), further comprising:

[0135] (10) In the computer, and notifying a user that processing of the first visually coded pattern will be automatically performed by displaying an indicator on a first region corresponding to the first visually coded pattern. The program according to any one of (1) to (9), further comprising:

[0136] (11) In the computer, detecting a first region including the first visually coded pattern; detecting a second region including the second visually coded pattern; and highlighting the first region and the second region in different colors; The program according to any one of (1) to (10), further comprising:

[0137] (12) In the computer, receiving a plurality of images in sequence in an image capture mode; and and displaying a notification on a display of the computer to confirm whether to enter a pattern scan mode when the image including the first visually coded pattern and the second visually coded pattern is received. The program according to any one of (1) to (11), further comprising:

[0138] (13) A pattern recognition device, an image sensor configured to receive an image including a first visually coded pattern and a second visually coded pattern; a pattern type determination unit configured to determine a pattern type of the first visually coded pattern and a pattern type of the second visually coded pattern; and a pattern processing unit configured to compare the priority of a pattern type of the first visually coded pattern with that of a pattern type of the second visually coded pattern; Equipped with the pattern processing unit is further configured to automatically perform processing of the first visually coded pattern when it determines that a pattern type of the first visually coded pattern has a higher priority than a pattern type of the second visually coded pattern. Pattern recognition device.

[0139] (14) To the computer, receiving an image including a first visually coded pattern and a second visually coded pattern; generating a preview of the first visually coded pattern; generating a preview of the visually coded second pattern; displaying a preview of the first pattern and a preview of the second pattern in a preview area on a display of the computer; receiving a selection for a preview of the first pattern from a user; and performing an operation on the first visually coded pattern in response to the selection. A program that executes the following.

[0140] (15) The step of displaying the preview of the first pattern and the preview of the second pattern in a preview area on the display of the computer includes: calculating a confidence score for the first visually coded pattern based on at least one of a position or a size of the first visually coded pattern within the received image; calculating a confidence score for the second visually coded pattern based on at least one of a position or a size of the second visually coded pattern within the received image; and and displaying a preview of the first visually coded pattern at an upper end of a preview area on a display of the computer when the reliability score of the first visually coded pattern is determined to be higher than the reliability score of the second visually coded pattern. The program according to (14), comprising:

[0141] (16) In the computer, detecting a first region including the first visually coded pattern; detecting a second region including the second visually coded pattern; and highlighting the first region and the second region in different colors; The program according to (14) or (15), further comprising:

[0142] (17) generating a preview of the visually coded first pattern includes: transmitting URL information coded by the first visually coded pattern to an external device if the first visually coded pattern corresponds to encoding of a URL; receiving at least one of a thumbnail, a title, or a description associated with the URL information from the external device; and displaying at least one of the received thumbnail, title, or description in a preview area on the display. The program according to any one of (14) to (16), further comprising:

[0143] (18) A pattern recognition device, an input interface configured to receive input from a user; an image sensor configured to receive an image including a first visually coded pattern and a second visually coded pattern; a pattern preview generator configured to generate a preview of the first visually coded pattern and a preview of the second visually coded pattern; a display configured to output a preview of the generated first and second visually coded patterns in a preview area; and a pattern processor configured to perform processing on the visually coded first pattern in response to receiving a selection for a preview of the first pattern through the input interface; A pattern recognition device comprising: [Explanation of symbols]

[0144] 110:User 120: User terminal 130: Object 132, 134, 136: Visually coded patterns 210, 220, 230: User terminal 240: Network 250: Server 300: User terminal 310: Image sensor 320: Display 330: Input interface 340: Communication module 350: Processor 352: Pattern area detection unit 354: Pattern type determination unit 356: Pattern preview generation section 358: Pattern processing section 360: Storage area

Claims

1. 1. A pattern recognition method executed in a user terminal, comprising: receiving an image including a first visually coded pattern and a second visually coded pattern; displaying the first visually coded pattern and the second visually coded pattern as selectable options on a display of the user terminal; and displaying a preview of at least one of the first pattern or the second pattern in a preview area on the display. A pattern recognition method comprising:

2. a preview of the first pattern or a preview of the second pattern is simultaneously displayed in the preview area without user input; 2. The pattern recognition method according to claim 1.

3. The step of displaying at least one of the preview of the first pattern or the preview of the second pattern in a preview area on the display includes: determining a pattern type of the first pattern based on a service associated with the first pattern; determining a pattern type of the second pattern based on a service associated with the second pattern; comparing the priority of the pattern type of the first pattern with the pattern type of the second pattern; and and displaying a preview of the first pattern at an upper end of the preview area when it is determined that the pattern type of the first pattern has a higher priority than the pattern type of the second pattern.

3. The pattern recognition method of claim 2, comprising:

4. determining a pattern type of the first visually coded pattern based on a service associated with the first pattern, determining whether the first pattern is a pattern associated with a particular service; and determining a pattern type of the first pattern as a first pattern type when the first pattern is determined to be a pattern associated with a specific service; 4. The pattern recognition method of claim 3, comprising:

5. The step of displaying at least one of the preview of the first pattern or the preview of the second pattern in a preview area on the display includes: If it is determined that the pattern type of the first pattern has the same priority as the pattern type of the second pattern, calculating a confidence score for the first pattern based on at least one of a position or a size of the first pattern within the received image; calculating a confidence score for the second pattern based on at least one of a position or a size of the second pattern within the received image; and displaying a preview of the first pattern on the display if the confidence score of the first pattern is determined to be higher than the confidence score of the second pattern.

4. The pattern recognition method of claim 3, further comprising:

6. The step of displaying at least one of the preview of the first pattern or the preview of the second pattern in a preview area on the display includes: calculating a confidence score for the first pattern based on at least one of a position or a size of the first pattern within the received image; calculating a confidence score for the second pattern based on at least one of a position or a size of the second pattern within the received image; and displaying a preview of the first pattern at an upper end of the preview area when it is determined that the reliability score of the first pattern is higher than the reliability score of the second pattern.

3. The pattern recognition method of claim 2, comprising:

7. receiving a selection from a user for a preview of the first pattern; and performing an operation on the first pattern in response to the selection; The pattern recognition method of claim 2 further comprising:

8. detecting a first region including the first pattern; detecting a second region including the second pattern; and highlighting the first region and the second region in different colors; The pattern recognition method of claim 1 further comprising:

9. The step of displaying at least one of the preview of the first pattern or the preview of the second pattern in a preview area on the display includes: displaying a preview of the first pattern in the preview area in response to a user selecting the first area.

9. The pattern recognition method of claim 8, comprising:

10. The step of generating a preview of the first pattern includes: If the first pattern corresponds to encoding of a Uniform Resource Locator (URL), transmitting URL information coded according to the first pattern to an external device; receiving at least one of a thumbnail, a title, or a description associated with the URL information from the external device; and displaying at least one of the received thumbnail, title, or description in the preview area.

2. The pattern recognition method of claim 1, comprising:

11. A computer program that causes a computer to execute the pattern recognition method according to any one of claims 1 to 10.

12. A pattern recognition device, comprising: display, memory, and at least one processor coupled to the memory and configured to execute at least one computer-readable program contained in the memory; Including, The at least one program receiving an image including a first visually coded pattern and a second visually coded pattern; generating a preview of the first pattern; generating a preview of the second pattern; a command for displaying at least one of the first pattern preview and the second pattern preview in a preview area on a display; Pattern recognition device.

13. a preview of the first pattern or a preview of the second pattern is simultaneously displayed in the preview area without user input; The pattern recognition device according to claim 12.

14. Displaying at least one of the first pattern preview and the second pattern preview in a preview area on the display includes: determining a pattern type of the first pattern based on a service associated with the first pattern; determining a pattern type of the second pattern based on a service associated with the second pattern; comparing the priority of the pattern type of the first pattern with the pattern type of the second pattern; and and displaying a preview of the first pattern at an upper end of a preview area of ​​the display when it is determined that the pattern type of the first pattern has a higher priority than the pattern type of the second pattern.

14. The pattern recognition device according to claim 13, comprising:

15. Displaying at least one of the first pattern preview and the second pattern preview in a preview area on the display includes: calculating a confidence score for the first pattern based on at least one of a position or a size of the first pattern within the received image; calculating a confidence score for the second pattern based on at least one of a position or a size of the second pattern within the received image; and If the confidence score of the first pattern is determined to be higher than the confidence score of the second pattern, a preview of the first pattern is displayed at an upper end of a preview area on the display.

14. The pattern recognition device according to claim 13, comprising:

16. The at least one program receiving a selection from a user for a preview of the first pattern; further comprising instructions for performing a process on the first pattern in response to the selection. The pattern recognition device according to claim 13.

17. The at least one program Detecting a first region including the first pattern; Detecting a second region including the second pattern; further comprising a command to highlight the first region and the second region in different colors. The pattern recognition device according to claim 12.

18. generating a preview of the first pattern, If the first pattern corresponds to encoding of a Uniform Resource Locator (URL), transmitting URL information coded according to the first pattern to an external device; receiving at least one of a thumbnail, a title, or a description associated with the URL information from the external device; and Displaying at least one of the received thumbnail, title, or description in the preview area. The pattern recognition device according to claim 12, comprising:

Images