Visually Coded Pattern Recognition Method and Apparatus

Abstract

To provide a method for recognizing a pattern that a user terminal executes.SOLUTION: The method for recognizing a pattern includes the steps of: receiving an image including a first pattern and a second pattern; determining the pattern type of the first pattern; determining the pattern type of the second pattern; comparing the priorities of the pattern type of the first pattern and the pattern type of the second pattern; and automatically executing processing of the first pattern if the pattern type of the first pattern is determined to have a higher priority than that of the second pattern.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present disclosure relates to a method and apparatus for recognizing visually coded patterns, and more particularly, to a method and apparatus for recognizing and automatically executing a plurality of visually coded patterns, or for displaying a preview of a plurality of visually coded patterns.

Background Art

[0002] Terminals such as smartphones, tablets, PCs, laptop PCs, and desktop PCs with built-in cameras have become widely popular. In such an environment, the use of cameras built into terminals is increasing. As a typical example, many services that utilize barcodes photographed by cameras built into terminals are provided.

[0003] At this time, when a plurality of barcodes are included in an image photographed by a camera built into a terminal, the barcodes may not be recognized, or barcodes that the user does not want may be recognized. In such a case, in order to receive the service desired by the user, the inconvenience of having to photograph the barcode to be recognized again occurs. As a result, it takes time to receive the service desired by the user, and it becomes difficult to provide a smooth user experience.

Summary of the Invention

Problems to be Solved by the Invention

[0004] The present disclosure provides a pattern recognition method, apparatus, and computer program recorded on a recording medium for solving the above-described problems.

[0005] Provided are a method, an apparatus, and a computer program recorded on a recording medium, which receive an image including a plurality of visually-coded patterns at a user terminal, compare the priority orders of the pattern types of the patterns detected from the received image by determining the pattern types of the patterns, and automatically execute the processing of the pattern with a high priority order based on this.

[0006] Provided are a method, an apparatus, and a computer program recorded on a recording medium, which receive an image including a plurality of visually-coded patterns at a user terminal, generate a preview of the detected pattern (one or more patterns) from the received image, and display the preview in a preview area on a display of the user terminal.

Means for Solving the Problem

[0007] The present disclosure can be realized in various ways including a method, an apparatus, or a computer program recorded on a computer-readable recording medium.

[0008] A pattern recognition method executed by a user terminal according to an embodiment of the present disclosure includes the steps of receiving an image including a visually-coded first pattern and a visually-coded second pattern, determining the pattern type of the visually-coded first pattern, determining the pattern type of the visually-coded second pattern, comparing the priority orders of the pattern type of the visually-coded first pattern and the pattern type of the visually-coded second pattern, and automatically executing the processing of the visually-coded first pattern when it is determined that the pattern type of the visually-coded first pattern has a higher priority order than the pattern type of the visually-coded second pattern.

[0009] According to one embodiment, the step of determining the pattern type of the visually coded first pattern includes determining whether the visually coded first pattern is a pattern related to a specific service, and when it is determined that the visually coded first pattern is a pattern related to a specific service, determining the pattern type of the visually coded first pattern as the first pattern type.

[0010] According to one embodiment, the step of determining the pattern type of the visually coded first pattern further includes determining whether the visually coded first pattern is a pattern related to the service of a specific company, and when it is determined that the visually coded first pattern is a pattern related to the service of a specific company, determining the pattern type of the visually coded first pattern as the second pattern type.

[0011] According to one embodiment, when it is determined that the visually coded first pattern is not a pattern related to the service of a specific company, if the visually coded first pattern is a two-dimensional pattern, the step of determining the pattern type of the visually coded first pattern as the third pattern type, and if the visually coded first pattern is a one-dimensional pattern, the step of recognizing the pattern type of the visually coded first pattern as the fourth pattern type are further included.

[0012] According to one embodiment, the first pattern type, the second pattern type, the third pattern type, and the fourth pattern type have a high priority in this order.

[0013] According to one embodiment, when it is determined that the pattern type of the first visually coded pattern has the same priority as the pattern type of the second visually coded pattern, calculating a reliability score of the first visually coded pattern based on at least one of the position or size of the first visually coded pattern in the received image, calculating a reliability score of the second visually coded pattern based on at least one of the position or size of the second visually coded pattern in the received image, and when it is determined that the reliability score of the first visually coded pattern is higher than the reliability score of the second visually coded pattern, further including the step of displaying a preview of the first visually coded pattern on the display of the user terminal.

[0014] According to one embodiment, the step of calculating a reliability score of the first visually coded pattern based on at least one of the position or size of the first visually coded pattern in the received image includes calculating the distance between the center point of the received image and the center point of the first visually coded pattern.

[0015] According to one embodiment, the step of calculating a reliability score of the first visually coded pattern based on at least one of the position or size of the first visually coded pattern in the received image includes calculating the ratio of the area occupied by the region corresponding to the first visually coded pattern in the received image.

[0016] According to one embodiment, when it is determined that the pattern type of the visually coded first pattern has the same priority as the pattern type of the visually coded second pattern, calculating a reliability score of the visually coded first pattern based on at least one of the position or size of the visually coded first pattern in the received image, calculating a reliability score of the visually coded second pattern based on at least one of the position or size of the visually coded second pattern in the received image, and when it is determined that the reliability score of the visually coded first pattern is higher than the reliability score of the visually coded second pattern, further including the step of automatically executing the process of the visually coded first pattern.

[0017] According to one embodiment, further including the step of notifying the user of automatically executing the process of the visually coded first pattern by displaying an indicator on the first area corresponding to the visually coded first pattern.

[0018] According to one embodiment, further including the step of detecting a first area containing the visually coded first pattern, the step of detecting a second area containing the visually coded second pattern, and the step of highlighting the first area and the second area in different colors from each other.

[0019] According to one embodiment, further including the step of sequentially receiving a plurality of images in the image capture mode, and when receiving an image including the visually coded first pattern and the visually coded second pattern, displaying a notification for confirming whether to enter the pattern scan mode on the display of the user terminal.

[0020] Provided is a computer program recorded on a computer-readable recording medium for executing, by a computer, the pattern recognition method described in one embodiment of the present disclosure.

[0021] A pattern recognition device according to an embodiment of the present disclosure includes an image sensor configured to receive an image including a visually coded first pattern and a visually coded second pattern, a pattern type determination unit configured to determine a pattern type of the visually coded first pattern and a pattern type of the visually coded second pattern, and a pattern processing unit configured to compare priorities of the pattern type of the visually coded first pattern and the pattern type of the visually coded second pattern. The pattern processing unit is further configured to automatically execute processing of the visually coded first pattern when it is determined that the pattern type of the visually coded first pattern has a higher priority than the pattern type of the visually coded second pattern.

[0022] A pattern recognition method executed by a user terminal according to an embodiment of the present disclosure includes receiving an image including a visually coded first pattern and a visually coded second pattern, generating a preview of the visually coded first pattern, generating a preview of the visually coded second pattern, displaying the preview of the first pattern and the preview of the second pattern in a preview area on a display of the user terminal, receiving a selection from the user for the preview of the first pattern, and executing processing on the visually coded first pattern in response to the selection.

[0023] According to one embodiment, the step of displaying the preview of the first pattern and the preview of the second pattern in a preview area on the display of the user terminal includes calculating a reliability score of the visually coded first pattern based on at least one of the position or size of the visually coded first pattern in the received image, calculating a reliability score of the visually coded second pattern based on at least one of the position or size of the visually coded second pattern in the received image, and when it is determined that the reliability score of the visually coded first pattern is higher than the reliability score of the visually coded second pattern, displaying the preview of the visually coded first pattern at the upper end of the preview area on the display of the user terminal.

[0024] According to one embodiment, it further includes detecting a first area including the visually coded first pattern, detecting a second area including the visually coded second pattern, and highlighting the first area and the second area with different colors from each other.

[0025] According to one embodiment, the step of generating a preview of the visually coded first pattern further includes, when the visually coded first pattern corresponds to the encoding of a URL (Uniform Resource Locator), transmitting the URL information encoded by the visually coded first pattern to an external device, receiving at least one of a thumbnail, a title, or a description text related to the URL information from the external device, and displaying at least one of the received thumbnail, title, or description text in the preview area on the display.

[0026] Provided is a computer program recorded on a computer-readable recording medium for executing, by a computer, the pattern recognition method described in one embodiment of the present disclosure.

[0027] A pattern recognition device according to an embodiment of the present disclosure includes an input interface configured to receive an input from a user, an image sensor configured to receive an image including a visually coded first pattern and a visually coded second pattern, a pattern preview generation unit configured to generate a preview of the visually coded first pattern and a preview of the visually coded second pattern, a display configured to output the generated previews of the visually coded first pattern and second pattern to a preview area, and a pattern processing unit configured to execute processing on the visually coded first pattern in response to receiving a selection for the preview of the first pattern through the input interface.

Advantages of the Invention

[0028] According to an embodiment of the present disclosure, by automatically executing pattern processing on a plurality of visually coded patterns based on the priority order of pattern types and / or the reliability scores of the patterns, the accuracy of pattern recognition is improved, and a smooth user experience can be provided by making it possible to easily provide the service desired by the user.

[0029] According to an embodiment of the present disclosure, by displaying previews of a plurality of patterns in a preview area on the display of the user terminal, it is possible to enable the user to select a pattern related to the service desired by the user.

[0030] The effects of the present disclosure should not be limited to those described above, and other effects not mentioned will be clearly understood by those skilled in the art from the following description.

Brief Description of the Drawings

[0031] Embodiments of the present disclosure will be described with reference to the accompanying drawings. Although similar reference numerals are used to indicate similar elements, it is not limited thereto.

[0032]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

[0033] Hereinafter, specific contents for implementing the present disclosure will be described in detail with reference to the accompanying drawings. However, in the following description, when there is a risk of unnecessarily obscuring the gist of the present disclosure, specific descriptions of well-known functions and configurations will be omitted.

[0034] In the accompanying drawings, the same or corresponding components are given the same reference numerals. Also, in the following description of the embodiments, redundant descriptions of the same or corresponding components may be omitted. However, even if the description of a component is omitted, it is not intended that this component is not included in one embodiment.

[0035] The advantages and features of the disclosed embodiments, or the methods for achieving them, will become clear by referring to the embodiments described below together with the accompanying drawings. However, the present disclosure should not be limited to the embodiments disclosed below, and can be realized in various different forms. This embodiment is provided only to enable a complete understanding of the present disclosure and to fully inform those with ordinary knowledge in the technical field to which the present disclosure belongs of the scope of the invention.

[0036] First, the terms used in this specification will be briefly explained, and then the disclosed embodiments will be specifically described.

[0037] The terms used in this specification are chosen from commonly used general terms that are currently widely used as much as possible while considering the functions of the present disclosure. However, this may vary depending on the intentions or precedents of those skilled in the relevant art, the emergence of new technologies, etc. In addition, terms arbitrarily selected by the applicant in specific cases are also included. In this case, the meaning thereof will be described in detail in the description of the corresponding invention. Therefore, the terms used in the present disclosure should not be mere names of terms, but should be defined based on the meaning contained in the terms and the overall content of the present disclosure.

[0038] The singular expressions used in this specification include plural expressions unless specifically specified as singular in the context. Also, plural expressions include singular expressions unless specifically specified as plural in the context.

[0039] Throughout the specification, when a part states that a certain component "includes", it means that other components are further included, rather than excluding other components, unless there is a contrary description.

[0040] Also, the terms "module" or "section" used in the specification mean software or hardware components, and the "module" or "section" performs some role. However, the "module" or "section" should not be limited to software or hardware. The "module" or "section" may be configured to exist on an addressable recording medium or to cause a further processor to reproduce. Thus, by way of example, the "module" or "section" may include at least one of software components, object-oriented software components, components such as class components and task components, processes, functions, attributes, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, or variables. The functions provided within a component and the "module" or "section" may be combined into even fewer components and "modules" or "sections" or further separated into additional components and "modules" or "sections".

[0041] According to an embodiment of the present disclosure, a "module" or "unit" may be implemented by a processor and a memory. The "processor" should be broadly interpreted to include general-purpose processors, central processing units (CPUs), microprocessors, digital signal processors (DSPs), controllers, microcontrollers, state machines, etc. In certain environments, the "processor" may mean an application-specific integrated circuit (ASIC), a programmable logic device (PLD), a field-programmable gate array (FPGA), etc. The "processor" may mean, for example, a combination of a DSP and a microprocessor, a combination of multiple microprocessors, a combination of one or more microprocessors coupled to a DSP core, or any other such combination of processing devices. Also, the "memory" should be broadly interpreted to include any electronic component capable of recording electronic information. The "memory" may mean various types of processor-readable media such as RAM (Random Access Memory), ROM (Read Only Memory), NVRAM (Nonvolatile Random Access Memory), PROM (Programmable Read Only memory), EPROM (Erasable Programmable Read Only memory), EEPROM (Electrically Erasable Programmable Read Only memory), flash memory, magnetic or optical data recording devices, registers, etc. If the processor can read information from the memory or record information in the memory, it can be said that the memory and the processor are in an electronic communication state. The memory integrated with the processor is in an electronic communication state with the processor.

[0042] In the present disclosure, an "image" may include an image captured by an image sensor built into a user terminal, and an image captured by an image sensor configured to communicate with the user terminal. Also, the image may include a partial image extracted from the captured video footage.

[0043] In the present disclosure, "visually coded pattern" means a code that displays data or information in a machine-readable visual form. An application program of a device with a built-in camera, such as a pattern reader or a smartphone, is used to scan / read a visually coded pattern. Visually coded patterns include visually coded one-dimensional patterns and visually coded two-dimensional patterns. A visually coded one-dimensional pattern is composed of parallel lines with different widths and intervals, and includes, for example, UPC (Universal Product Code). A visually coded two-dimensional pattern is composed of geometric patterns such as squares, dots, and hexagons, and includes Data Matrix, Maxi Code, QR (Quick Response) code, and the like. In this specification, "visually coded pattern" may also be described as "pattern".

[0044] FIG. 1 is a diagram showing an example of a pattern recognition method executed by a user terminal 120 in an embodiment of the present disclosure. As shown in FIG. 1, the user 110 may hold the user terminal 120 so that the image sensor of the user terminal 120 faces the object 130 in order to scan one of the first pattern 132, the second pattern 134, and the third pattern 136 included in the object 130. For example, the object 130 may be a document printed with a plurality of visually coded patterns. As shown in the figure, the first pattern 132 and the second pattern 134 may be two-dimensional patterns, and the third pattern 136 may be a one-dimensional pattern.

[0045] When the user terminal 120 receives an image including the first pattern 132, the second pattern 134, and the third pattern 136, the user terminal 120 may detect regions including the visually-coded patterns 132, 134, 136 from the image and determine the pattern types of the respective patterns. The user terminal 120 may compare the priorities of the pattern types of the respective patterns and automatically execute the process of the pattern with the highest priority. For example, when the user terminal 120 determines that the priority of the first pattern 132 is higher than the priorities of the second pattern 134 and the third pattern 136, the user terminal 120 may automatically execute the process of the first pattern 132 without user input.

[0046] For this purpose, the user terminal 120 may record priorities for each predefined pattern type in advance. In one embodiment, the user terminal 120 determines that patterns related to financial services of a specific company (e.g., LINE Pay service) are of the first pattern type, patterns related to non-financial services of a specific company (e.g., LINE Messenger service) are of the second pattern type, two-dimensional patterns related to services of other companies are of the third pattern type, and other one-dimensional patterns are of the fourth pattern type. In this case, the user terminal 120 may define in advance that the first pattern type, the second pattern type, the third pattern type, and the fourth pattern type have high priorities in this order.

[0047] If there are multiple patterns of the same pattern type in one image and there is no pattern with a higher priority than the corresponding pattern, the user terminal 120 may provide a preview of the corresponding pattern to the user so that the user can select one pattern to execute (or scan). Alternatively, the user terminal may calculate the confidence score of the corresponding pattern and automatically execute the process of the pattern with the highest confidence score. In this case, the confidence score of each pattern may be calculated based on at least one of the position or size of the pattern in the image.

[0048] In other embodiments, instead of automatically performing processing on one of the plurality of patterns in the image, the user terminal 120 may allow the user 110 to select one pattern from among the plurality of patterns 132, 134, 136 included in the image. In this case, the user terminal 120 may display the patterns 132, 134, 146 detected from the image as selectable options on the display. Additionally or alternatively, the user terminal 120 may generate and display a preview of the patterns 132, 134, 146 detected from the image on the display. In this case, after the user 110 checks the preview of the patterns displayed on the display, the user 110 may select one pattern and perform processing on the corresponding pattern.

[0049] FIG. 2 is a diagram showing a configuration in which a plurality of user terminals 210, 220, 230 are communicably connected to a server 250 to perform pattern recognition in one embodiment of the present disclosure. In FIG. 2, a mobile phone terminal 210, a tablet terminal 220, and a notebook PC 230 are shown as examples of user terminals, but the present disclosure is not limited thereto. The user terminal may be any computer device capable of wired and / or wireless communication, having a built-in image sensor (e.g., a camera) or being communicable with an image sensor, and having an application program installed thereon that can scan visually coded patterns and is executable. For example, the user terminal may include a smart phone, a mobile phone, a navigation device, a PC, a notebook PC, a digital broadcast terminal, a PDA (Personal Digital Assistants), a PMP (Portable Multimedia Player), a tablet, a game console, a wearable device, an IoT (internet of things) device, a VR (virtual reality) device, an AR (augmented reality) device, and the like.

[0050] Also, although FIG. 2 shows three user terminals 210, 220, and 230 communicating with a server 250 via a network 240, it is not limited thereto, and other numbers of user terminals may be configured to communicate with the server 250 via the network 240. There is no limitation on the communication method via the network 240, and it may include a mobile communication network, a wired Internet, a wireless Internet, a broadcast network, a satellite network, short-range wireless communication between user terminals, and the like. For example, the network 240 may include any one or more of networks such as a PAN (personal area network), a LAN (local area network), a CAN (campus area network), a MAN (metropolitan area network), a WAN (wide area network), a BBN (broadband network), and the Internet. Also, the network 240 may include any one or more of network topologies including a bus network, a star network, a ring network, a mesh network, a star-bus network, a tree or hierarchical network, etc., but is not limited thereto.

[0051] According to one embodiment, the server 250 may include one or more server devices and / or databases capable of recording, providing, and executing computer-executable programs and data, or one or more distributed computing devices and / or distributed databases of a cloud computing service infrastructure. In one embodiment, the server 250 may be a web server configured to record at least one of a thumbnail, a title, or a description text of a URL (Uniform Resource Locator).

[0052] In one embodiment, when the pattern corresponds to the encoding of a URL (Uniform Resource Locator), the user terminals 210, 220, 230 may send the corresponding pattern to the server 250 or send the URL information encoded by the pattern to the server 250 in order to generate a preview of the corresponding pattern. In this case, the server 250 may provide at least one of a thumbnail, a title, or a description text related to the corresponding URL to the user terminals 210, 220, 230. The user terminals 210, 220, 230 may display at least one of the received thumbnail, title, or description text on the display as a preview of the corresponding pattern.

[0053] FIG. 3 is a block diagram showing the configuration of a user terminal 300 in one embodiment of the present invention. The user terminal 300 may be any device that is pre-equipped with capture and image processing capabilities. As shown in the figure, the user terminal 300 may include an image sensor 310, a display 320, an input interface 330, a communication module 340, a processor 350, and a storage unit 360.

[0054] The image sensor 310 may be configured to receive one or more images. For example, the image sensor 310 may be a camera module. The images received by the image sensor 310 may be temporarily or permanently stored in the storage unit 360 of the user terminal 300 in the form of electronic files. In one embodiment, the images received by the image sensor 310 may be provided to the processor 350 to determine whether the corresponding image includes a pattern visually encoded. In other embodiments, the image sensor 310 may be configured to respond to receiving an input command from the user through the input interface 330 and store the image in the storage unit 360 or provide it to the processor 350.

[0055] The display 320 may be configured to output information and / or data generated by the user terminal 300. According to one embodiment, the image received by the image sensor 310 may be displayed on the display 320. In addition to the display 320, devices such as speakers, tactile feedback devices, etc. may be provided as output devices together with the display 320.

[0056] The input interface 330 may be configured to allow a user to input information and / or data into the user terminal 300. For example, the input interface 330 may include devices such as a touch screen, buttons, keypads, touch pads, keyboards, microphones, mice, etc. In FIG. 3, the display 320 and the input interface 330 are shown as separate components, but it is not limited thereto, and the display 320 and the input interface 330 may be integrated and provided as an input / output device.

[0057] As shown in FIG. 3, the user terminal 300 may be configured to communicate information and / or data with other devices via the network 370 using the communication module 340. As an example, a request generated by the processor 350 of the user terminal 300 according to the program code recorded in the storage unit 360 may be transmitted to other devices (e.g., servers) via the network 370 under the control of the communication module 340. Conversely, control signals, instructions, contents, files, etc. provided by other devices may be received by the user terminal 300 through the communication module 340. For example, control signals, instructions, contents, files, etc. of other devices received through the communication module 340 may be transmitted to the processor 350 and the storage unit 360.

[0058] Processor 350 may be configured to process instructions of a computer program by performing basic arithmetic, logic, and input / output operations. The instructions may be provided to processor 350 by storage unit 360 or communication module 340. For example, processor 350 may be configured to execute instructions received according to the program code recorded in storage unit 360. In one embodiment, processor 350 may include a pattern area detection unit 352, a pattern type determination unit 354, a pattern preview generation unit 356, and a pattern processing unit 358.

[0059] Storage unit 360 may include any computer-readable recording medium. According to one embodiment, storage unit 360 may include recording devices such as RAM (Random Access Memory), ROM (Read Only Memory), disk drive, SSD (Solid State Drive), flash memory, etc. At least one program code such as an operating system or an application program may be recorded in storage unit 360.

[0060] In one embodiment, image sensor 310 may receive an image including a plurality of patterns. In this case, the image received by image sensor 310 may be output on display 320 and provided to processor 350. Processor 350 provides the received image to pattern area detection unit 352, and pattern area detection unit 352 may detect an area containing a visually coded pattern in the image and determine whether a visually coded pattern is included in the image.

[0061] When the pattern area detection unit 352 determines that the received image does not contain a pattern visually coded therein, the processor 350 may cause the image sensor 310 to receive a subsequent image. When the pattern area detection unit 352 determines that the received image contains one visually coded pattern, the pattern area detection unit 352 may provide the corresponding pattern to the pattern processing unit 358, and the pattern processing unit 358 may execute the processing of the corresponding pattern. Further, when the pattern area detection unit 352 determines that the received image contains a plurality of patterns, the pattern area detection unit 352 may provide the detected plurality of patterns (or an image of the area where the patterns are detected) to the pattern type determination unit 354.

[0062] The pattern type determination unit 354 may determine the pattern type of each of the received plurality of patterns. For example, a pattern associated with a financial service of a specific company (e.g., LINE Pay service) may be determined as the first pattern type, a pattern associated with a non - financial service of a specific company (e.g., LINE Messenger service) may be determined as the second pattern type, a two - dimensional pattern associated with services of other companies may be determined as the third pattern type, and a one - dimensional pattern may be determined as the fourth pattern type. In one embodiment, the pattern type determination unit 354 may transmit the corresponding pattern or the information coded by the corresponding pattern to another module (e.g., LINE Pay application) to determine whether each pattern is a pattern associated with a financial service of a specific company (e.g., LINE Pay service). In other embodiments, the pattern type determination unit 354 may include a module (e.g., LINE Pay application) for determining whether each pattern is a pattern associated with a financial service of a specific company (e.g., LINE Pay service).

[0063] The pattern processing unit 358 may receive the pattern type of each of the plurality of patterns from the pattern type determination unit 354, and compare the priority levels of the patterns based on the priority levels for each pattern type pre-recorded in the storage unit 360. For example, in the storage unit 360, it may be predefined in advance that the first pattern type, the second pattern type, the third pattern type, and the fourth pattern type have high priority levels in this order. When it is determined that there is only one pattern with the highest priority level, the pattern processing unit 358 may automatically execute the processing of the corresponding pattern without user input.

[0064] On the other hand, when it is determined that there are two or more patterns with the highest priority level, the pattern processing unit 358 may calculate the reliability scores of each pattern. In one embodiment, the reliability score of a visually coded pattern may be calculated based on at least one of the position or size of the corresponding pattern. For example, the reliability score of a visually coded pattern may be calculated based on the distance between the center point of the image and the center point of the corresponding pattern, or based on the ratio of the area occupied by the corresponding pattern in the image. After that, the pattern processing unit 358 may automatically execute the processing of the pattern with the highest reliability score without user input.

[0065] In other embodiments, when it is determined that there are two or more patterns with the highest priority, the pattern processing unit 358 may provide the pattern with the highest priority to the pattern preview generation unit 356. In this case, the pattern preview generation unit 356 may generate previews of the respective patterns and display them on the display 320 so that the user can select one pattern. At this time, the pattern preview generation unit 356 may calculate the reliability score of each pattern and display the preview of the pattern with a higher reliability score at the upper end of the preview area on the display 320. When the user selects one of the previews displayed on the display 320 using the input interface 330, the pattern processing unit 358 may execute the process of the pattern associated with the preview selected by the user. Alternatively, the preview generation unit 356 may display only the preview of the pattern with the highest reliability score in the preview area on the display 320.

[0066] In still other embodiments, when the pattern area detection unit 352 detects a plurality of patterns from the received image, it may be configured to provide the detected plurality of patterns to the pattern preview generation unit 356 instead of the pattern type determination unit 354. In this case, the pattern preview generation unit 356 may generate previews of the respective patterns and display them on the display 320. For example, the preview of each pattern may be displayed in the preview area on the display 320. At this time, the pattern preview generation unit 356 may determine the display order of the pattern previews based on the priority of the patterns and / or the reliability score of the patterns. When the user selects one of the plurality of previews displayed on the display 320, the pattern processing unit 358 may execute the process of the pattern associated with the preview selected by the user.

[0067] Exemplarily, when the visually-coded pattern corresponds to the encoding of a URL (Uniform Resource Locator), the pattern preview generation unit 356 may transmit the corresponding pattern or the URL information coded by the corresponding pattern to an external device (e.g., a web server) through the communication module 340. In this case, the user terminal 300 may receive at least one of a thumbnail, a title, or a description text related to the corresponding URL from the external device. The pattern preview generation unit 356 may generate a preview of the corresponding pattern based on at least one of the thumbnail, the title, or the description text received from the external device.

[0068] As another example, when the visually-coded pattern corresponds to the encoding of text information, the pattern preview generation unit 356 may generate a visually-coded pattern preview based on the corresponding text information. When the visually-coded pattern corresponds to the encoding of information (e.g., URL information) related to a financial service of a specific company or a non-financial service of a specific company, the pattern preview generation unit 356 may generate, as a preview, a user interface capable of executing the financial service or the non-financial service of the specific company coded by the corresponding pattern.

[0069] The user terminal 300 may include more components than those shown in FIG. 3. According to one embodiment, the user terminal 300 may further include other components such as a transceiver, a GPS (Global Positioning System) module, various sensors, a database, and the like. For example, when the user terminal 300 is a smartphone, it may include components generally provided in a smartphone. For example, various components such as an acceleration sensor, a gyro sensor, various physical buttons, buttons using a touch panel, an input / output port, and a vibrator for vibration may be further included in the user terminal 300. In this case, when a pattern is detected from the received image, the user terminal 300 may be configured to provide feedback (e.g., tactile feedback) to the user.

[0070] FIG. 4 is a diagram showing a lookup table 400 that defines priorities for each pattern type in one embodiment of the present invention. In one embodiment, visually coded patterns may be classified into patterns related to financial services of a specific company (e.g., LINE Pay service), patterns related to non-financial services of a specific company (e.g., LINE Messenger service), two-dimensional patterns related to services of other companies, and one-dimensional patterns. According to the lookup table 400, the patterns related to financial services of a specific company, the patterns related to non-financial services of a specific company, the two-dimensional patterns related to services of other companies, and the one-dimensional patterns have higher priorities in this order.

[0071] For example, patterns related to the financial services of a specific company may include patterns for paying fees (e.g., store QR codes deployed in stores, QR codes used by users to pay fees in stores, etc.), patterns for split bills (e.g., QR codes for multiple users to split and pay fees), patterns for fee remittance, and the like. Patterns related to the non-financial services of a specific company may include patterns for adding friends (e.g., QR codes for adding a specific user as a friend), chat room invitation patterns, open chat invitation patterns, patterns for performing login (e.g., QR code login), and the like. Two-dimensional patterns related to the services of other companies may include two-dimensional patterns for encoding URLs, two-dimensional patterns for encoding text information, and the like.

[0072] FIG. 5 is a diagram showing an example of automatically executing pattern processing without user input in one embodiment of the present invention. FIG. 5 shows a first operation stage 510 and a second operation stage 520 of a user terminal. The first operation stage 510 indicates that among a plurality of patterns 512 and 514 included in an image received by an image sensor of the user terminal, the first pattern 512 is automatically selected without user input. The first pattern 512 may be a two-dimensional pattern related to the financial services of a specific company, and the second pattern 514 may be a two-dimensional pattern related to the non-financial services of a specific company. In this case, it may be determined that the first pattern 512 has a higher priority than the second pattern 514.

[0073] In the first operation stage 510, the user terminal may highlight, on the display, the area including the first pattern 512 and the area including the second pattern 514 in different colors from each other. Although FIG. 5 shows that the detected first pattern 512 and second pattern 514 are highlighted in different colors from each other, the present invention is not limited thereto, and the first pattern 512 and the second pattern 514 may be displayed in various ways such as shading display so that the user can easily confirm each pattern area on the display. Further, the user terminal may display an indicator 516 on the first pattern 512 in order to notify the user that the process of the first pattern 512 with a higher priority is automatically executed.

[0074] The second operation stage 520 shows an example of a screen in which the process of the first pattern 512 is automatically executed without user input. Since the first pattern 512 is a pattern related to the financial service of a specific company, a financial service providing screen of the specific company related to the corresponding pattern may be displayed on the display of the user terminal. In this case, the user may proceed with the payment of the fee by selecting a payment button 522 displayed on the display.

[0075] FIG. 6 is a flowchart showing a visually coded pattern recognition method 600 according to an embodiment of the present disclosure. At step 610, the method 600 may begin by the user terminal receiving an image including a first pattern and a second pattern. For example, assume that an image including a plurality of patterns is received while the user terminal is in a pattern scan mode for recognizing a visually coded pattern. At this time, the user terminal may detect an area including the visually coded patterns (the first pattern and the second pattern) from the image and determine the pattern type of each pattern (step 620). In one embodiment, the user terminal may determine that a pattern related to a financial service of a specific company (for example, the LINE Pay service) is of the first pattern type, a pattern related to a non-financial service of a specific company (for example, the LINE Messenger service) is of the second pattern type, a two-dimensional pattern related to a service of another company is of the third pattern type, and a one-dimensional pattern is of the fourth pattern type.

[0076] At step 630, the user terminal may compare the priorities of the pattern type of the first pattern and the pattern type of the second pattern. For example, the user terminal may be pre-defined such that the first pattern type, the second pattern type, the third pattern type, and the fourth pattern type are in descending order of priority. Based on the determination result at step 630, the user terminal may determine whether the priorities of the pattern type of the first pattern and the pattern type of the second pattern are the same (step 640).

[0077] In step 640, if it is determined that the priorities of the pattern types of the first pattern and the second pattern are not the same (i.e., NO in step 640), the user terminal may automatically execute the process of the pattern determined to have a higher priority without user input (step 650). With such a configuration, even if the user does not press the camera shooting button or select one of the multiple patterns, the process of the pattern with the highest priority is automatically executed, so that a smooth user experience can be provided. On the contrary, in step 640, if it is determined that the priorities of the pattern types of the first pattern and the second pattern are the same (i.e., YES in step 640), method 600 may proceed to method 700 in FIG. 7 or method 800 in FIG. 8.

[0078] FIG. 7 is a flowchart showing a method 700 for processing one of the patterns with the same priority in an embodiment of the present disclosure. In step 640 of FIG. 6, if it is determined that the priorities of the pattern types of the first pattern and the second pattern are the same (i.e., YES in step 640), the user terminal may calculate the reliability score of the first pattern (step 710). For example, the reliability score of the first pattern may be calculated based on the result of calculating the distance between the center point of the first pattern and the center point of the received image. In this case, the reliability score of the first pattern may be inversely proportional to the distance between the center point of the first pattern and the center point of the received image. That is, the closer the center point of the visually coded pattern is to the center point of the received image, the higher the reliability score of the pattern may be calculated. This is because it is highly likely that the user holds the user terminal so that the pattern of greater interest is arranged closer to the center point of the image.

[0079] Alternatively, the reliability score of the first pattern may be calculated based on the ratio of the area occupied by the region corresponding to the first pattern in the received image. In this case, the larger the ratio of the area occupied by the region corresponding to the pattern in the received image, the higher the reliability score of the pattern may be calculated. This is because the user is likely to hold the user terminal so that the pattern of more interest occupies a higher ratio of the area in the image.

[0080] In step 720, the user terminal may calculate the reliability score of the second pattern. The reliability score of the second pattern may be calculated in the same manner as when calculating the reliability score of the first pattern. That is, based on the result of calculating the distance between the center point of the second pattern and the center point of the received image, or based on the ratio of the area occupied by the region corresponding to the second pattern in the received image, the reliability score of the second pattern may be calculated. In FIG. 7, step 710 of calculating the reliability score of the first pattern is shown to be performed before step 720 of calculating the reliability score of the second pattern, but it is not limited thereto. The reliability score of any pattern may be calculated first, or the reliability scores may be calculated simultaneously.

[0081] Next, the user terminal may compare the calculated reliability score of the first pattern and the reliability score of the second pattern, and display a preview of the pattern with the higher reliability score on the display (step 730). For example, if the first pattern and the second pattern have the same priority and the first pattern is located closer to the center of the image than the second pattern, the user terminal may determine that the reliability score of the first pattern is higher than the reliability score of the second pattern, and display a preview of the first pattern on the display.

[0082] In one embodiment, in the case of a pattern related to a financial service of a specific company or a non-financial service of a specific company, the user terminal may display, as a preview, a user interface capable of executing the corresponding service (the financial service of the specific company or the non-financial service of the specific company) on a display. In the case where the pattern is a URL-type pattern related to a service of another company, the user terminal may receive at least one of a thumbnail, a title, or a description text of a URL coded by the corresponding pattern from an external server and display it on the display as a preview. In the case of a pattern coding text information, the user terminal may generate a preview based on the corresponding text information and display it on the display.

[0083] At step 740, the user terminal may receive a user's selection for the pattern displayed as a preview. For example, the user's selection may be an input for executing processing of a pattern related to the preview displayed on the display. In this case, the user terminal may execute processing of the pattern selected by the user (step 750). In FIG. 7, a preview of a pattern with a higher reliability score is shown to be displayed on the display, but it is not limited thereto, and all previews of patterns with the same priority may be displayed on the display. In this case, a preview of a pattern with a higher reliability score may be displayed at the upper end of the preview area on the display.

[0084] FIG. 8 is a flowchart showing a method 800 for processing one of the patterns having the same priority in one embodiment of the present disclosure. At step 640 in FIG. 6, when it is determined that the priorities of the pattern type of the first pattern and the pattern type of the second pattern are the same (i.e., YES at step 640), the user terminal may calculate the reliability score of the first pattern (step 810). For example, the reliability score of the first pattern may be calculated based on the result of calculating the distance between the center point of the first pattern and the center point of the received image. In this case, the reliability score of the first pattern may be inversely proportional to the distance between the center point of the first pattern and the center point of the received image. That is, the closer the center point of the visually coded pattern is to the center point of the received image, the higher the reliability score of the pattern may be calculated. This is because the user is likely to hold the user terminal so that the more interesting pattern is placed near the center point of the image.

[0085] Alternatively, the reliability score of the first pattern may be calculated based on the ratio of the area occupied by the region corresponding to the first pattern in the received image. In this case, the larger the ratio of the area occupied by the region corresponding to the pattern in the received image, the higher the reliability score of the pattern may be calculated. This is because the user is likely to hold the user terminal so that the more interesting pattern occupies a high ratio of the area within the image.

[0086] At step 820, the user terminal may calculate the reliability score of the second pattern. The reliability score of the second pattern may be calculated in the same manner as when calculating the reliability score of the first pattern. That is, based on the result of calculating the distance between the center point of the second pattern and the center point of the received image, or the ratio of the area occupied by the region corresponding to the second pattern in the received image, the reliability score of the second pattern may be calculated. In FIG. 8, the step 810 of calculating the reliability score of the first pattern is shown to be performed before the step 820 of calculating the reliability score of the second pattern, but it is not limited thereto. The reliability score of any pattern may be calculated first, or the reliability scores may be calculated simultaneously.

[0087] Next, the user terminal may compare the calculated reliability scores of the first pattern and the second pattern, and automatically execute the process of the pattern with the higher reliability score without user input (step 830). For example, when the first pattern and the second pattern have the same priority and the first pattern is located closer to the center of the image than the second pattern, the user terminal may determine that the reliability score of the first pattern is higher than the reliability score of the second pattern, and automatically execute the process of the first pattern without user input. With such a configuration, even if the user does not press the camera shooting button or select one of a plurality of patterns, the process of the pattern with the highest priority and the highest reliability score is automatically executed, so that a smooth user experience can be provided.

[0088] FIG. 9 is a flowchart showing a method 900 for determining the pattern type of a visually coded pattern in an embodiment of the present disclosure. The user terminal may start determining the pattern type for each pattern, for example, when a plurality of patterns are detected from an image during activation of the pattern scan mode (step 910). The user terminal may first determine whether a pattern is related to a financial service of a specific company (for example, the LINE Pay service) for one pattern (step 920).

[0089] If it is determined that the corresponding pattern is a pattern related to the financial services of a specific company, the user terminal may determine that the corresponding pattern is of the first pattern type (step 930). On the contrary, if it is determined that the corresponding pattern is not a pattern related to the financial services of a specific company, the user terminal proceeds to step 940 and may determine whether the corresponding pattern is a pattern related to the non-financial services of a specific company (e.g., LINE Messenger service). Here, if it is determined that the corresponding pattern is a pattern related to the non-financial services of a specific company, the user terminal may determine that the corresponding pattern is of the second pattern type (step 950). On the contrary, if it is determined that the corresponding pattern is not a pattern related to the non-financial services of a specific company either, the user terminal proceeds to step 960 and may determine whether the corresponding pattern is a two-dimensional pattern.

[0090] If it is determined that the corresponding pattern is a two-dimensional pattern, the user terminal may determine that the corresponding pattern is of the third pattern type (step 970). On the contrary, if it is determined that the corresponding pattern is not a two-dimensional pattern either (e.g., a one-dimensional pattern), the user terminal may determine that the corresponding pattern is of the fourth pattern type (step 980).

[0091] FIG. 10 is a diagram showing an example of displaying a notification for confirming whether to enter the pattern scan mode when the user terminal receives a plurality of visually coded patterns in an embodiment of the present disclosure. In the image capture mode, the user terminal may be configured to capture a photo when the user selects the image capture button 1016 by a touch input or the like while receiving a plurality of images in sequence by the image sensor. Also, in the pattern scan mode, the user terminal may be configured to automatically execute the detected pattern or display a preview of the detected pattern on the display without user input when a pattern is detected from the received images while receiving a plurality of images in sequence by the image sensor. Additionally, in the image capture mode, when a pattern is detected from the received images, the user terminal may confirm with the user whether to enter the pattern scan mode.

[0092] The first operation stage 1010 indicates that the user terminal receives an image including a plurality of patterns 1012 and 1014 by the image sensor in the image capture mode. In this case, the user terminal may detect the plurality of patterns 1012 and 1014 from the received image. The second operation stage 1020 indicates that the user terminal detects the plurality of patterns 1012 and 1014 from the received image in the image capture mode and displays a notification 1022 for confirming whether to enter the pattern scan mode on the display of the user terminal. When the user selects the notification 1022 by a touch input or the like, the user terminal may enter the pattern scan mode and process the plurality of patterns 1012 and 1014.

[0093] FIG. 11 is a diagram showing an example of displaying a preview of a pattern selected by a user on a display in one embodiment of the present disclosure. FIG. 11 shows a first operation stage 1110, a second operation stage 1120, and a third operation stage 1130 of a user terminal. The first operation stage 1110 indicates that when the user terminal receives an image including a first pattern 1112 and a second pattern 1114 visually coded by an image sensor while the pattern scan mode is activated, the first pattern 1112 and the second pattern 1114 are detected and the detected patterns 1112, 1114 are displayed on the display as selectable options. In this case, the area including the first pattern 1112 and the area including the second pattern 1114 may be highlighted in different colors on the display.

[0094] The second operation stage 1120 indicates that the user terminal receives a user's selection for the first pattern 1112 from the user and displays a preview 1124 of the first pattern 1112 in a preview area on the display. When the user selects the first pattern 1112 by a touch input or the like, the user terminal may display an indicator 1122 indicating that the first pattern 1112 has been selected on the first pattern 1112 and display a preview 1124 therefor in the preview area of the display. As shown in the figure, the first pattern 1112 may be a pattern for adding "James" as a friend in a non-financial service of a specific company (i.e., the LINE messenger service).

[0095] The preview 1124 of the first pattern may include the logo of the LINE Messenger, information about the user to be added as a friend (e.g., "James"), the friend addition button 1126, and the sharing button 1128. When the user selects the friend addition button 1126 by touch input or the like, the user terminal may execute the friend addition function in the LINE Messenger. When the user selects the sharing button 1128 by touch input or the like, the user terminal enables the user to share the information coded by the first pattern 1112 or the first pattern 1112 with other users. In the second operation stage 1120, the preview 1124 of the first pattern is shown to include the logo of the LINE Messenger, information about the user to be added as a friend ("James"), the friend addition button 1126, and the sharing button 1128, but is not limited thereto and may further include other information.

[0096] The third operation stage 1130 indicates that the user terminal receives the user's selection for the second pattern 1114 from the user and displays the preview 1134 of the second pattern in the preview area on the display. When the user selects the second pattern 1114 by touch input or the like, the user terminal may display an indicator 1132 notifying that the second pattern 1114 has been selected on the second pattern 1114 and display the preview 1134 therefor in the preview area of the display. As shown in the figure, the second pattern 1114 may be a pattern that codes URL information related to services of companies other than a specific company.

[0097] The user terminal may send the corresponding pattern to an external device (e.g., a web server) or send the URL information encoded by the corresponding pattern to the external device in order to generate a preview of the second pattern 1114 that codes the URL information. In this case, the user terminal may receive at least one of a thumbnail, a title, or a description text related to the corresponding URL from the external device. The preview 1134 of the second pattern may include the thumbnail, the title, the description text, and the share button 1136 received from the external device.

[0098] When the user selects the preview 1134 of the second pattern by a touch input or the like, the user terminal may execute the processing of the URL information encoded by the second pattern 1114 in a web browser application or the like. When the user selects the share button 1136 by a touch input or the like, the user terminal enables the user to share the second pattern 1114 or the URL information encoded by the second pattern 1114 with other users. In the third operation stage 1130, the preview 1134 of the second pattern is shown to include the thumbnail, the title, the description text, and the share button 1136, but is not limited thereto, and may further include other information.

[0099] In FIG. 11, although the first pattern 1112 and the second pattern 1114 are shown to be displayed on the display, the present invention is not limited thereto. When three or more patterns are included in the received image, three or more patterns may be displayed as selectable options on the display. Also, in FIG. 11, it is shown that if the user selects one of the selectable options (patterns) displayed on the display, a preview of the corresponding pattern is displayed. However, the present invention is not limited thereto. Alternatively, based on the priority order of the patterns in the image, a preview of the pattern with the highest priority may be automatically displayed without user input. In this case, when the priority orders of the patterns in the image are the same, the reliability scores of each pattern may be calculated, and a preview of the pattern with the highest reliability score may be automatically displayed without user input.

[0100] FIG. 12 is a diagram showing an example of displaying previews of a plurality of patterns included in an image on a display in one embodiment of the present disclosure. FIG. 12 shows a first operation stage 1210 and a second operation stage 1220 of a user terminal. The first operation stage 1210 shows that when the user terminal is in the state of activating the pattern scan mode and receives an image including a first pattern 1212 and a second pattern 1214 visually coded by an image sensor, the first pattern 1212 and the second pattern 1214 are detected, and the areas including the detected first pattern 1112 and the second pattern 1114 are highlighted in different colors on the display.

[0101] The second operation stage 1220 shows that in response to detecting the first pattern 1212 and the second pattern 1214 from the image received by the user terminal, the preview 1222 of the first pattern and the preview 1228 of the second pattern are automatically displayed in the preview area on the display. As shown in FIG. 12, the first pattern 1212 is a two-dimensional pattern for adding "James" as a friend in a non-financial service of a specific company (i.e., the LINE messenger service), and the second pattern 1214 may be a two-dimensional pattern in which text is coded. The preview 1222 of the first pattern may include the logo of the LINE messenger, the information of the user to be added as a friend ("James"), the friend addition button 1224, and the sharing button 1226. When the user selects the friend addition button 1246 in the preview 1222 of the first pattern by touch input or the like, the user terminal may execute the friend addition function in the LINE messenger. When the user selects the sharing button 1224 in the preview 1222 of the first pattern by touch input or the like, the user terminal enables the user to share the first pattern 1212 or the information coded by the first pattern 1212 with other users.

[0102] The preview 1228 of the second pattern may include the text information coded by the pattern, the text copy button 1230, and the sharing button 1232. When the user selects the text copy button 1230 by touch input or the like, the text information coded by the pattern may be copied to the memory area of the user terminal. When the user selects the sharing button 1232 by touch input or the like, the user terminal enables the user to share the second pattern 1214 or the text information coded by the second pattern 1214 with other users.

[0103] In FIG. 12, a preview 1222 of the first pattern and a preview 1228 of the second pattern are shown as being displayed on the display, but it is not limited thereto. When three or more patterns are included in the received image, previews for each of the three or more patterns may be displayed on the display. Further, in FIG. 12, the previews are shown as being displayed in the order of the patterns in the image, but it is not limited thereto. Alternatively, based on the priority order of the patterns in the image, a preview of the pattern with the higher priority order may be displayed at the upper end of the preview area. In this case, when the priority orders of the patterns in the image are the same, a reliability score for each pattern may be calculated, and a preview of the pattern with the higher reliability score may be displayed at the upper end of the preview area.

[0104] FIG. 13 is a diagram showing an example of displaying a preview of a pattern selected by a user on a display in one embodiment of the present disclosure. FIG. 13 shows a first operation stage 1310, a second operation stage 1320, and a third operation stage 1330 of a user terminal. The first operation stage 1310 shows that when the user terminal is in a state of activating the pattern scan mode and receives an image including a first pattern 1312 and a second pattern 1314 visually coded by an image sensor, the first pattern 1312 and the second pattern 1314 are detected. In this case, the area including the first pattern 1312 and the area including the second pattern 1314 may be highlighted in different colors from each other on the display.

[0105] The second operation stage 1320 shows that the user terminal displays the detected patterns 1312, 1314 as selectable options 1324, 1326 in an option area 1322 on the display. For example, the selectable options 1324, 1326 may be icons generated by resizing the detected patterns 1312, 1314. All of the detected patterns may be displayed in the option area 1322.

[0106] When the user selects the first option 1324 by touch input or the like, a preview 1328 of the first pattern 1312 associated with the first option 1324 may be displayed in the preview area of the display. As shown in the figure, the first pattern 1312 may be a two-dimensional pattern for adding "James" as a friend in a non-financial service of a specific company (i.e., the LINE Messenger service). In this case, the selected option (i.e., the second option 1326) may be shaded to indicate that the first option 1324 has been selected. When the user selects the friend addition button in the preview 1328 of the first pattern 1312 associated with the first option 1324 by touch input or the like, the user terminal may execute the friend addition function in the LINE Messenger.

[0107] The third operation stage 1330 indicates that the user selects the second option 1326 by touch input or the like, and a preview 1332 of the second pattern 1314 associated with the second option 1326 is displayed in the preview area of the display. As shown in the figure, the second pattern 1314 may be a two-dimensional pattern for coding URL information related to services of other companies. In this case, the user terminal may receive at least one of a thumbnail, a title, or a description text related to the corresponding URL from an external device, and display at least one of the thumbnail, the title, or the description text received from the external device as the preview 1332 of the second pattern 1314 in the preview area on the display. When the user selects the preview 1332 of the second pattern 1314, the user terminal may execute the processing of the URL information coded by the second pattern 1314 in a web browser application or the like.

[0108] FIG. 14 is a flowchart showing a visually coded pattern recognition method 1400 in an embodiment of the present disclosure. At step 1410, step 1400 may begin when the user terminal receives an image including a first pattern and a second pattern. For example, while the user terminal is activating a pattern scan mode for recognizing a visually coded pattern, an image including a plurality of patterns may be received.

[0109] Thereafter, the user terminal may detect an area including the visually coded patterns (the first pattern and the second pattern) from the image and display them on the display as selectable options (step 1420). In one embodiment, the area including the first pattern and the area including the second pattern may be highlighted in different colors on the display and used as selectable options. In other embodiments, icons generated by resizing the detected patterns may be displayed in an option area on the display as selectable options.

[0110] At step 1430, the user terminal may receive a selection of the first pattern from the user. In this case, a preview of the selected first pattern may be displayed in a preview area on the display (step 1440). At step 1450, if a selection of the preview of the first pattern is received from the user, the user terminal may execute processing of the first pattern (step 1460).

[0111] In FIG. 14, it is shown that the first pattern and the second pattern are displayed in an option area on the display as selectable options, but it is not limited thereto. When the received image includes three or more patterns, three or more patterns may be displayed in the option area on the display as selectable options.

[0112] FIG. 15 is a flowchart showing a visually coded pattern recognition method 1500 according to an embodiment of the present disclosure. At step 1510, step 1500 may begin when the user terminal receives an image including a first pattern and a second pattern. For example, while the user terminal is activating a pattern scan mode for recognizing visually coded patterns, an image including a plurality of patterns may be received.

[0113] Thereafter, the user terminal may detect the first pattern and the second pattern from the received image, and display a preview of the first pattern and a preview of the second pattern in a preview area on the display (step 1520). In one embodiment, the arrangement of the previews of the first pattern and the second pattern may be determined by comparing the priority orders of the pattern types of the first pattern and the second pattern. Alternatively or additionally, the arrangement of the previews of the first pattern and the second pattern may be determined by comparing the confidence scores of the first pattern and the confidence scores of the second pattern.

[0114] The user terminal may receive a selection from the user for the preview of the first pattern displayed on the display (step 1530). In this case, the user terminal may execute the process of the selected first pattern (step 1540). In FIG. 15, the previews of the first pattern and the second pattern are shown to be displayed in the preview area on the display, but it is not limited thereto. When the received image includes three or more patterns, the previews of the three or more patterns may be displayed in the preview area on the display.

[0115] The above-described visually-coded pattern recognition method may be implemented by a computer-readable code (e.g., a computer program) on a computer-readable recording medium. The computer-readable recording medium includes all types of recording devices on which data readable by a computer system is recorded. Examples of the computer-readable recording medium include ROM, RAM, CD-ROM, magnetic tape, floppy (registered trademark) disk, optical data recording device, and the like. Further, the computer-readable recording medium may be distributed over a computer system connected by a network, and the computer-readable code may be recorded and executed in a distributed state. Also, functional programs, codes, and code segments for realizing the above-described embodiments can be easily inferred by a programmer in the technical field to which the present invention pertains.

[0116] The methods, operations, or techniques of the present disclosure may be implemented by various means. For example, such techniques may be implemented by hardware, firmware, software, or combinations thereof. It will be understood by those skilled in the art that the various illustrative logical blocks, modules, circuits, and algorithm steps described in connection with the present disclosure may also be implemented as electronic hardware, computer software, or combinations thereof. For the sake of clearly explaining such a mutual substitution between hardware and software, the various illustrative components, blocks, modules, circuits, and steps have been generally described in terms of their functions. Whether such functions are implemented as hardware or as software depends on the design requirements assigned to a particular application and the overall system. Those skilled in the art can also implement the functions described in various ways for each particular application, but such implementations should not be construed as departing from the scope of the present disclosure.

[0117] In a hardware implementation, the processing unit utilized to execute the technique may be implemented within the scope of one or more ASICs, DSPs, digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, electronic devices, other electronic units designed to execute the functions described in this disclosure, computers, or combinations thereof.

[0118] Accordingly, the various exemplary logical blocks, modules, and circuits described in connection with this disclosure may be implemented or executed by any combination, such as a general-purpose processor, DSP, ASIC, FPGA, or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or those designed to execute the functions described in this application. The general-purpose processor may be a microprocessor, but alternatively, the processor may be any existing processor, controller, microcontroller, or state machine. Further, the processor may be implemented by a combination of computer devices, for example, a DSP and a microprocessor, multiple microprocessors, one or more microprocessors in association with a DSP core, or any other configuration combination.

[0119] In the implementation of firmware and / or software, the technique may be implemented as instructions (e.g., a computer program) recorded on a computer-readable medium such as RAM (Random Access Memory), ROM (Read Only Memory), NVRAM (Non-Volatile Random Access Memory), PROM (Programmable Read Only Memory), EPROM (Erasable Programmable Read Only Memory), EEPROM (Electrically Erasable PROM), flash memory, CD (compact disc), magnetic or optical data storage devices, etc. The instructions may be executable by one or more processors, and the processor(s) may be made to execute specific aspects of the functions described in this disclosure.

[0120] When implemented in software, the techniques described above may be recorded on or transmitted by a computer-readable medium as one or more instructions or code. A computer-readable medium includes any medium that facilitates the transfer of a computer program from one place to another and includes all computer recording media and communication media. A recording medium may be any available medium accessible by a computer. By way of non-limiting example, such computer-readable media may include RAM, ROM, EEPROM, CD-ROM, or other optical disk storage, magnetic disk storage, or other magnetic storage devices, or any other medium that may be used to transfer or record the desired program code in the form of instructions or data structures and includes any other medium accessible by a computer. Also, any connection may suitably be termed a computer-readable medium.

[0121] For example, if software is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair wire, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then the coaxial cable, fiber optic cable, twisted pair wire, digital subscriber line, or technologies such as infrared, radio, and microwave are included within the definition of the medium. As used herein, disk and disc include CD, laser disk, optical disk, DVD (digital versatile disc), floppy, and Blu-ray disk, where disks typically reproduce data magnetically and discs reproduce data optically using a laser. Such combinations must also be included within the scope of computer-readable media.

[0122] A software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, a hard disk, a removable disk, a CD-ROM, or any other known form of recording medium. An exemplary recording medium may be connected to the processor such that the processor can read information from, or write information to, the recording medium. Alternatively, the recording medium may be integrated into the processor. The processor and the recording medium may reside in an ASIC. The ASIC may reside in a user terminal. Alternatively, the processor and the recording medium may reside as separate components in a user terminal.

[0123] The embodiments described above have been described as leveraging aspects of the presently disclosed subject matter in one or more stand-alone computer systems, but the present disclosure should not be so limited and may be implemented in conjunction with any computing environment, such as a network or distributed computing environment. Further, aspects of the subject matter of the present disclosure may be implemented on multiple processing chips or devices, and storage may similarly be affected across multiple devices. Such devices may include a PC, a network server, and a mobile device.

[0124] Although the present disclosure has been described in connection with some embodiments, various modifications and changes can be made within the scope that those skilled in the art of the technical field to which the present invention belongs can easily understand without departing from the scope of the present invention. Also, such modifications and changes must be construed as falling within the scope of the claims appended hereto.

Explanation of Reference Numerals

[0125] 110: User 120: User Terminal 130: Object 132, 134, 136: Visually Coded Patterns 210, 220, 230: User Terminals 240: Network 250: Server 300: User Terminal 310: Image Sensor 320: Display 330: Input Interface 340: Communication Module 350: Processor 352: Pattern Region Detection Unit 354: Pattern Type Determination Unit 356: Pattern Preview Generation Unit 358: Pattern Processing Unit 360: Storage Unit

Claims

1. causing a computer to receive an image including a visually-coded first pattern and a visually-coded second pattern; determining a pattern type of the visually-coded first pattern based on a service associated with the visually-coded first pattern; determining a pattern type of the visually-coded second pattern based on a service associated with the visually-coded second pattern; comparing a priority order of the pattern type of the visually-coded first pattern and the pattern type of the visually-coded second pattern; and automatically executing processing of the visually-coded first pattern when it is determined that the pattern type of the visually-coded first pattern has a higher priority than the pattern type of the visually-coded second pattern A program for causing the above to be executed.

2. The step of determining a pattern type of the visually-coded first pattern based on a service associated with the visually-coded first pattern according to Claim 1 includes determining whether the visually-coded first pattern is a pattern associated with a specific service; and when it is determined that the visually-coded first pattern is a pattern associated with a specific service, determining the pattern type of the visually-coded first pattern as a first pattern type The program according to Claim 1, including the above.

3. The step of determining a pattern type of the visually-coded first pattern based on a service associated with the visually-coded first pattern according to Claim 1 includes determining whether the visually-coded first pattern is a pattern associated with a service of a specific company; and When it is determined that the visually coded first pattern is a pattern related to the service of a specific company, determining the pattern type of the visually coded first pattern as a second pattern type The program according to claim 2, further comprising:

4. Based on the service related to the visually coded first pattern, the step of determining the pattern type of the visually coded first pattern is: When it is determined that the visually coded first pattern is not related to the service of a specific company, When the visually coded first pattern is a two-dimensional pattern, determining the pattern type of the visually coded first pattern as a third pattern type, and When the visually coded first pattern is a one-dimensional pattern, recognizing the pattern type of the visually coded first pattern as a fourth pattern type The program according to claim 3, further comprising:

5. The first pattern type, the second pattern type, the third pattern type, and the fourth pattern type are in a descending order of priority, The program according to claim 4.

6. To the computer, When it is determined that the pattern type of the visually coded first pattern has the same priority as the pattern type of the visually coded second pattern, Calculating a reliability score of the visually coded first pattern based on at least one of the position or size of the visually coded first pattern in the received image, Calculating a reliability score of the visually coded second pattern based on at least one of the position or size of the visually coded second pattern in the received image, and When it is determined that the reliability score of the visually coded first pattern is higher than the reliability score of the visually coded second pattern, displaying a preview of the visually coded first pattern on the computer display The program according to any one of claims 1 to 5, further causing the execution of

7. The step of calculating the reliability score of the visually coded first pattern based on at least one of the position or size of the visually coded first pattern in the received image is including calculating the distance between the center point of the received image and the center point of the visually coded first pattern The program according to claim 6

8. The step of calculating the reliability score of the visually coded first pattern based on at least one of the position or size of the visually coded first pattern in the received image is including calculating the ratio of the area occupied by the region corresponding to the visually coded first pattern in the received image The program according to claim 6

9. On the computer When it is determined that the pattern type of the visually coded first pattern has the same priority as the pattern type of the visually coded second pattern the step of calculating the reliability score of the visually coded first pattern based on at least one of the position or size of the visually coded first pattern in the received image the step of calculating the reliability score of the visually coded second pattern based on at least one of the position or size of the visually coded second pattern in the received image, and When it is determined that the reliability score of the visually coded first pattern is higher than the reliability score of the visually coded second pattern, the step of automatically executing the process of the visually coded first pattern The program according to any one of claims 1 to 5, further causing the above to be executed.

10. To the computer, The step of notifying the user that the process of the visually coded first pattern will be automatically executed by displaying an indicator on the first area corresponding to the visually coded first pattern The program according to any one of claims 1 to 9, further causing the above to be executed.

11. To the computer, The step of detecting a first area including the visually coded first pattern, The step of detecting a second area including the visually coded second pattern, and The step of highlighting the first area and the second area in different colors from each other The program according to any one of claims 1 to 10, further causing the above to be executed.

12. To the computer, The step of sequentially receiving a plurality of images in the image capture mode, and When receiving the image including the visually coded first pattern and the visually coded second pattern, the step of displaying a notification for confirming whether to enter the pattern scan mode on the display of the computer The program according to any one of claims 1 to 11, further causing the above to be executed.

13. A pattern recognition device, An image sensor configured to receive an image including a visually coded first pattern and a visually coded second pattern A pattern type determination unit configured to determine a pattern type of the visually coded first pattern and a pattern type of the visually coded second pattern based on services associated with the visually coded first pattern and services associated with the visually coded second pattern, and a pattern processing unit configured to compare priorities of the pattern type of the visually coded first pattern and the pattern type of the visually coded second pattern are provided, and when the pattern processing unit determines that the pattern type of the visually coded first pattern has a higher priority than the pattern type of the visually coded second pattern, the pattern processing unit is further configured to automatically execute processing of the visually coded first pattern, a pattern recognition device.

Images