Method, device and program for detecting signs using image matching

Abstract

According to an embodiment of the present invention, a landmark detection device using image matching is provided, which includes: a damaged content acquisition unit, which acquires damaged content generated based on original content composed of multiple clipped parts; a pre-processed content generation unit, which generates pre-processed content by merging one or more images included in the damaged content or removing local areas of one or more clipped parts included in the damaged content; a matching content generation unit, which generates matching content with the size or position of one or more clipped parts included in the pre-processed content adjusted by comparing with the one or more clipped parts included in the original content; and a matching unit, which matches the original content with the matching content according to the clipped parts.

Description

Technical Field

[0001] The present invention relates to a method, device and program for detecting a mark using image matching, and more specifically, to a method, device and program for detecting a mark using image matching that compares original content and content damaged during leakage according to cutouts to adjust the size and position of the cutouts included in the leaked content, thereby making it easy to detect a mark added to the leaked content. Background Art

[0002] As the online content market has grown rapidly, the easy duplication of digital data has led to content leakage and the market for unauthorized illegal copying is also expanding. To prevent unauthorized content copying, methods such as adding watermarks to content are used, but there are problems such as watermarks that reduce the visual confirmation of content and the technology required to remove watermarks is not difficult, making it difficult to detect leakers using content that displays watermarks.

[0003] On the other hand, comics generally refer to paintings that depict the appearance of people, animals, things, etc. in a concise and humorous way or in an exaggerated and unrestrained way, with a short short text or a certain plot inserted for reading. Recently, the sale of online comics allows users to obtain pleasure and information by reading comics. For example, Korean Patent Publication No. 10-2011-0123393 (published on November 15, 2011) discloses providing comics in the form of mobile digital content through online direct transactions. Summary of the invention

[0004] An object of the present invention is to compare original content with content damaged during leakage according to clipped parts, so as to generate matching content with the size and position of the clipped parts included in the leaked content adjusted.

[0005] Another object of the present invention is to match matching content with original content to detect a marker from the leaked content.

[0006] According to an embodiment of the present invention, a landmark detection device using image matching includes: a damaged content acquisition unit, which acquires damaged content generated based on original content consisting of multiple clipped parts; a pre-processed content generation unit, which generates pre-processed content by merging one or more images included in the damaged content or removing local areas of one or more clipped parts included in the damaged content; a matching content generation unit, which generates matching content with the size or position of one or more clipped parts included in the pre-processed content adjusted by comparing with the one or more clipped parts included in the original content; and a matching unit, which matches the original content with the matching content according to the clipped parts.

[0007] The present invention may further include a marker detection unit that detects a marker that is not present in the specific cutout portion of the original content but is present in the specific cutout portion of the matching content that matches the specific cutout portion of the original content.

[0008] In the present invention, the matching unit may display a portion where the original content and the matching content are identical with each other in a first color, and may display a portion where the original content and the matching content are different with each other in a second color.

[0009] In the present invention, the matching content generation unit can compare one or more clipped parts included in the original content with one or more clipped parts included in the pre-processed content to determine the clipped part position of each of the one or more clipped parts included in the pre-processed content, the matching content generation unit can generate one or more blank images having the same size as one or more images of the original content, and the matching content generation unit can arrange the one or more clipped parts included in the pre-processed content on the one or more blank images based on the determined clipped part positions.

[0010] In the present invention, the matching content generating unit may compare the cutout portion of the original content with the cutout portion of the damaged content to calculate a scale ratio for each cutout portion, and then adjust the size of one or more cutout portions of the damaged content based on the calculated scale ratio.

[0011] In the present invention, the pre-processed content generating unit may generate a combined image by combining individual images included in the damaged content, and remove an outer area of ​​a cutout portion of the combined image to generate the pre-processed content consisting of only the cutout portion.

[0012] In the present invention, when the content is vertical, the matching content generation unit may calculate the scale ratio based on the horizontal length of the cut portion; when the content is horizontal, the matching content generation unit may calculate the scale ratio based on the vertical length of the cut portion.

[0013] In the present invention, the matching content generating unit may adjust the sizes of other cutout portions of the damaged content based on the calculated scale ratio of the specific cutout portion of the damaged content.

[0014] In the present invention, the mark detection unit may display an enlarged area of ​​the content where the original content matches the matching content, to which the mark can be added.

[0015] According to an embodiment of the present invention, there is provided a method for detecting a mark using image matching, which comprises: a damaged content acquisition step of acquiring damaged content generated based on original content consisting of a plurality of clipped parts; a preprocessed content generation step of generating preprocessed content by merging one or more images included in the damaged content or removing local areas of one or more clipped parts included in the damaged content; a matching content generation step of generating matching content with the size or position of one or more clipped parts included in the preprocessed content adjusted by comparing with the one or more clipped parts included in the original content; and a matching step of matching the original content with the matching content according to the clipped parts.

[0016] The present invention may further include a marker detection step, in which a marker is detected, wherein the marker does not exist in a specific excerpt of the original content but exists in a specific excerpt of the matching content that matches the specific excerpt of the original content.

[0017] In the present invention, the matching content generation step may include the following steps: comparing one or more clipped parts included in the original content with one or more clipped parts included in the pre-processed content to determine a clipped part position for each of the one or more clipped parts included in the pre-processed content; generating one or more blank images having the same size as one or more images of the original content; and configuring the one or more clipped parts included in the pre-processed content on the one or more blank images based on the determined clipped part positions.

[0018] In the present invention, the preprocessed content generation step may include the following steps: merging the individual images included in the damaged content to generate a merged image; and removing the outer area of ​​the cropped portion of the merged image to generate preprocessed content consisting only of the cropped portion.

[0019] In the present invention, in the matching content generation step, after comparing the clipped portion of the original content with the clipped portion of the damaged content and calculating a scale ratio according to the clipped portions, the size of one or more clipped portions of the damaged content may be adjusted based on the calculated scale ratio.

[0020] In addition, a computer program recorded on a computer-readable recording medium is provided for executing the method according to the present invention.

[0021] According to the present invention, a mark showing the viewer information is added to the leaked content, so the content leaker can be detected using the added mark.

[0022] According to the present invention, by generating matching content in which the size and position of a clipped portion included in the leaked content are adjusted, it is possible to provide a highly accurate matching result even when the content is damaged during the leak.

[0023] The present invention can detect a marker by matching the matching content with the original content, thereby obtaining the leaker information by using the detected marker. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 is a diagram showing an example of a network environment according to an embodiment of the present invention.

[0025] Figure 2 It is a block diagram for explaining the internal structure of a user terminal and a server in one embodiment of the present invention.

[0026] Figure 3 is a diagram showing the internal structure of a processor according to an embodiment of the present invention.

[0027] Figure 4 FIG. 1 is a diagram showing a landmark detection method using image matching according to an embodiment of the present invention in chronological order.

[0028] Figure 5 is a diagram for schematically illustrating a process of generating damaged leaked content according to an embodiment of the present invention.

[0029] Figure 6 The figure schematically shows the original content and the damaged content.

[0030] Figure 7 FIG. 1 is a diagram showing an example of merging images of contents according to an embodiment of the present invention.

[0031] Figure 8 FIG. 1 is a diagram showing an example of removing an outer area of ​​a cutout portion according to an embodiment of the present invention.

[0032] Fig. 9 FIG. 1 is a diagram showing an example of adjusting the size of a cutout portion according to an embodiment of the present invention.

[0033] Fig.10 FIG. 1 is a diagram showing an example of adjusting the position of a cutout portion according to an embodiment of the present invention.

[0034] Fig.11 This is a diagram showing an example of matching original content and matching content according to an embodiment of the present invention.

[0035] Fig.12 FIG. 1 is a diagram showing an example of detecting a mark according to an embodiment of the present invention. DETAILED DESCRIPTION

[0036] The detailed description of the present invention below is an example of a specific embodiment that can implement the present invention, and reference can be made to the accompanying drawings. Such an embodiment is sufficiently detailed that a person skilled in the art can implement the present invention. It should be understood that the various embodiments of the present invention, although different, are not mutually exclusive. For example, for the specific shapes, structures and characteristics recorded in this specification, it is possible to change from one embodiment to another embodiment and implement it without departing from the concept and scope of the present invention. In addition, it should be understood that the position or configuration of individual structural elements within each embodiment can also be changed without departing from the concept and scope of the present invention. Therefore, the detailed description below is not intended to limit the present invention, and the scope of the present invention is within the scope of the claims in the claims and all equivalent scopes thereof. Similar figure marks in the drawings are represented by the same or similar structural elements in each figure.

[0037] Figure 1 FIG. 2 is an example showing a network environment according to an embodiment of the present invention.

[0038] Figure 1 The network environment of FIG. 1 shows an example including a plurality of user terminals 110, 120, 130, 140, a server 150, and a network 160. Figure 1 For the purpose of illustrating an example of the present invention, the number of user terminals or the number of servers is not limited to Figure 1 The same number of conditions in .

[0039] The plurality of user terminals 110, 120, 130, 140 may be fixed terminals or mobile terminals implemented by computer devices. The plurality of user terminals 110, 120, 130, 140 may be, for example, smart phones, mobile phones, navigation, computers, notebooks, digital broadcast terminals, PDAs (Personal Digital Assistants), PMPs (Portable Multimedia Players), tablet computers, etc. As an example, the first user terminal 110 may communicate with other user terminals 120, 130, 140 and / or the server 150 via the network 160 in a wireless or wired communication manner.

[0040] The communication method is not limited, and can include not only the communication methods of the communication networks that the application network 160 can include (for example, mobile communication networks, wired Internet, wireless Internet, broadcasting networks), but also short-range wireless communications between devices. For example, the network 160 can include any one or more of the networks such as PAN (personal area network), LAN (local area network), CAN (campus area network), MAN (metropolitan area network), WAN (wide area network), BBN (broadband network), the Internet, etc. In addition, the network 160 can include any one or more network topologies, including bus networks, star networks, ring networks, mesh networks, star bus networks, tree or hierarchical networks, etc., but is not limited thereto.

[0041] The server 150 may be implemented by a computer device or multiple computer devices that communicate with multiple user terminals 110 , 120 , 130 , 140 through the network 160 to provide instructions, codes, files, content, services, etc.

[0042] As an example, the server 150 may provide a file for installing an application to the first user terminal 110 connected via the network 160. In this case, the first user terminal 110 may install the application using the file provided by the server 150. In addition, according to the control of the operating system (OS) and at least one program (such as a browser or an installed application) included in the first user terminal 110, the server 150 is connected to receive the service or content provided by the server 150. For example, if the first user terminal 110 transmits a flag detection request to the server 150 via the network 160 according to the control of the application, the server 150 compares the leaked content with the original content to generate matching content for matching, and may transmit the matching result of the matching content with the original content and the detected flag information to the first user terminal 110, and the first user terminal 110 may display the matching result and the detected flag information according to the control of the application. As another example, the server 150 may also set up a communication session for data transmission and reception, and route the data transmission and reception between the plurality of user terminals 110 , 120 , 130 , and 140 through the set up communication session.

[0043] Figure 2It is a block diagram for explaining the internal structure of a user terminal and a server in one embodiment of the present invention.

[0044] exist Figure 2 In the embodiment, the internal structure of the first user terminal 110 as an example of a user terminal and the internal structure of the server 150 as an example of a server are described. The other user terminals 120, 130, 140 may also have the same or similar internal structure.

[0045] The first user terminal 110 and the server 150 may include memories 211, 221, processors 212, 222, communication modules 213, 223, and input / output interfaces 214, 224. The memories 211, 221, as computer-readable recording media, may include RAM (random access memory), ROM (read only memory), and a permanent mass storage device such as a hard disk drive. In addition, the memories 211, 221 may store an operating system and at least one program code (as an example, a browser that is set in the first user terminal 110 and driven or a code for the above-mentioned application, etc.). Such software structural elements may be loaded from a computer-readable recording medium that is independent of the memories 211, 221 using a drive mechanism. Such independent computer-readable recording media may include computer-readable recording media such as a floppy disk drive, a disk, a tape, a DVD / CD-ROM drive, a memory card, and the like. In another embodiment, the software structural elements may also be loaded into the memory 211, 221 via the communication module 213, 223 instead of the computer-readable recording medium. For example, at least one program may be loaded into the memory 211, 221 based on a program (for example, the above-mentioned application) set by a file provided by a developer or a file distribution system (for example, the above-mentioned server 150) that distributes the installation file of the application via the network 160.

[0046] The processors 212 and 222 may be configured to perform basic calculations, logic, and input / output operations, thereby processing instructions of a computer program. Instructions may be provided to the processors 212 and 222 via the memory 211 and 221 or the communication modules 213 and 223. For example, the processors 212 and 222 may be configured to execute instructions received according to program codes stored in a recording device such as the memory 211 and 221.

[0047] The communication modules 213 and 223 can provide a function for the first user terminal 110 and the server 150 to communicate with each other through the network 160, and can provide a function for communicating with other user terminals (for example, the second user terminal 120) or other servers (for example, the server 150). As an example, the request generated by the processor 212 of the first user terminal 110 according to the program code stored in the recording device such as the memory 211 can be transmitted to the server 150 through the network 160 according to the control of the communication module 213. Conversely, the control signal or instruction, content, file, etc. provided according to the control of the processor 222 of the server 150 can pass through the communication module 223 and the network 160, and be received by the first user terminal 110 through the communication module 213 of the first user terminal 110. For example, the control signal or instruction of the server 150 received through the communication module 213 can be transmitted to the processor 212 or the memory 211, and the content or file, etc. can be stored in the storage medium that the first user terminal 110 can further include.

[0048] The input-output interface 214, 224 may be a unit for interfacing with the input-output device 215. For example, the input device may be a device such as a keyboard or a mouse, and the output device may include a device such as a display for displaying a communication session of an application. As another example, the input-output interface 214 may also be a unit for interfacing with a device that integrates input and output functions into one such as a touch screen. As a more specific example, when the processor 212 of the first user terminal 110 processes the instructions of the computer program loaded into the memory 211, the service screen or content formed by the data provided by the server 150 or the second user terminal 120 may be displayed on the display through the input-output interface 214.

[0049] In addition, in other embodiments, the first user terminal 110 and the server 150 may include Figure 2 However, it is not necessary to clearly illustrate most of the structural elements belonging to the prior art. For example, the first user terminal 110 can be configured to include at least a part of the above-mentioned input and output device 215, or can also include other structural elements such as a transceiver, a GPS (Global Positioning System) module, a camera, various sensors, and a database.

[0050] Figure 3 is a diagram showing the internal structure of a processor according to an embodiment of the present invention.

[0051] The processor 212 may include a web browser or application program that can receive a web page online and output the web page. Figure 3 As shown, the structure of the processor 212 that executes the landmark detection function using image matching according to an embodiment of the present invention may include a damaged content acquisition unit 310, a pre-processed content generation unit 320, a matching content generation unit 330, a matching unit 340, and a landmark detection unit 350. According to the embodiment, the structural elements of the processor 212 may be included in the processor 212 or not included in the processor 212 in an optional manner. In addition, according to the embodiment, in order to present the functions of the processor 212, the structural elements of the processor 212 may be separated or combined.

[0052] Here, the structural elements of the processor 212 can be a presentation unit of different functions of the processor 212 executed by the processor 212 according to instructions provided by the program code stored in the first user terminal 110 (as an example, instructions provided by a web browser driven in the first user terminal 110).

[0053] The processor 212 and the structural elements of the processor 212 can control the first user terminal 110 to execute Figure 4 For example, the processor 212 and the components of the processor 212 may be configured to execute instructions according to the operating system code and at least one program code included in the memory 211.

[0054] Figure 4 FIG. 1 is a diagram showing a method for detecting a landmark using image matching according to an embodiment of the present invention in chronological order. Figure 3 as well as Figure 4 The method, device and program for detecting a mark using image matching of the present invention are described in detail.

[0055] First, the damaged content acquisition unit 310 acquires damaged content generated based on original content composed of a plurality of cutout parts ( S1 ).

[0056] Due to various reasons, the content leaked by the content leaker may be damaged compared to the original content. As an example, the leaked content may be damaged in the following ways: it is enlarged or reduced, or the content has interrupted parts, or the arrangement between the intercepted parts is different from the original content, or part of it is missing, or it is repeated, etc. In this case, the information of the leaker can only be obtained if there is information about the position of the content being viewed where the mark is attached, but in the case where the leaked content is damaged, even if the leaked content is matched with the original content, it is difficult to know the accurate information of the mark position. In this regard, the present application is characterized in that, in order to match the leaked content with the original content, the size or position of the leaked content is adjusted to generate matching content, and the matching content is matched with the original content to detect the mark.

[0057] Figure 5 is a diagram for schematically illustrating a process of generating damaged leaked content according to an embodiment of the present invention.

[0058] exist Figure 5 In the example of , (a) indicates original content, (b) indicates marked content with a mark added to the original content, and (c) indicates leaked content leaked by a user. In addition, (d) to (e) are examples of displays on the screen of the user terminal 110.

[0059] First, refer to Figure 5 (d) The user can select the desired content 51 to be read from the content directory 53. As described above, the content 51 to be read in the present invention may be a content consisting of cutouts 51c1 and 51c2, each of which may be a partial set of series content. For example, Figure 5 The "Internet Friends Episode 5" in the content directory shown may be the 5th episode of a series of content consisting of episodes 1 to 5. The user may select "Internet Friends Episode 5" in the displayed content as the content to be read, and the selected content information and the reader information are received.

[0060] Next, the processors 212 and 222 may generate the marked content 52 based on the viewing target content 51. Marks 52s1 and 52s2 based on the viewer information may be added to each intercepted portion 52c1 and 52c2 of the marked content. That is, when a content viewer who does not have the content publishing right leaks the content without authorization, in order to detect the leaker, the marked content to which the mark based on the leaker information is added may be provided to the user terminal instead of the original content. In addition, the leaker information may be acquired by detecting the mark of the leaked content.

[0061] In more detail, Figure 5As shown in (e), the user terminal may display a mark content 52, and the mark content (b) includes marks 52s1 and 52s2 for detecting the leaker in the original content (a). On the other hand, if the user downloads or takes a screenshot of the mark content 52 displayed on the user terminal and leaks it to his or her SNS account or other places, the leaked content may be damaged. Figure 5 As shown in (f), the content 53 leaked to the user's own SNS account (My Blog) may be content with damaged logo content. This is because in the process of downloading or taking screenshots of the logo content and re-uploading it, some content may be missing, or the size may be changed, or the configuration may be different. In other words, it may be like Figure 5 (c) , compared with the original logo content, the outer part of the first cut-out part 53c1 of the leaked content is cropped, and the middle part of the second cut-out part 53c2 is missing.

[0062] According to the present invention, one of the purposes is to detect the position of the mark by matching the original content with the mark content, and then obtain the information of the leaker according to the position of the mark. Figure 5 In the case where the leaked content is damaged as in (c), it is difficult to detect the exact position of the mark by matching it with the original content. In this regard, in the present invention, in order to detect the mark by matching it with the original content, the size and position of the leaked content are adjusted to generate matching content, and the matching content is matched with the original content to detect the position of the mark.

[0063] Figure 6 The figure schematically shows the original content and the damaged content.

[0064] Figure 6 (a) can be the original content, Figure 6(b) may be the damaged content that is leaked. In more detail, the original content is composed of four images 6a1, 6a2, 6a3, and 6a4, and each image may include a cut portion. At this time, the cut portion may be composed of a cut portion line (Cut line) and the cut portion content inside the cut portion line. In addition, although the damaged content that is leaked may also include four images 6b1, 6b2, 6b3, and 6b4, only three images 6b1, 6b3, and 6b4 in the damaged content that is leaked include a cut portion. That is, the cut portion of the second cut portion 6a2 in the damaged content is missing, and the damaged content actually consists of three cut portions. In more detail, the cut portion of the first image 6b1 in the damaged content that is leaked is formed by the middle cut of the cut portion of the first image 6a1 in the original content, and the cut portion is misaligned. In addition, the third image 6b3 of the damaged content that is leaked is formed by keeping the third image 6a1 of the original content unchanged. In addition, the cutout portion of the fourth image 6b4 of the leaked damaged content is a cutout portion in which the middle portion of the cutout portion of the fourth image 6a4 of the original content is partially overlapped and missing. That is, the cutout portion of the damaged content may be formed by deforming the cutout portion of the original content. In the following description, a method of adjusting the damaged content to generate matching content is described.

[0065] First, the preprocessing content generation unit 320 merges one or more images included in the damaged content, or removes local areas of one or more clipped parts included in the damaged content to generate preprocessing content (S2). In more detail, the preprocessing content generation unit 320 merges the single images included in the damaged content to generate a merged image, and removes the outer area of ​​the clipped part of the merged image to generate preprocessing content consisting only of the clipped part. According to an embodiment of the present invention, the preprocessing content generation unit 320 performs preprocessing such as merging images and removing blank spaces outside the clipped parts, so that matching with the original image can be achieved more quickly.

[0066] In more detail, in order to merge more than one image included in the damaged content, the pre-processed content generation unit 320 may splice the images horizontally or vertically in sequence. When the content is expanded vertically, the images are spliced ​​vertically, and when the content is expanded horizontally, the images are spliced ​​horizontally. At this time, the pre-processed content generation unit 320 may also splice the multiple images included in the original content horizontally or vertically.

[0067] Figure 7 FIG. 1 is a diagram showing an example of merging images of contents according to an embodiment of the present invention.

[0068] Reference Figure 7 , showing Figure 6The first to fourth images 6b1 to 6b4 of the damaged content are shown as a combined content 7b which is vertically spliced. Figure 7 It can be seen that there is no gap between the images of the combined content 7b. In addition, the pre-processed content generation unit 320 may also vertically splice the images of the first to fourth images 6a1 to 6a4 of the original content to generate the combined content 7a.

[0069] In addition, the preprocessed content generation unit 320 may also generate preprocessed content consisting of only the cutouts by removing the outer regions of one or more cutouts included in the damaged content. More specifically, the preprocessed content generation unit 320 may remove the regions outside the cutout lines of each image to which the damaged content belongs. If there is no cutout belonging to the damaged content, the preprocessed content generation unit 320 may remove the corresponding image itself. Furthermore, the preprocessed content generation unit 320 may remove the outer regions of the multiple cutouts included in the original content.

[0070] According to an embodiment of the present invention, the content composed of the clipped parts has a high probability that the area outside the clipped part line is a blank space that does not include the content due to its characteristics, so in order to generate the matching content, the blank space can be removed. In this case, the outer area of ​​the clipped part can generally refer to the outer area of ​​the clipped part line. The clipped part line can refer to an area divided by the author to represent each scene in the content composed of the clipped part group, such as in a comic.

[0071] Figure 8 FIG. 1 is a diagram showing an example of removing an outer area of ​​a cutout portion according to an embodiment of the present invention.

[0072] Reference Figure 8 , can be obtained by Figure 7 The preprocessed content is generated by removing the outer area of ​​the cutout part of the merged content 7a, 7b generated by this embodiment. In more detail, the outer area of ​​the cutout part of the merged content 7b based on the damaged content, that is, the outer area of ​​the cutout line, can be removed to generate the preprocessed content 8b composed of the cutout parts 8b1, 8b3, and 8b4. At this time, like the first cutout part of the merged content 7b, when the cutout part is divided into several blocks and the alignment of the blocks is not accurate, it is possible to align them like the first cutout part 8b1 while generating the preprocessed content 8b. Similarly, the preprocessed content 8a composed of the cutout parts 8a1, 8b2, 8b3, and 8b4 can be generated by removing the outer area of ​​the cutout part of the merged content 7a based on the original content.

[0073] Next, the matching content generation unit 330 generates matching content in which the size or position of the one or more clipped parts included in the preprocessed content is adjusted by comparing the clipped parts with the one or more clipped parts included in the original content (S3). More specifically, the matching content generation unit 330 may compare the clipped parts of the original content with the clipped parts of the damaged content to calculate a scale ratio, and then adjust the size of the clipped parts of the damaged content based on the calculated scale.

[0074] According to an embodiment of the present invention, preprocessed content is generated in the preprocessing process, in which only the pure clipped portion exists after the outer area of ​​the clipped portion is removed. Therefore, the size of the clipped portion of the damaged content can be made consistent with the size of the clipped portion of the original content by comparing the size of the clipped portion included in the preprocessed content. For example, the size of the first clipped portion of the preprocessed content based on the original content can be compared with the size of the first clipped portion of the preprocessed content based on the damaged content to calculate the scale ratio, and the size of the first clipped portion of the damaged content can be adjusted based on the scale ratio.

[0075] In an optional embodiment, the matching content generation unit 330 may calculate the scale ratio based on the horizontal length or vertical length of the cut portion according to the content type. More specifically, when the original content is vertical content, the matching content generation unit 330 calculates the scale ratio based on the horizontal length of the cut portion, and when the original content is horizontal content, the matching content generation unit 330 calculates the scale ratio based on the vertical length of the cut portion. This is because, when the original content is vertical content, there is a high possibility that the content is missing or overlapped in the horizontal direction ( Figure 6 In the example of FIG. 6A , the first image 6b1 or the cutout portion of the second image 6b4 is displayed. When the content is horizontal, there is a high possibility that the content will be missing or overlapped in the vertical direction.

[0076] In an optional embodiment, the matching content generation unit 330 may adjust the size of other cutouts of the damaged content based on the calculated scale ratio of a specific cutout. For example, the matching content generation unit 330 may adjust the size of other cutouts of the damaged content based on the calculated scale ratio of the cutout with the smallest deformation. This is because the scale ratios of the cutouts in the damaged content are almost the same or are likely to be the same between the cutouts, so the scale ratios of other cutouts are also determined based on the scale ratio of one cutout, thereby reducing the effort of calculating the scale ratios of all cutouts.

[0077] Fig. 9 FIG. 1 is a diagram showing an example of adjusting the size of a cutout portion according to an embodiment of the present invention.

[0078] Reference Fig. 9 , showing that Figure 8 The cutouts 8a1, 8a2, 8a3, 8a4 based on the original content and the cutouts 8b1, 8b3, 8b4 based on the damaged content are shown. According to an embodiment of the present invention, the scale ratio of the cutouts based on the original content and the cutouts based on the damaged content can be calculated according to the cutouts. At this time, Fig. 9 Since the content is vertically expanded, the scale ratio can be calculated based on the horizontal length of the cut portion. For example, the scale ratio of the original content and the first cut portions 8a1 and 8b1 of the damaged content can be 1:0.5. That is, when the leaker leaks the damaged content, it is possible that the size of the first cut portion is reduced to half and leaked. In this case, the size of the first cut portion 8b1 of the damaged content can be doubled to generate the resized first cut portion 9b1, thereby making it consistent with the size of the first cut portion 8a1 of the original content.

[0079] On the other hand, according to an embodiment of the present invention, Fig. 9 In the embodiment of the present invention, the scale ratio of the third cut-out portion 8b3 of the damaged content with the least degree of damage is used as a reference to adjust the sizes of the cut-out portions of other damaged content. For example, when the scale ratio between the original content and the third cut-out portions 8a1 and 8b1 of the damaged content is 1:0.25, the sizes of the cut-out portions 8b1, 8b3, and 8b4 of the damaged content can all be enlarged to four times to generate the cut-out portions 9b1, 9b3, and 9b4 with adjusted sizes.

[0080] In addition, the matching content generation unit 330 compares the clipped portion of the original content with the clipped portion of the preprocessed content, and determines the position of the clipped portion on the image according to the clipped portion of the damaged content. In this case, the clipped portion of the preprocessed content may be the clipped portion adjusted in size as described above. When comparing the clipped portion of the preprocessed content with the clipped portion of the original content, it can be determined where the clipped portion is located in the image. Figure 6 As shown, the original content of the present invention may be composed of more than one image including the clipped portion, and the position of the clipped portion on the image is also one of the main features of the content. Therefore, according to an embodiment of the present invention, by comparing the clipped portion of the preprocessed content with the clipped portion of the original content, the position of the clipped portion on the image can be determined according to each clipped portion.

[0081] In addition, the matching content generation unit 330 generates a blank image of the same size as the image of the original content, and arranges the clipped portion of the preprocessed image on the generated blank image based on the determined clipped portion position. At this time, the clipped portion missing in the damaged content can be left as a blank image.

[0082] Fig.10 FIG. 1 is a diagram showing an example of adjusting the position of a cutout portion according to an embodiment of the present invention.

[0083] Reference Fig.10 , showing that Fig. 9 9b1, 9b3, 9b4 of the resized preprocessed content. In addition, images 6a1, 6a2, 6a3, 6a4 of the original content are shown. The first image 6a1 of the original content includes a cutout portion that is arranged to the left. To reflect this, according to one embodiment of the present invention, the first cutout portion 9b1 of the preprocessed content is compared with the cutout portion of the first image 6a1 of the original content, and the accurate position of the first cutout portion 9b1 is determined in such a way that the cutout portion is arranged to the left of the image.

[0084] In addition, a blank image identical to the original content images 6a1, 6a2, 6a3, 6a4 is generated, and the clipped parts 9b1, 9b3, 9b4 are arranged on the blank image based on the determined positions to generate matching content images 10b1, 10b2, 10b3, 10b4. Fig.10 As shown, since there is no cutout portion of the damaged content corresponding to the second image 10b2, the second image 10b2 is left as a blank image. In addition, referring to the first image 10b1, even if the middle part of the cutout portion is cut off like the first cutout portion 9b1, by adjusting the position, a cutout portion that is continuous with the first cutout portion 6a1 of the original content can be generated. In addition, referring to the fourth image 10b4, the fourth cutout portion 9b4 can be arranged on the blank image to reflect the part of the fourth cutout portion 9b4 that is missing due to overlap. In this way, matching content can be generated with the size and position of the cutout portion adjusted.

[0085] Next, the matching unit 340 matches the original content with the matching content according to the intercepted portion (S4). Matching the original content with the matching content means comparing the original content with the matching content according to the intercepted portion to determine whether there are other parts. In addition, the matching unit 340 matches the original content with the matching content according to the intercepted portion, and displays the same part and the different part of the original content and the matching content in different ways and provides them to the user terminal. In an optional embodiment, the matching unit 340 represents the same part of the original content and the matching content with a first color, and represents the different part of the original content and the matching content with a second color. For example, the same part of the original content and the matching content is represented in gray, and the different part is represented in black, so as to be provided to the user.

[0086] Fig.11 This is a diagram showing an example of matching original content and matching content according to an embodiment of the present invention.

[0087] Reference Fig.11 , (a) is an example of original content, and (b) is an example of matching content. In (a) as the original content, the position of the additional mark can be specified, where the mark is used to indicate a string that can represent the leaker information. For example, Fig.11 As shown in (a), the 11a1 region can correspond to the letter A, and the other regions can also correspond to the letters B to I. Fig.11 For the sake of convenience, only the area corresponding to letters A to I is shown in the figure, but the area corresponding to all letters and numbers can be specified in one cutout. The specified area is stored in an independent management system and is not shown in the original text image.

[0088] In addition, the matching content (b) is the content after the size and position of the damaged content generated based on the marker content are adjusted, so the marker 11b1 may be added. According to one embodiment of the present invention, by matching the original content and the matching content according to the cutout portion, it is possible to detect the marker that does not exist in the original content but exists in the matching content. For this purpose, it is possible to generate Fig.11 (c) such original content matches the matching content. According to an optional embodiment of the present invention, in the image Fig.11 (c) When such matched content is provided to the user, the portion of the original content that is consistent with the matching content is represented in gray or not represented, and only the portion of the original content that is inconsistent with the matching content is represented in white or black, so that the different portions can be easily detected.

[0089] Next, the mark detection unit 350 detects a mark that does not exist in a specific cutout portion of the original content but exists in a specific cutout portion of the matching content according to the cutout portion of the original content and the matching content (S5). The mark exists only in the specific cutout portion of the matching content and does not exist in the corresponding specific cutout portion of the original content because the matching content is a mark content obtained by attaching a mark for detecting a leaker to the original content. Therefore, by detecting such a mark and referring to the position where the mark is attached, the leaker information can be obtained.

[0090] Next, refer to Fig.11 The example shows that the mark (11b1) displayed in the matching content (b) can represent the information of the leaker. Fig.11 If the original content is matched with the matching content as in (c), there is no mark in the area corresponding to 11c1 of the original content, but there is a mark in the area corresponding to 11c2 of the original content, so the matching content can represent the leaker information corresponding to the letter F.

[0091] Fig.12 FIG. 1 is a diagram showing an example of detecting a mark according to an embodiment of the present invention.

[0092] Fig.12 (a) to (i) are examples showing how the area corresponding to letters A to I is magnified after matching a specific excerpt of the original content with a specific excerpt of the matching content according to one embodiment of the present invention. That is, according to an embodiment of the present invention, after the position to which the mark is attached is magnified and analyzed and the mark is detected, the corresponding leaker information can be determined. According to other embodiments of the present invention, the area corresponding to letters A to I can be magnified and displayed like a picture according to the user's operation. Fig.12 When users see Fig.12 After looking at the images (a) to (i), it is found that there is a dot on (f), so it can be inferred that the information of the leaker corresponding to the corresponding intercepted part is the letter F.

[0093] The embodiments of the present invention described above can be implemented in the form of a computer program that can be executed on a computer through a variety of structural elements, and such a computer program can be recorded in a computer-readable medium. At this time, the medium can always store a program that can be executed by a computer, or store the program when it is used for execution or downloading. In addition, the medium can be a variety of recording units or storage units in a single or multiple hardware combination mode, and is not limited to a medium directly connected to a certain computer system, and can also be a medium that exists dispersed on a network. As an example of a medium, it can include magnetic media such as hard disks, floppy disks, and magnetic disks, optical recording media such as CD-ROMs and DVDs, optical magnetic media such as floppy disks (floptical disks), and ROMs, RAMs, flash memories, and is configured to store program instructions. In addition, as other examples of the medium, it can also be listed as a recording medium or storage medium managed by an application store that sells application programs or a web page, server, etc. that provides and sells a variety of other software.

[0094] The present invention is described above through limited embodiments of specific matters such as specific structural elements and drawings. These are only provided to help better understand the overall invention. The present invention is not limited to the above embodiments, and those skilled in the art can obtain various modifications and changes based on the above description.

[0095] Therefore, the concept of the present invention should not be limited to the embodiments described above, and the appended claims and all scopes equivalent to the claims or modified in an equivalent manner belong to the concept of the present invention.

Claims

1. A logo detection device using image matching, comprising: a damaged content acquisition unit that acquires damaged content generated based on original content composed of a plurality of intercepted parts; a preprocessed content generation unit that generates preprocessed content by merging one or more images included in the damaged content or removing local regions of one or more intercepted parts included in the damaged content; a matching content generation unit that generates matching content with the size or position of one or more intercepted parts included in the preprocessed content adjusted by comparing with one or more intercepted parts included in the original content; a matching unit that matches the original content and the matching content by intercepted part; and a logo detection unit that detects a logo that does not exist in a specific intercepted part of the original content but exists in a specific intercepted part of the matching content that matches the specific intercepted part of the original content.

2. The logo detection device using image matching according to claim 1, wherein the matching unit represents the same parts of the original content and the matching content with a first color and represents the different parts of the original content and the matching content with a second color.

3. The logo detection device using image matching according to claim 1, wherein the matching content generation unit compares one or more intercepted parts included in the original content with one or more intercepted parts included in the preprocessed content to determine the intercepted part position of each intercepted part among the one or more intercepted parts included in the preprocessed content, the matching content generation unit generates one or more blank images having the same size as one or more images of the original content, and the matching content generation unit arranges the one or more intercepted parts included in the preprocessed content on the one or more blank images based on the determined intercepted part positions.

4. The logo detection device using image matching according to claim 1, wherein after the matching content generation unit calculates a scale ratio by intercepted part by comparing the intercepted part of the original content with the intercepted part of the damaged content, the matching content generation unit adjusts the size of one or more intercepted parts of the damaged content based on the calculated scale ratio.

5. The logo detection device using image matching according to claim 1, wherein the preprocessed content generation unit merges single images included in the damaged content to generate a merged image, and removes the external regions of the intercepted parts of the merged image to generate preprocessed content composed only of intercepted parts.

6. The logo detection device using image matching according to claim 4, wherein when it belongs to vertical type content, the matching content generation unit calculates the scale ratio based on the horizontal length of the intercepted part; when it belongs to horizontal type content, the matching content generation unit calculates the scale ratio based on the vertical length of the intercepted part.

7. The logo detection device using image matching according to claim 4, wherein The matching content generating unit adjusts the size of another cutout portion of the damaged content based on the calculated scale ratio of the specific cutout portion of the damaged content.

8. The sign detection device using image matching according to claim 1, in, The mark detection unit displays an enlarged area of ​​the content to which the mark can be added in the content matching the original content and the matching content.

9. A landmark detection method using image matching, include: a damaged content acquisition step, acquiring damaged content generated based on the original content consisting of a plurality of intercepted parts; a pre-processed content generating step, generating the pre-processed content by merging one or more images included in the damaged content or removing local areas of one or more clipped parts included in the damaged content; a matching content generating step, generating matching content by adjusting the size or position of the one or more clipped parts included in the pre-processed content by comparing with the one or more clipped parts included in the original content; A matching step, matching the original content with the matching content according to the intercepted parts; as well as A marker detection step is performed to detect a marker that does not exist in the specific cutout portion of the original content but exists in the specific cutout portion of the matching content that matches the specific cutout portion of the original content.

10. The method for detecting a landmark using image matching according to claim 9, in, The matching content generation step includes the following steps: Comparing the one or more intercepted parts included in the original content with the one or more intercepted parts included in the preprocessed content to determine the intercepted part position of each of the one or more intercepted parts included in the preprocessed content; generating one or more blank images having the same size as the one or more images of the original content; and Based on the determined clipped portion positions, the one or more clipped portions included in the preprocessing content are arranged on the one or more blank images.

11. The method for detecting a landmark using image matching according to claim 9, in, The pre-processing content generation step comprises the following steps: Merging the individual images included in the damaged content to generate a merged image; and The outer area of ​​the clipped portion of the merged image is removed to generate pre-processed content consisting of only the clipped portion.

12. The method for detecting a landmark using image matching according to claim 9, in, In the matching content generation step, after comparing the cutout portion of the original content with the cutout portion of the damaged content and calculating a scale ratio for each cutout portion, the size of one or more cutout portions of the damaged content is adjusted based on the calculated scale ratio.

13. A computer program recorded in a computer-readable recording medium, for executing the method according to any one of claims 9 to 12.

Images