Still image-based media content monitoring system and method

Abstract

The present invention relates to a still image-based media content monitoring system and method, and more particularly to a still image-based media content monitoring system and method for monitoring illegally copied content of original video content such as movie content and broadcast content based on still images to prevent illegal distribution of the copied content.

Description

[Technical field]

[0001] The present invention relates to a still image-based media content monitoring system and method thereof, and more particularly to a still image-based media content monitoring system and method thereof for monitoring illegally copied content of original content, such as video content such as movie content and broadcasting content, based on still images to prevent illegal distribution of the copied content. [Background technology]

[0002] Currently, with the development of the Internet, services are being provided that enable users to share various video contents, such as movie contents, broadcasting contents, etc., through various media platforms, such as web hardware.

[0003] However, because such video content is composed of digital data and is easily copied, there have been many recent cases of illegally copying and sharing video content, causing considerable damage to the copyright holders of the video content. In addition, when such copied content is shared through the media platforms, the speed at which illegal content spreads is high, making it difficult to track down the illegal users who created the copied content.

[0004] Existing methods to block the production and sharing of such illegal content include security technologies such as DRM (Digital Rights Management) and digital watermarking, which allow the original video content to be played only on authorized devices and detect the person who is illegally copying it at the outset. However, these existing security methods also have the problem that they can easily be rendered ineffective as illegal users improve their security deactivation technology.

[0005] Therefore, recently, monitoring methods have emerged that detect the registered duplicate content when the copied content of the original content is registered on the web and block the distribution of the duplicate content in advance, as well as detect the illegal user who originally registered the duplicate content. However, these existing monitoring methods compare the duplicate content with the original content for all images contained in the content, which is inefficient in that it requires a considerable amount of time and effort to compare the duplicate content with the original content, especially in the case of video content with a long playback time. Not only that, because there is a considerable amount of content registered on media platforms, it is difficult to monitor all of the content using existing inefficient monitoring methods. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Korean Patent No. 10-2052535 Summary of the Invention [Problem to be solved by the invention]

[0007] The present invention aims to significantly reduce the time and load required to detect copied video content by accurately detecting copied content that is an illegal copy of an original video content through comparison of feature points and similarity with a small number of frames extracted from the original content, thereby blocking the distribution of illegally copied content in advance.

[0008] Another object of the present invention is to facilitate detection of illegally copied content even in content where it is difficult to secure the original content by supporting easy detection of copied content that is a copy of the original content corresponding to the trailer even when only the trailer corresponding to the original content is available. [Means for solving the problem]

[0009] A still image-based media content monitoring method according to an embodiment of the present invention includes an extraction step of a video registration unit receiving original content, which is a video content, and extracting a plurality of still images from the original content according to a preset condition; a registration step of the video registration unit converting the extracted still images into a preset standard to generate original frames, extracting feature points from the original frames to generate feature point information, and matching the feature point information with content identification information of the original content and the original frames corresponding to the content identification information and storing the content identification information in a DB; a receiving step of a video identification unit extracting still images from the video to be searched at a preset time interval when the video identification unit receives a search target video, converting the extracted still images into the preset standard to generate a plurality of search target frames; The method may further include a comparison step of performing an operation of extracting feature points from a search target frame, and calculating a visual similarity between each of one or more original frames that match the search target frame by comparing feature point information for each original frame stored in the DB with the feature point information for each original frame stored in the DB, for each of the plurality of search target frames, and generating comparison result information by combining original frames whose visual similarity is equal to or greater than a first predetermined reference value for each of the plurality of search target frames; and a detection step of a detection unit calculating an identification number by adding up a number of pieces of comparison result information in which an original frame corresponding to the original content exists among a plurality of pieces of comparison result information respectively corresponding to the plurality of search target frames for different original contents, and detecting original content that matches the search target image based on the identification number.

[0010] As one example related to the present invention, the extraction step may further include a step of the video registration unit identifying whether the original content is a trailer, and if so, extracting n still images per second from the original content.

[0011] As an example related to the present invention, the extraction step may further include a step of, if the original content is not a trailer, the video registration unit extracting one still image per m seconds, or grouping a plurality of frames of images that are sequential in chronological order in the original content and have a visual similarity equal to or greater than a second predetermined reference value to generate a plurality of groups, and then extracting one representative image for each of the plurality of groups as a still image.

[0012] As an example related to the present invention, the standard may be characterized as a preset resolution for normalizing a high resolution image with a low resolution image of a preset size.

[0013] As an example related to the present invention, the receiving step may further include the steps of: extracting a still image when the video identification unit receives the search target video, extracting feature points from the extracted still image to generate reference feature point information, comparing the reference feature point information with feature point information stored in the DB to identify original content that matches the reference feature point information by a second reference value or more and is most similar to the reference feature point information, identifying the search target video as a trailer if the identified original content is a trailer, and identifying the search target video as a non-trailer if the identified original content is not a trailer; and extracting n still images per second within a predetermined time period based on a still image used when identifying the search target video as a trailer from the search target video if the search target video is not a trailer.

[0014] As one example related to the present invention, the receiving step may further include a step of: if the search target video is not a trailer, the video identification unit sets an arbitrary section in the search target video and extracts still images at a preset time interval within the arbitrary section, or generates a plurality of different image groups by grouping frame images having a visual similarity equal to or greater than a preset third reference value within the arbitrary section, and then extracts one still image from each of the plurality of image groups.

[0015] As an example related to the present invention, the comparing step may be characterized in that the image identification unit calculates the visual similarity through a Mean Square Error (MSE) or a Structural Similarity Index Map (SSIM).

[0016] As an example related to the present invention, the detection step may further include a step of detecting an original content having the largest number of identification numbers, the number of identification numbers being equal to or greater than a predetermined reference number, based on the number of identification numbers calculated by the detection unit for each of the original contents, as a content matching the search target image.

[0017] As one example related to the present invention, the detection step may further include a step of generating and providing detection result information including content identification information of a specific original content detected by the detection unit as an original content matching the search target image and identification information of the search target image.

[0018] The still image-based media content monitoring system according to an embodiment of the present invention includes a video registration unit that receives original content, which is video content, extracts a plurality of still images from the original content according to a preset condition, converts the extracted still images into a preset standard to generate original frames, extracts feature points from the original frames to generate feature point information, matches the feature point information with content identification information of the original content and original frames corresponding to the feature point information, and stores the content in a DB; and when a search target video is received, extracts still images from the search target video at a preset time interval, converts the still images into the preset standard to generate a plurality of search target frames, extracts feature points from the search target frames, and stores the search target frames by original content in the DB. the plurality of search target frames is compared with the feature point information obtained by comparing the feature point information obtained by the search target frames with the plurality of original frames, the plurality of search target frames being matched with each other by a predetermined first reference value or more, and a video identification unit is configured to generate comparison result information by combining original frames having the visual similarity equal to or greater than a predetermined first reference value for each of the plurality of search target frames; and a detection unit is configured to calculate an identification number by adding up a number of comparison result information in which an original frame corresponding to an original content exists among a plurality of comparison result information respectively corresponding to the plurality of search target frames for different original contents, and to detect a specific original content having the identification number equal to or greater than a predetermined reference value and the largest identification number as a content matching the search target video. Effect of the Invention

[0019] The present invention eliminates the inefficient process of comparing all frames of an original content and a copied content one-to-one as in the conventional method, and after selecting a small number of still images from the copied content, it is possible to easily and accurately detect an original content having many similar frames to the selected still images as a copy target of the copied content. In addition, the present invention can detect copied content by simply converting a small number of still images from the original content and the copied content to a lower resolution and comparing them with each other, thereby greatly reducing the time and load required to detect copied content generated through illegal copying and greatly improving the efficiency of detecting illegally copied content.

[0020] Furthermore, as described above, the present invention has the effect of being able to quickly and accurately detect copied content with a low load, thereby preventing the distribution of illegally copied content in advance.

[0021] Furthermore, the present invention has the effect of helping to easily detect copied content that is a copy of the original content corresponding to the trailer even when only the trailer corresponding to the original content is available, thereby helping to easily detect illegally copied content even in content where it is difficult to secure the original content. [Brief description of the drawings]

[0022] [Figure 1] 1 is a configuration diagram of a still image-based media content monitoring system according to an embodiment of the present invention. [Diagram 2] 4 is a flowchart illustrating a process for registering original content in a still image-based media content monitoring system according to an embodiment of the present invention. [Diagram 3] 4 is a flowchart illustrating a process for detecting duplicate content in a still image-based media content monitoring system according to an embodiment of the present invention. [Figure 4]4 is a flowchart illustrating a process for detecting duplicate content in a still image-based media content monitoring system according to an embodiment of the present invention. [Diagram 5] 1 is a diagram illustrating an example of the operation of a still image-based media content monitoring system according to an embodiment of the present invention. [Figure 6] 1 is a diagram illustrating an example of the operation of a still image-based media content monitoring system according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0023] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, detailed embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a diagram showing the configuration of a still image-based media content monitoring system (hereinafter, referred to as a monitoring system) according to an embodiment of the present invention.

[0024] As shown, the monitoring system may be configured to include an image registration unit (110), an image identification unit (120), and a detection unit (130).

[0025] The monitoring system may be composed of one service device, and the video registration unit 110, the video identification unit 120 and the detection unit 130 may be included as components of the service device. The service device may be composed of a server.

[0026] Alternatively, the image registration unit 110, the image identification unit 120, and the detection unit 130 may each be configured as a separate device and communicate with each other via a communication network. For example, the image registration unit 110, the image identification unit 120, and the detection unit 130 may each be configured as a separate server.

[0027] In addition, the communication network described in the present invention may include wired / wireless communication networks. Examples of such wireless communication networks include Wireless LAN (WLAN), DLNA (Digital Living Network Alliance), Wibro (Wireless Broadband: Wibro), Wimax (World Interoperability for Microwave Access: Wimax), GSM (Global System for Mobile communication), CDMA (Code Division Multi Access), CDMA2000 (Code Division Multi Access 2000), EV-DO (Enhanced Voice-Data Optimized or Enhanced Voice-Data Only), WCDMA (Wideband CDMA), HSDPA (High Speed ​​Downlink Packet Access), HSUPA (High Speed ​​Uplink Packet Access), IEEE802.16, Long Term Evolution (LTE (Long Term Evolution)), LTE-A (Long Term Evolution-Advanced), and Wireless Mobile Broadband (Wireless Mobile Broadband). These include Wireless Broadband Service (WMBS), 5G mobile communications services, Bluetooth (registered trademark), LoRa (Long Range), RFID (Radio Frequency Identification), Infrared Data Association (IrDA), Ultra Wideband (UWB), ZigBee (registered trademark), Near Field Communication (NFC), Ultra Sound Communication (USC), Visible Light Communication (VLC), Wi-Fi, and Wi-Fi Direct (registered trademark).Furthermore, examples of wired communication networks include wired LAN (Local Area Network), wired WAN (Wide Area Network), Power Line Communication (PLC), USB communication, Ethernet (registered trademark), serial communication, and optical / coaxial cables.

[0028] Also, any one of the image registration unit 110, the image identification unit 120 and the detection unit 130 may be included within the other one.

[0029] Hereinafter, the detailed operation of the monitoring system will be described based on the configuration of FIG. 1 and with reference to the flowcharts of FIG. 2 to FIG.

[0030] FIG. 2 is a flowchart showing the operation of the image registration unit (110) according to an embodiment of the present invention. As shown in the figure, the video registration unit (110) can receive original content, which is video content, and for example, can receive the original content by communicating with a user terminal via a communication network (S1).

[0031] Furthermore, the original content described in the present invention may include video content requiring copyright protection, such as movie content, broadcasting content, or trailers.

[0032] Also, the image registration unit 110 can extract a plurality of still images from the original content according to a first preset condition.

[0033] For example, if the original content is a trailer (S2), the image registration unit (110) can extract n still images per second from the original content (S3).

[0034] In addition, if the original content is a trailer in a user terminal that is attempting to register the original content, trailer-related information for identifying the original content as a trailer is set in the original content and transmitted, or the trailer-related information is transmitted together with the original content, so that the video registration unit (110) can check whether the original content received from the user terminal is a trailer based on the trailer-related information.

[0035] In addition, the video registration unit (110) can extract n still images per second from the remaining sections of the original content corresponding to the trailer, excluding a start section (or intro section), which is a first time section that is a continuous pre-set time section including the start point of playback, and an end section (or outro section), which is a second time section that is a continuous pre-set time section including the end point of playback.

[0036] In other words, the start section includes an introduction and characters in the content, and the end section includes the distribution date and distribution company of the content. The image of the same frame as the main content, which is used to identify the copied content from the trailer-related content, is included in the remaining section excluding the start and end sections, so that the video registration unit (110) can obtain and register still images from the remaining section excluding the start and end sections to facilitate future comparison with the copied content that is a copy of the main content.

[0037] Further, the main content refers to the content that is used to generate the preview-related content and is screened, distributed, or broadcast after the preview-related content is distributed.

[0038] Alternatively, if the original content is not a trailer (S2), the video registration unit (110) may extract one still image per m seconds from the original content (S4).

[0039] Alternatively, if the original content is not a trailer, the video registration unit (110) may generate a plurality of groups by grouping images of a plurality of frames that are chronologically consecutive in the original content and have a visual similarity to each other that is equal to or greater than a preset reference value, select one representative image for each of the plurality of groups, and extract the selected representative image as the still image.

[0040] In addition, the image registration unit (110) can calculate the visual similarity between the images of the frames according to an image similarity comparison algorithm such as MSE (Mean Square Error) or SSIM (Structural Similarity Index Map).

[0041] As an example, the image registration unit (110) may select the first frame image, which is the first (earliest) frame image in the playback order, as a reference image among the multiple frame images included in the original content, calculate the visual similarity between the first frame image and the reference image in the playback order for each of the multiple frame images that follow the first frame image, and when identifying a specific frame image whose calculated visual similarity is less than a preset reference value, group the reference image (first frame image) to the frame image preceding the specific frame image into a specific group.

[0042] In addition, after generating the specific group, the image registration unit (110) sets the specific frame image as the reference image, and then calculates the visual similarity between each of a plurality of frame images subsequent to the specific frame image from the original content in the order of playback and the reference image. When checking another specific frame image whose calculated visual similarity is less than a preset reference value, the image registration unit (110) may group the reference image (specific frame image) to the frame image preceding the other specific frame image as another group different from the specific group.

[0043] As described above, the image registration unit 110 generates a group by combining one or more frame images whose visual similarity with the reference image is equal to or greater than a preset reference value each time a frame image whose visual similarity with the reference image is less than a preset reference value is identified, and changes the reference image to the identified frame image. The image registration unit 110 repeats the process of generating a new group in the same manner as the previous group until the final frame image of the original content is identified, thereby generating a plurality of groups, and selecting a representative image for each of the generated groups.

[0044] In addition, the image registration unit (110) may select the image of the frame that is played back earliest from each of the plurality of groups as the representative image, or may select the image of the frame with the highest resolution from each of the plurality of groups as the representative image, and may select the representative image in various manners without being limited thereto.

[0045] In addition, the monitoring system can be configured to include a content DB (101) and a feature point DB (102) as shown in FIG. 1, and the content DB (101) and the feature point DB (102) can also be configured as a single original DB.

[0046] Moreover, the video registration unit (110) can store the original content in the content DB (101).

[0047] In addition, the image registration unit (110) may convert each of a plurality of still images extracted from the original content into a preset standard to generate an original frame (S5), extract feature points for each original frame to generate feature point information (S6), and then match the feature point information with the content identification information and original frame of the original content corresponding to the content identification information and store the feature point information in a feature point DB (102) (S7).

[0048] In addition, the image registration unit (110) can convert and normalize the still image into a preset standard corresponding to a resolution of 320 x 240, and the resolution set in the standard can be changed and set in various sizes.

[0049] That is, the standard set in the image registration unit 110 is a preset resolution for normalizing a high-resolution image into a low-resolution image of a preset size.

[0050] In addition, the video registration unit (110) can match content identification information (eg, content ID, content title, etc.) along with the original content with the feature information and the original frame, and store the matched information in the feature DB (102).

[0051] In addition, the feature point information may include frame identification information (eg, content ID, frame time information, etc.) of a still image corresponding to the original frame and feature point data of feature points extracted from the original frame.

[0052] As described above, the image registration unit (110) communicates with one or more external devices via a communication network to receive a plurality of original contents from the one or more external devices, extracts a plurality of still images for each of the plurality of original contents and converts them into original frames, matches the feature point information generated for each of the plurality of original frames with content identification information of the corresponding original contents, and stores the feature point information in the feature point DB (102). The image registration unit (110) stores the plurality of original contents in the content DB (101), thereby sorting the plurality of original contents, the feature point information, and the plurality of original frames by original content and registering them in the original DB including the content DB (101) and the feature point DB (102).

[0053] In addition, when storing feature information in the feature DB (102), the image registration unit (110) matches the content identification information (e.g., a content ID) of the original content corresponding to the feature information with the feature information and stores the matched information in the feature DB (102), and when storing the original content in the content DB (101), matches the content identification information of the original content with the original content and stores the matched information in the content DB (101). Thus, for each of a plurality of original contents, the feature information corresponding to the original content and the original content can be matched with each other and registered in the original DB.

[0054] Meanwhile, FIG. 3 and FIG. 4 are flow charts showing the operation of the video identification unit 120 and the detection unit 130 in detecting illegally copied content according to an embodiment of the present invention.

[0055] First, the video identification unit (120) can receive request information including the search target video in a request to confirm illegally copied content of the search target video from an external device such as a user terminal via a communication network, or can connect to an external server via a communication network and receive (or collect) the search target video from the external server (S11).

[0056] Also, the image identification unit 120 can extract a plurality of still images from the original content according to a second preset condition.

[0057] First, the image identification unit (120) can receive request information including the image to be searched or, when receiving the image to be searched, extract one still image per m seconds, extract feature points from the extracted still image to generate reference feature point information, and then compare the reference feature point information with feature point information stored in the feature point DB (102). The image identification unit (120) can identify content identification information of an original content that matches the most matching feature point information from the feature point DB (102) among the feature point information stored in the feature point DB (102) that matches the reference feature point information by a preset value or more.

[0058] In addition, the set value may refer to a value previously set in the image identification unit (120) to be compared with the degree of matching between the reference feature point information and the feature point information calculated by comparing the reference feature point information with the feature point information stored in the feature point DB (102).

[0059] In addition, if there is no feature point information that matches the reference feature point information of the extracted still image by more than the set value, the image identification unit (120) may generate reference feature point information in a still image extracted from the search target image after the extracted still image, and then perform the comparison process and identification process as described above.

[0060] Thereafter, the video identification unit (120) searches the content DB (101) based on the specific content identification information identified in the feature DB (102) based on the reference feature point information, identifies specific original content matching the specific content identification information, and can identify whether the specific original content is a trailer (S12).

[0061] In addition, if the specific original content is a trailer (S13), the video identification unit (120) identifies the search target video as a trailer, extracts n still images per second (e.g., 10 per second) within a predetermined time period based on a still image used to identify the search target video as a trailer from the search target video (S14), and then converts them to a predetermined standard to generate multiple search target frames (S16).

[0062] In addition, the image identification unit (120) can generate the search target frame by converting a still image extracted from the search target image into a normalized image of 320 x 240 resolution, which is the same standard as the standard set in the image registration unit (110), and these standards can be changed and set in various ways.

[0063] That is, the standard set in the image identification unit 120 is a preset resolution for normalizing a high-resolution image into a low-resolution image of a preset size.

[0064] In addition, if the specific original content identified in accordance with the reference feature point information is not a trailer (S13), the video identification unit (120) may set an arbitrary section (arbitrary playback section) from the video to be searched and extract still images at a preset time interval within the arbitrary section, or may generate a plurality of different image groups by grouping frame images that are consecutive in chronological order within the arbitrary section and have a visual similarity equal to or greater than a preset first reference value, and then extract one still image from each of the plurality of image groups (S15).

[0065] As an example, if the video to be searched is not a trailer, the video identification unit (120) may set an arbitrary section from the video to be searched, since the number of frame images is enormous, and may extract one still image per m seconds within the arbitrary section, or one still image per minute (for example, one per minute).

[0066] As another example, as shown in FIG. 5, the image identification unit (120) selects the first frame image, which is the image that is the earliest (first) in the playback order, from among a plurality of frame images included in an arbitrary section from the image to be searched, as a reference image, and calculates the visual similarity between each of a plurality of frame images that are consecutive in the playback order within the arbitrary section after the first frame image, in the order of playback, and the reference image. When a first frame image is identified whose calculated visual similarity is less than a preset first reference value, the image identification unit (120) may group the reference image (first frame image) to the frame image preceding the first frame image into a first group.

[0067] Also, the first reference value set in the image identification unit 120 and the reference value set in the image registration unit 110 may be the same or different from each other.

[0068] In addition, after generating the first group in the search target image, the image identification unit (120) sets the image of the first frame as the reference image, and then calculates the visual similarity between each of the images of a plurality of frames subsequent to the image of the first frame within the arbitrary section and the reference image (image of the first frame) in the playback order. When an image of a second frame is identified whose calculated visual similarity is less than a preset first reference value, the image identification unit (120) may group the image of the reference image (image of the first frame) to the image of the frame preceding the image of the second frame into a second group.

[0069] As described above, the image identification unit (120) may select an arbitrary section from the search target image, select an image of a frame that is earliest in the playback order within the arbitrary section, and set it as a reference image. Then, each time a frame image having a visual similarity to the reference image that is less than a first reference value set in advance is identified, a group is generated and the reference image is changed to the image of the identified frame. Then, a new group is generated based on the changed reference image. This process is repeated until the final frame image of the search target image is identified, thereby generating a plurality of groups corresponding to the search target image, and selecting one representative image for each of the generated groups.

[0070] Also, the image identification unit 120 may extract, as the still image, one representative image selected from each of a plurality of groups generated to correspond to the search target image.

[0071] In addition, the image identification unit (120) may select the image of the frame that is earliest (first) in the playback order from each of the plurality of groups as the representative image, or may select the image of the frame with the highest resolution from each of the plurality of groups as the representative image, and may select the representative image in various manners without being limited thereto.

[0072] Also, the image identification unit 120 can calculate the visual similarity according to an image similarity comparison algorithm such as Mean Square Error (MSE) or Structural Similarity Index Map (SSIM).

[0073] Meanwhile, the image identification unit (120) may pre-set an operation algorithm (or a first algorithm) for performing an operation process of extracting feature points from the search target frame, and then calculating the visual similarity between each of one or more original frames that match the feature points or more than a criterion, in comparison with the feature point information stored for each original content in the content DB (101) (or original DB).

[0074] Thus, when a plurality of search target frames are generated in the search target image, the image identification unit 120 can perform the operation process in accordance with the algorithm on each of the plurality of search target frames.

[0075] As an example, the image identification unit (120) may extract feature points from a specific search target frame among a plurality of search target frames generated to correspond to the search target image, generate search target feature point information including the feature points extracted from the specific search target frame, and compare the search target feature point information with a plurality of feature point information stored in the feature point DB (102) (S17).

[0076] In addition, if there is no feature point information in the feature point DB (102) that matches the generated feature point information for at least one of the multiple search target frames by a standard or more (S18, S19), the video identification unit (120) can determine that there is no original content that matches the search target video and provide determination result information that the search target video is not illegally copied content (S20, S21).

[0077] Also, the image identification unit (120) can extract one or more pieces of feature point information that are identical (similar) to the search target feature point information by a second reference value or more set in advance from the feature point DB (102).

[0078] In addition, the second reference value may mean a reference numerical value preset in the image identification unit (120) to be compared with a degree of matching between the search target feature point information and the feature point information stored in the feature point DB (102) calculated by comparing the search target feature point information with the feature point information stored in the feature point DB (102), and the feature point information stored in the feature point DB (102) whose degree of matching is equal to or greater than the second reference value may be identified as matching with the search target feature point information at a standard level or higher.

[0079] Also, the set value set in the image discriminating unit 120 and the second reference value may be the same.

[0080] In addition, the video identification unit (120) can extract, from the feature point DB (102), original frames that match specific feature point information extracted from the feature point DB (102) based on the search target feature point information.

[0081] As a result, the image identification unit (120) performs the same operational process as the operational process of extracting original frames corresponding to (matching) the specific feature point information from each of one or more feature point information extracted from the feature point DB (102) that matches the search target feature point information by a predetermined second reference value or more, and can obtain one or more original frames each corresponding to the extracted one or more feature point information so as to correspond to the specific search target frame corresponding to the search target feature point information.

[0082] Additionally, the one or more acquired original frames may be images corresponding to a single original content, or images corresponding to different original contents.

[0083] In addition, the image identification unit (120) may perform a visual comparison with the specific search target frame for each of one or more original frames acquired to correspond to the specific search target frame, and calculate a visual similarity between the specific search target frame and each of the acquired one or more original frames (S22).

[0084] In addition, the image identification unit (120) can calculate the visual similarity between the specific search target frame and the original frame through an image similarity comparison algorithm such as MSE (Mean Square Error) or SSIM (Structural Similarity Index Map).

[0085] In addition, the image identification unit (120) may perform the process of extracting feature points from the specific search target frame, acquiring one or more original frames based on the extracted feature points, and calculating a visual similarity for each of the acquired one or more original frames for each of the multiple search target frames generated corresponding to the search target image, and may acquire one or more original frames for each of the multiple search target frames, and then calculate a visual similarity for each of the acquired original frames.

[0086] Meanwhile, as shown in FIG. 6, the image identification unit (120) checks the visual similarity for one or more original frames acquired for the plurality of search target frames, and checks, as candidate images, original frames for which the visual similarity is equal to or greater than a third predetermined reference value for each of the plurality of search target frames, and combines each candidate image as it is checked to generate comparison result information including one or more original frames checked as candidate images (S23).

[0087] For ease of explanation, the original frames identified as the candidate images will be described below as the candidate images.

[0088] Additionally, the comparison result information may include a search target frame and one or more candidate images that match the search target frame.

[0089] In addition, the image identification unit 120 may generate comparison result information including one or more candidate images for each of the plurality of search target frames, thereby generating a plurality of pieces of comparison result information each corresponding to the plurality of search target frames.

[0090] In addition, the image identification unit (120) may provide (transmit) to the detection unit (130) search request information included in a plurality of comparison result information corresponding to the plurality of search target frames and the search target images corresponding to the plurality of search target frames, which are mutually matched.

[0091] In addition, when receiving the search request information from the image identification unit (120), the detection unit (130) checks whether a candidate image corresponding to a specific original content exists from each of a plurality of pieces of comparison result information included in the search request information, and calculates the number of identifications by adding up the number of pieces of comparison result information in which a candidate image corresponding to the specific original content exists among the plurality of pieces of comparison result information (S24).

[0092] In addition, the detection unit (130) can identify content identification information matching a candidate image in a feature point DB (102) and identify original content corresponding to the candidate image based on the identified content identification information, or can search the content DB (101) based on the identified content identification information to identify original content corresponding to the candidate image.

[0093] In addition, the detection unit (130) may calculate the number of identifications for each of a plurality of original contents corresponding to a plurality of candidate images included in the plurality of comparison result information, detect a specific original content among the plurality of candidate contents, the number of identifications of which is equal to or greater than a predetermined reference number, and the number of identifications of which is the largest, as a content that matches the search target image, and generate and provide detection result information including the specific original content and the search target image (S25, S26).

[0094] That is, the detection unit (130) can generate and provide detection result information corresponding to the search target image, thereby providing that the search target image is a duplicate content that is a duplicate of a specific original content included in the detection result information.

[0095] In addition, the detection unit (130) can extract content identification information of a specific original content detected as an original content matching the search target video from the content DB (101) or the feature point DB (102), and generate and provide detection result information including the identification information of the specific original content and the identification information of the search target video.

[0096] In addition, the detection unit 130 may transmit the detection result information to an external device that transmitted the search target video via a communication network.

[0097] As an example of the above-mentioned configuration, as shown in FIG. 6, the image identification unit (120) may acquire one or more original frames having feature points that match a first search target frame through feature point comparison among five search target frames obtained in the search target image, calculate a visual similarity between each of the one or more original frames and the first search target frame, and generate first comparison result information in which each of the original frames having a visual similarity equal to or greater than a preset third reference value is regarded as a candidate image.

[0098] In addition, the image identification unit (120) may generate second through fifth comparison result information corresponding to the second through fifth search target frames, respectively, in the same process as generating first comparison result information corresponding to the first search target frame in each of the five search target frames.

[0099] Next, the detection unit (130) can identify content identification information of a first original content (content 1) corresponding to a first candidate image included in any one of the first to fifth comparison result information generated through the image identification unit (120) in the feature point DB (102), and can search for the first to fifth comparison result information based on the content identification information of the first original content and the information stored in the feature point DB (102), and can confirm whether a candidate image corresponding to the first original content exists from each of the first to fifth comparison result information.

[0100] Also, the detection unit 130 may calculate the number of identifications as five if the candidate images corresponding to the first original content are all present in the first through fifth comparison result information.

[0101] In addition, even if there are multiple candidate images corresponding to a specific original content in a specific comparison result information, the detection unit (130) may count the number of identifications counted corresponding to the specific comparison result information and the specific original content as one.

[0102] In addition, the detection unit (130) searches for the first to fifth comparison result information based on content identification information of a second original content (content 2) corresponding to a second candidate image included in any one of the first to fifth comparison result information generated through the image identification unit (120) and information stored in the feature point DB (102), and when the comparison result information in which a candidate image corresponding to the second original content exists is the first comparison result information, the third comparison result information, and the fifth comparison result information, the detection unit (130) may calculate the number of identifications corresponding to the second original content to be three.

[0103] In addition, the detection unit (130) may calculate the number of classifications to be two for a third original content (content 3) corresponding to a third candidate image included in any one of the first to fifth comparison result information generated through the image classification unit (120).

[0104] Thus, the detection unit (130) can detect the first original content having the highest identification number among the first to third original contents, which is equal to or greater than a preset reference number of three, as a content that matches the search target image corresponding to the information of the first to fifth comparison results.

[0105] As described above, the present invention extracts a small number of still images from each of the original content, which is a video content, and the copied content obtained by illegally copying the original content, converts the extracted images to a low-quality resolution, and selects one or more frames of the original content that have a feature point matching with a frame of the copied content or more than a reference point through a feature point comparison. Then, the present invention calculates a visual similarity between the selected frames and the frames of the copied content. A specific original content that includes many frames with a visual similarity above a reference point can be easily identified as a content that matches the copied content, and the copied content can be detected as a content obtained by illegally copying a specific original content, which is different from the conventional method of detecting the original content. By eliminating the inefficient process of comparing all frames of the copied content one-to-one, it is possible to easily and accurately detect the original content, which has many similar frames even if they are not exactly the same as the selected still images, as the copy target of the copied content after selecting a small number of still images from the copied content. It is also possible to detect the copied content by simply converting a small number of still images from the original content and the copied content to a lower resolution and comparing them with each other, which greatly reduces the time and load required to detect copied content generated through illegal copying and greatly increases the efficiency of detecting illegally copied content.

[0106] Furthermore, as described above, the present invention can quickly and accurately detect duplicated content with a low load, and can block the distribution of illegally duplicated content in advance.

[0107] Furthermore, the present invention can easily detect copied content that is a copy of the original content corresponding to the trailer, even when only the trailer corresponding to the original content is available, thereby making it possible to easily detect illegally copied content even in content where it is difficult to secure the original content.

[0108] The components described in the embodiments of the present invention may be embodied using one or more general-purpose or special-purpose computers, such as hardware including a storage unit such as a memory, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable array (FPA), a programmable logic unit (PLU), a microprocessor, software including a command set, or a combination thereof, or any other device capable of executing and responding to instructions.

[0109] The above contents should be modifiable and changed by a person having ordinary skill in the art to which the present invention belongs, without departing from the essential characteristics of the present invention. Therefore, the embodiments disclosed in the present invention are for illustration purposes, not for limiting the technical phenomena of the present invention, and the scope of the technical phenomena of the present invention is not limited by such embodiments. The scope of protection of the present invention should be interpreted according to the following claims, and all technical phenomena within the scope equivalent thereto should be interpreted as being included in the scope of the present invention. [Explanation of symbols]

[0110] 101 Content DB 102 Feature point DB 110 Video Registration Department 120 Video Identification Unit 130 Detection unit

Claims

1. an extraction step of receiving an original content, which is a video content, by a video registration unit and extracting a plurality of still images from the original content according to a preset condition; a registration step in which the image registration unit converts the extracted still image into a preset format to generate an original frame, extracts feature points from the original frame to generate feature point information, and matches the feature point information with content identification information of the original content and the original frame corresponding to the content identification information and stores the content identification information in a DB; a receiving step of extracting still images from the search target image at a preset time interval and converting the extracted still images into the standard to generate a plurality of search target frames when the image identification unit receives the search target image; a comparison step of performing an operation process for each of the plurality of search target frames, in which the image identification unit extracts feature points from the search target frames, and then compares the feature points with feature point information for each original frame stored in the DB to calculate a visual similarity between each of the plurality of search target frames and the search target frames, the visual similarity being equal to or greater than a predetermined first reference value, for each of the plurality of search target frames, and generating information on a comparison result by combining original frames having the visual similarity equal to or greater than a predetermined first reference value; and a detection step of the detection unit calculating an identification number by adding up a number of pieces of comparison result information each corresponding to the plurality of search target frames for each different original content, the number of pieces of comparison result information each corresponding to an original frame corresponding to the original content, and detecting an original content that matches the search target image based on the identification number; The standard is a preset resolution for normalizing a high-resolution image with a low-resolution image of a preset size, The receiving step includes: the video identification unit extracts a still image when receiving the video to be searched, extracts feature points from the extracted still image to generate reference feature point information, compares the reference feature point information with feature point information stored in the DB, and identifies original content that matches the reference feature point information by a second reference value or more and is most similar to the reference feature point information, and identifies the video to be searched as a trailer if the identified original content is a trailer, and identifies the video to be searched as a non-trailer if the identified original content is not a trailer; and The video identification unit further includes extracting n still images per second within a preset time period based on still images used when identifying the search target video as a trailer from the search target video when the search target video is not a trailer. A method for monitoring media content based on still images.

2. The extracting step further includes the step of: identifying whether the original content is a trailer, and extracting n still images per second from the original content if the original content is a trailer, by the video registration unit. The method of claim 1, wherein the still image-based media content monitoring method comprises:

3. The extracting step may further include, if the original content is not a trailer, extracting one still image per m seconds by the video registration unit, or grouping a plurality of frames of images that are sequential in time order in the original content and have a visual similarity equal to or greater than a second reference value set in advance to generate a plurality of groups, and then extracting one representative image for each of the plurality of groups as a still image. The method of claim 2, further comprising:

4. The receiving step comprises: The method further includes, when the video identification unit determines that the video to be searched is not a trailer, setting an arbitrary section in the video to be searched and extracting still images at a preset time interval within the arbitrary section, or grouping frame images having a visual similarity equal to or greater than a preset third reference value within the arbitrary section to generate a plurality of different image groups, and then extracting one still image from each of the plurality of image groups. The method of claim 1, wherein the still image-based media content monitoring method comprises:

5. The comparison step is characterized in that the image identification unit calculates the visual similarity via MSE (Mean Square Error) or SSIM (Structural Similarity Index Map). The method of claim 1, wherein the still image-based media content monitoring method comprises:

6. The detection step further includes a step of: determining whether the number of identifications calculated by the detection unit for each of the original contents is equal to or greater than a predetermined reference number; and determining whether the original content having the largest number of identifications matches the search target image. The method of claim 1, wherein the still image-based media content monitoring method comprises:

7. The detection step further includes a step of generating and providing detection result information including content identification information of a specific original content detected by the detection unit as an original content matching the search target video and identification information of the search target video. The method of claim 1, wherein the still image-based media content monitoring method comprises:

8. A video registration unit that receives original content which is video content, extracts a plurality of still images from the original content according to preset conditions, converts the extracted still images into a preset standard to generate original frames, extracts feature points from the original frames to generate feature point information, and then matches the feature point information with content identification information of the original content and the original frames corresponding to the feature point information and stores the information in a DB; an image identification unit which, when receiving a search target image, extracts still images from the search target image at a preset time interval, converts the extracted still images into the standard to generate a plurality of search target frames, extracts feature points from the search target frames, and performs an operation of calculating a visual similarity between the search target frame and one or more original frames that match the extracted still images by a predetermined standard or more, in comparison with feature point information stored in the DB for each original content, for each of the plurality of search target frames, and generates information of the comparison result by combining original frames whose visual similarity is equal to or greater than a preset first standard value for each of the plurality of search target frames; and a detection unit for calculating an identification number by summing up a number of pieces of comparison result information each corresponding to the plurality of search target frames for each different original content, in which an original frame corresponding to the original content exists, and detecting a specific original content having the identification number equal to or greater than a predetermined reference value and the largest identification number as a content matching the search target image, The standard is a preset resolution for normalizing a high-resolution image with a low-resolution image of a preset size, The video identification unit, When the search target video is received, a still image is extracted, and feature points are extracted from the extracted still image to generate reference feature point information. The reference feature point information is compared with feature point information stored in the DB to identify original content that matches the reference feature point information by a second reference value or more and is most similar to the reference feature point information. If the identified original content is a trailer, the search target video is identified as a trailer, and if the identified original content is not a trailer, the search target video is identified as a video that is not a trailer. If the search target video is not a trailer, n still images are extracted per second within a preset time period based on still images used when identifying the search target video as a trailer from the search target video. A still image-based media content monitoring system.

Images