Electronic apparatus for identifying pirated video and operating method thereof

Abstract

The present disclosure relates to an electronic device for identifying illegally copied images and a method of operation thereof. In a method of operation of an electronic device for identifying illegally copied images according to an embodiment of the present disclosure, the method of operation may include the steps of: obtaining query image information for a target image content to be checked for illegal copying; deriving first vector information corresponding to the query image information using a Fourier transform; requesting comparison target information from a database based on first metadata included in the query image information; and identifying whether the target image content is illegally copied based on the comparison target information and the first vector information.

Description

Technology Field

[0001] The present disclosure relates to content protection and management technology, and more specifically, to an electronic device for identifying illegally copied images using Fourier transform and a method of operation thereof. Background Technology

[0002] Recently, with the proliferation of OTT (Over-The-Top) services that provide online streaming video content, users have been able to conveniently access video content. As OTT services have expanded, various OTT platforms have emerged, and users can enjoy video content provided by these platforms through subscriptions. However, since each platform can only provide users with video content for which the rights belong to that platform, users must subscribe to multiple OTT platforms to access the video content they desire.

[0003] Under these circumstances, illegal streaming sites are emerging that aggregate and provide video content from various OTT platforms through a single site. These illegal streaming sites illegally copy video content provided by OTT platforms and offer it to users, and the damages caused by these sites are estimated to be at a serious level, reaching at least 5 trillion won as of 2023. Consequently, there is a growing need for technology to identify whether video content has been illegally copied in order to eradicate these illegal streaming sites.

[0004] In this regard, prior art such as KR10-2021-0154044A and KR10-2023-0120991A may be referenced. The problem to be solved

[0005] The present disclosure aims to provide an electronic device for identifying illegally copied images using a Fourier transform and a method of operating the same.

[0006] The present disclosure aims to provide an electronic device for identifying illegally copied images using low-frequency components extracted from an image frame and a method of operating the same.

[0007] The problems that this disclosure aims to solve are not limited to those described above, and other unmentioned problems will be clearly understood by a person skilled in the art from the description below. means of solving the problem

[0008] In a method of operation of an electronic device for identifying illegally copied images according to one embodiment of the present disclosure, the method of operation may include the steps of: obtaining query image information for a target image content to be checked for illegal copying; deriving first vector information corresponding to the query image information using a Fourier transform; requesting comparison target information from a database based on first metadata included in the query image information; and identifying whether the target image content is illegally copied based on the comparison target information and the first vector information.

[0009] In an embodiment, the step of deriving first vector information corresponding to the query image information using the Fourier transform may include the step of deriving a query image frame based on the query image information, the step of removing color information for the query image frame, the step of performing the Fourier transform on the query image frame from which the color information has been removed to separate the frequency domain, and the step of deriving the first vector information based on the frequency domain.

[0010] In an embodiment, the step of deriving the first vector information based on the frequency domain may include the step of rearranging the frequency domain around a low-frequency component, the step of extracting a center region for a query image frame rearranged around the low-frequency component, and the step of deriving the first vector information based on the center region.

[0011] In an example, the central area can be derived by cropping based on a preset size.

[0012] In an embodiment, the database stores second vector information corresponding to the original image content, and the second vector information can be derived from the original image information for the original image content using the Fourier transform.

[0013] In an embodiment, the comparison target information may include the second vector information derived from the original image information, which includes second metadata corresponding to the first metadata.

[0014] In an embodiment, the step of identifying whether the target video content is illegally copied may include the step of checking the similarity between the first vector information and the second vector information.

[0015] In an embodiment, the Fourier transform may be a Fast Fourier Transform (FFT).

[0016] An electronic device for identifying illegally copied images according to one embodiment of the present disclosure, wherein the electronic device comprises a transceiver, a memory for storing instructions, and at least one processor, and the at least one processor connected to the transceiver and the memory acquires query image information regarding a target image content to be checked for illegal copying, derives vector information corresponding to the query image information using a Fourier transform, requests comparison target information from a database based on metadata included in the query image information, and can identify whether the target image content is illegally copied based on the comparison target information and the vector information.

[0017] A computer program stored on a computer-readable storage medium according to one embodiment of the present disclosure, wherein when the computer program is executed on at least one processor, it performs the following operations to identify an illegally copied image, and the operations may include: an operation of obtaining query image information for a target image content to be checked for illegal copying; an operation of deriving vector information corresponding to the query image information using a Fourier transform; an operation of requesting comparison target information from a database based on metadata included in the query image information; and an operation of identifying whether the target image content is illegally copied based on the comparison target information and the vector information. Effects of the invention

[0018] According to the present disclosure, by using a Fourier transform to identify illegally copied images, the time required to identify illegally copied images can be reduced.

[0019] According to the present disclosure, by identifying illegally copied images using low-frequency components extracted from video frames, the computational cost for identifying illegally copied images can be reduced while the accuracy of the illegally copied image identification result can be improved.

[0020] The effects according to the present disclosure are not limited to those described above, and other unmentioned effects will be clearly understood by a person skilled in the art from the description below. Brief explanation of the drawing

[0021] FIG. 1 is a block diagram illustrating a system for identifying illegally copied images according to an embodiment of the present disclosure. FIG. 2 is a block diagram illustrating an electronic device for identifying illegally copied images according to an embodiment of the present disclosure. FIG. 3 is a flowchart illustrating a method of operation of an electronic device for identifying illegally copied images according to an embodiment of the present disclosure. FIG. 4 is a flowchart illustrating a method for generating vector information for an original image according to an embodiment of the present disclosure. FIG. 5 is a diagram illustrating, by way of example, the process of generating vector information for an original image according to an embodiment of the present disclosure. FIG. 6 is a flowchart illustrating a method for generating vector information for a query image according to an embodiment of the present disclosure. FIG. 7 is a block diagram for illustrating an electronic device according to an embodiment of the present disclosure. FIG. 8 is a table for explaining the results of a performance comparison of an electronic device according to an embodiment of the present disclosure. Specific details for implementing the invention

[0022] Hereinafter, exemplary embodiments according to the present invention will be described in detail with reference to the contents described in the attached drawings. However, the present invention is not limited or restricted by exemplary embodiments. Unless otherwise defined, all terms used in this specification (including technical and scientific terms) shall be used in a meaning that is commonly understood by those skilled in the art to which this disclosure belongs, but this may vary depending on the intent of those skilled in the art, case law, the emergence of new technology, etc.

[0023] Furthermore, terms defined in commonly used dictionaries are not to be interpreted ideally or excessively unless explicitly and specifically defined otherwise. In certain cases, terms have been selected at the applicant's discretion, and in such cases, their meanings will be described in detail in the relevant explanatory sections. Accordingly, terms used in this disclosure should be defined not merely by their names, but based on their meanings and the content throughout this disclosure.

[0024] Throughout this specification, when a part is described as "comprising" a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components. Furthermore, the singular form used in this specification includes the plural form unless specifically stated otherwise. Additionally, the expression "at least one of a, b, and / or c" as used throughout this specification may encompass 'a alone', 'b alone', 'c alone', 'a and b', 'a and c', 'b and c', or 'a, b, and c all'.

[0025] Meanwhile, terms such as "first and / or second" used in this specification may be used to describe various components, but they are used solely for the purpose of distinguishing one component from another and are not intended to limit the scope to the components referred to by such terms. For example, without departing from the scope of the present invention, the first component may be named the second component, and the second component may also be named the first component.

[0026] Additionally, terms such as “…part,” “…module,” etc., as described in this specification refer to a unit that processes at least one function or operation, which may be implemented in hardware or software, or a combination of hardware and software. Furthermore, embodiments of this disclosure may be represented in this specification by functional block configurations and various processing steps. These functional blocks may be implemented by various numbers of hardware and / or software configurations that execute specific functions. For example, embodiments of this disclosure may employ integrated circuit configurations such as memory, processing, logic, look-up tables, etc., which can execute various functions under the control of one or more microprocessors or other control devices.

[0027] In an embodiment according to the present disclosure, functions related to artificial intelligence may be implemented through a processor and memory. In this case, the processor may be any one of a general-purpose processor such as a CPU (Center Processing Unit), AP (Application Processor), DSP (Digital Signal Processor), a graphics-dedicated processor such as a GPU (Graphic Processing Unit) or VPU (Vision Processing Unit), and an artificial intelligence-dedicated processor such as an NPU (Neural Network Processing Unit). The processor may process input data according to predefined operation rules or artificial intelligence models stored in memory. Alternatively, if the processor is an artificial intelligence-dedicated processor, the artificial intelligence-dedicated processor may be designed with a hardware structure specialized for processing a specific artificial intelligence model. In some embodiments according to the present disclosure, functions related to artificial intelligence may be implemented through a plurality of processors.

[0028] In an embodiment according to the present disclosure, a predefined operation rule or artificial intelligence model may be configured to perform machine learning. Here, being configured to perform machine learning means that the predefined operation rule or artificial intelligence model is configured to perform a desired characteristic (or objective) by learning using a plurality of training data based on a learning algorithm. Such learning may be performed on the device itself in which the artificial intelligence according to the present disclosure is implemented, or it may be performed through a separate server and / or system.

[0029] Artificial intelligence models can be implemented as neural networks (or artificial neural networks) and can operate based on statistical learning algorithms that mimic biological neurons in machine learning and cognitive science. A neural network can refer to a model in which artificial neurons (nodes), which form a network through the connection of synapses, change the strength of synaptic connections through learning to possess problem-solving capabilities. A neural network can be composed of multiple neural network layers; for example, a neural network may include an input layer, a hidden layer, and an output layer. Each of the multiple neural network layers may include at least one node and at least one weight, and neural network operations can be performed through operations between the results of operations of the previous (precious) layer and the weights. At least one weight possessed by the multiple neural network layers may be optimized based on the learning results of the artificial intelligence model. For example, at least one weight may be updated so that the loss value or cost value obtained from the artificial intelligence model during the learning process is reduced or minimized. Neural networks can infer a result to be predicted from an arbitrary input.

[0030] The learning methods of artificial intelligence models can be classified according to the learning approach into supervised learning, where input and output data are provided as training data and the correct answer (output data) corresponding to the problem (input data) is predetermined; unsupervised learning, where only input data is provided without output data and the correct answer (output data) corresponding to the problem (input data) is not predetermined; and reinforcement learning, where a reward is granted whenever an action is taken from the current state and learning proceeds in a direction that maximizes this reward. Alternatively, they can be classified according to the architecture, which is the structure of the learning model.

[0031] In the embodiments of the present disclosure, the artificial intelligence model is a Convolutional Neural Network (CNN) such as GoogleNet, AlexNet, VGG Network, Region with Convolutional Neural Network (R-CNN), Region Proposal Network (RPN), Recurrent Neural Network (RNN), Stacking-based Deep Neural Network (S-DNN), State-Space Dynamic Neural Network (S-SDNN), Deconvolution Network, Deep Belief Network (DBN), Restructured Boltzmann Machine (RBM), Fully Convolutional Network, Long Short-Term Memory Network (LSTM), Classification Network, Generative Modeling, eXplainable AI, Continual AI, Representation Learning, AI for Material Design, BERT, SP-BERT, MRC / QA for Natural Language Processing, Text Analysis, Dialog System, GPT-3, GPT-4, Visual Analytics, Visual Understanding, Video Synthesis for Vision Processing, Anomaly Detection, Prediction, Time-Series Forecasting, Optimization, Recommendation for ResNet Data Intelligence, At least one of various artificial intelligence structures and algorithms, such as data creation, may be used. The examples described above are merely examples of artificial intelligence structures and algorithms used according to the embodiments of the present disclosure and do not limit the artificial intelligence structures and algorithms used according to the embodiments of the present disclosure.

[0032] Hereinafter, various embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In describing the embodiments, technical details that are well known in the art to which the present invention pertains and are not directly related to the present invention will be omitted. This is to ensure that the essence of the present invention is conveyed more clearly without obscuring it by omitting unnecessary explanations. For the same reason, some components in the accompanying drawings may be exaggerated, omitted, or schematically depicted. Furthermore, the size of each component does not entirely reflect its actual size. Throughout this specification, the same reference numerals may refer to the same or corresponding components.

[0033] FIG. 1 is a block diagram illustrating a system (10) for identifying illegally copied images according to an embodiment of the present disclosure.

[0034] Referring to FIG. 1, a system (10) for identifying illegally copied images according to an embodiment of the present disclosure may include a first external device (110), a second external device (120), and an electronic device (130).

[0035] Each of the first external device (110), the second external device (120), and the electronic device (130) illustrated in FIG. 1 may include a transceiver, memory, and a processor. Additionally, each of the first external device (110), the second external device (120), and the electronic device (130) represents a unit that processes at least one function or operation, which may be implemented as hardware or software, or a combination of hardware and software. According to an embodiment, at least some of the first external device (110), the second external device (120), and the electronic device (130) may include a plurality of computer systems or computer software implemented as network servers. For example, at least some of the first external device (110), the second external device (120), and the electronic device (130) may refer to computer systems and computer software that are connected to a sub-device capable of communicating with another network server through a computer network such as an intranet or the internet, receive a request to perform a task, perform the task, and provide the result.

[0036] In addition, at least some of the first external device (110), the second external device (120), and the electronic device (130) may be understood in a broad sense as including a series of applications that can operate on a network server and various databases built inside. For example, at least some of the first external device (110), the second external device (120), and the electronic device (130) may be implemented using network server programs provided in various ways depending on an operating system such as DOS, Windows, Linux, Unix, or MacOS.

[0037] In a system (10) for identifying illegally copied images according to an embodiment of the present disclosure, an electronic device (130) may be electrically connected to each of the first external device (110) and the second external device (120), or may communicate with each other through a network and exchange information. When the electronic device (130) communicates with each of the first external device (110) and the second external device (120) through a network, the communication method is not limited and may include not only a communication method utilizing a communication network that the network may include, but also short-range wireless communication between devices. For example, the network may include at least one network among a PAN (Personal Area Network), LAN (Local Area Network), CAN (Campus Area Network), MAN (Metropolitan Area Network), WAN (Wide Area Network), BBN (Broad Band Network), and the Internet. In addition, the network may include at least one of network topologies including a bus network, a star network, a ring network, a mesh network, a star-bus network, a tree or hierarchical network, etc.

[0038] According to an embodiment of the present disclosure, the first external device (110) can provide video content (original video content) to a user that is legally copyrighted. For example, the first external device (110) can provide video content to a user via cable or satellite, or provide video content to a user via an OTT (Over The Top) platform. The first external device (110) can store video information regarding various video contents provided to the user and can manage the stored video contents.

[0039] Meanwhile, the second external device (120) can provide video content to the user in a real-time streaming manner. In this specification, the video content provided from the second external device (120) may be understood as a concept including illegally copied video content. In FIG. 1, the electronic device (130) is shown as being connected to one first external device (110) and one second external device (120), but this is merely one embodiment according to the present disclosure and does not limit the configuration of the system (10) for identifying illegally copied video according to the present disclosure. For example, the system (10) for identifying illegally copied video may include a plurality of first external devices (110) or a plurality of second external devices (120).

[0040] An electronic device (130) according to an embodiment of the present disclosure may obtain original image information (V1) for original image content from a first external device (110). In this specification, original image information (V1) may mean image information for image content to which a legal copyright has been granted. Original image information (V1) may include first metadata for identifying the content of the original image content, and the electronic device (130) may build a database (not shown) related to the original image content based on the original image information (V1) obtained from the first external device (110).

[0041] Meanwhile, the electronic device (130) can obtain query video information (V2) regarding the copied video content from the second external device (120). In this specification, query video information (V2) may refer to video information regarding the target video content for which illegal copying is to be verified, and in the embodiment, the query video information (V2) may be video information regarding streaming video content. The query video information (V2) may include second metadata for identifying the content of the query video content. The electronic device (130) can verify whether the video content is illegally copied based on the query video information (V2). The method for constructing a database related to the original video content and the method for identifying whether the query video content is illegally copied, which are performed by the electronic device (130) according to the embodiment of the present disclosure, will be explained in detail through FIGS. 2 to 7, which will be described later.

[0042] A system (10) for identifying illegally copied images according to the present disclosure can verify whether a query image content is an illegally copied image by using a Fourier transform. According to an embodiment of the present disclosure, by identifying illegally copied images using a Fourier transform, the time required for identifying illegally copied images can be reduced. In addition, a system (10) for identifying illegally copied images according to the present disclosure can verify whether a query image content is an illegally copied image by using a component extracted from an image frame. According to an embodiment of the present disclosure, by identifying illegally copied images by extracting a portion of the component corresponding to the image frame, the computational cost for identifying illegally copied images can be reduced, while the accuracy of the illegally copied image identification result can be improved.

[0043] FIG. 2 is a block diagram illustrating an electronic device (20) for identifying illegally copied images according to an embodiment of the present disclosure.

[0044] The electronic device (20) for identifying illegally copied images shown in FIG. 2 may correspond to the electronic device (130, see FIG. 1) described above in FIG. 1. Referring to FIG. 2, the electronic device (20) for identifying illegally copied images according to an embodiment of the present disclosure may include an original image vector generation unit (210), a database (220), a query image vector generation unit (230), and an illegality determination unit (240).

[0045] According to an embodiment of the present disclosure, the original image vector generation unit (210) can generate first vector information (VT1) corresponding to original image information (V1) obtained from a first external device (110, see FIG. 1) and provide the generated first vector information (VT1) to a database (220). The database (220) can store the first vector information (VT1) obtained from the original image vector generation unit (210). In the embodiment, the database (220) can store first vector information (VT1) corresponding to each of various image contents, and the operation of generating first vector information (VT1) corresponding to the original image from the original image vector generation unit (210) will be explained in detail through FIG. 4 and FIG. 5, which will be described later. Meanwhile, in FIG. 2, the database (220) is shown as being built inside an electronic device (20), but this is merely an embodiment according to the present disclosure and does not limit the configuration of the electronic device (20) according to the present disclosure. In another embodiment, the database (220) may be built as a separate server outside the electronic device (130).

[0046] A query image vector generation unit (230) according to an embodiment of the present disclosure may generate second vector information (VT2) corresponding to query image information (V2) obtained from a second external device (120, see FIG. 1) and provide the generated second vector information (VT2) to an illegality determination unit (240). The operation of generating second vector information (VT2) corresponding to the query image obtained from the query image vector generation unit (230) will be explained in detail through FIG. 6, which will be described later.

[0047] When a second vector information (VT2) corresponding to the query image content is obtained from the query image vector generation unit (230), the illegality determination unit (240) may transmit a comparison target information request (RQ) corresponding to the content to the database (220). In an embodiment, the comparison target information request (RQ) may include metadata corresponding to the query image content. In response to the comparison target information request (RQ), the database (220) may provide comparison target information (RS) corresponding to the image content to the illegality determination unit (240) based on the metadata corresponding to the query image content. Specifically, the database (220) may select a comparison target image content by matching the metadata corresponding to the original image content stored in the database with the metadata corresponding to the query image content, and provide comparison target information (RS) including vector information corresponding to the selected comparison target image content to the illegality determination unit (240). The illegality determination unit (240) can identify whether the query video content is illegally copied based on a comparison of similarity between the second vector information (VT2) corresponding to the query video content and the comparison target information (RS), and the specific operation for identifying whether it is illegally copied will be described in detail in FIG. 3, which will be described later.

[0048] FIG. 3 is a flowchart illustrating the operation method of an electronic device (130, see FIG. 1) for identifying illegally copied images according to an embodiment of the present disclosure.

[0049] In step S310, the original image vector generation unit (310) included in the electronic device (130) according to an embodiment of the present disclosure may obtain original image information from a first external device (110, see FIG. 1). The original image information may include metadata for identifying the content of the original image content.

[0050] In step S320, the original image vector generation unit (310) included in the electronic device (130) according to an embodiment of the present disclosure can generate vector information corresponding to the original image content based on the original image information. The method for generating vector information corresponding to the original image content will be explained in detail through FIGS. 4 and 5, which will be described later.

[0051] In step S330, the original image vector generation unit (310) included in the electronic device (130) according to an embodiment of the present disclosure may store vector information corresponding to the original image content generated in step S320 in the database (320). Additionally, the original image vector generation unit (310) may provide metadata corresponding to the image to the database (320), and the database may store the metadata by matching it with the vector information.

[0052] In step S340, the query image vector generation unit (330) according to an embodiment of the present disclosure may obtain query image information for a real-time streaming video that is subject to identification of whether it is illegally copied from a second external device (120, see FIG. 1). The query image information may include URL information for the real-time streaming video and metadata for the video.

[0053] In step S350, the query image vector generation unit (330) included in the electronic device (130) according to an embodiment of the present disclosure can generate vector information corresponding to the query image content based on the query image information. The method of generating vector information corresponding to the query image content will be explained in detail through FIG. 6, which will be described later.

[0054] In step S360, the query image vector generation unit (330) included in the electronic device (130) according to an embodiment of the present disclosure may provide vector information corresponding to the query image content generated in step S350 to the illegality determination unit (340). Additionally, the query image vector generation unit (330) may provide metadata corresponding to the query image content to the illegality determination unit (340).

[0055] In step S370, the illegality determination unit (340) included in the electronic device (130) according to an embodiment of the present disclosure may request comparison target information from the database (320). In the embodiment, the comparison target information may refer to information regarding an image corresponding to the query image content among the vector information corresponding to the original image content stored in the database (320), and the comparison target information may be verified based on metadata for the query image content and metadata for the original image content.

[0056] In step S380, the database (320) included in the electronic device (130) according to an embodiment of the present disclosure may respond with comparison target information in response to a request from the illegality determination unit (340). The comparison target information provided to the illegality determination unit (340) may include vector information for the original image content corresponding to the query image content.

[0057] In step S390, the illegality determination unit (340) included in the electronic device (130) according to an embodiment of the present disclosure can determine (identify) whether the query video content is illegally copied based on vector information and comparison target information regarding the query video content. In an embodiment, the illegality determination unit (340) can determine whether the content is illegally copied based on frame-by-frame similarity between the vector information regarding the query video content and the original video content. In some embodiments, the illegality determination unit (340) can derive similarity based on the Euclidean distance between the vector information regarding the query video content and the vector information regarding the original video content.

[0058] Specifically, the illegality determination unit (340) according to an embodiment of the present disclosure can determine the similarity for each of the plurality of frames derived from the query video content. For example, when the illegality determination unit (340) determines the similarity for the first frame of the query video content, it determines the similarity based on the Euclidean distance between the vector information for the first frame of the query video content and the vector information for the first frame of the original video content corresponding to the first frame of the query video content, and when determining the similarity for the second frame of the query video content, it determines the similarity based on the Euclidean distance between the vector information for the second frame of the query video content and the vector information for the second frame of the original video content corresponding to the second frame of the query video content. In an embodiment, if it is determined that N (N is a natural number) or more frames have a similarity greater than or equal to a preset threshold, the illegality determination unit (340) can identify the query video content as an illegally copied video. In some embodiments, N may be set to a natural number greater than or equal to 3.

[0059] The aforementioned steps S310 to S330 may be performed independently of steps S340 to S390, steps S310 to S330 correspond to the operation of building a database, and steps S340 to S390 correspond to the operation of identifying whether the query video has been illegally copied. The operation of building a database may be performed repeatedly even while the operation of identifying whether the query video content has been illegally copied is not being performed.

[0060] FIG. 4 is a flowchart illustrating a method for generating vector information for an original image according to an embodiment of the present disclosure. FIG. 5 is a diagram illustrating, by example, a process for generating vector information for an original image according to an embodiment of the present disclosure. Steps S410 to S450 illustrated in FIG. 4 may correspond to step S320 of FIG. 3 described above. Hereinafter, with reference to FIG. 4 and FIG. 5, a method for generating vector information for an original image according to an embodiment of the present disclosure will be described.

[0061] In step S410, the original image vector generation unit (210, see FIG. 2) of the electronic device (130, see FIG. 1) according to an embodiment of the present disclosure may obtain original image information from a first external device (110, see FIG. 1). In the embodiment, the original image information may mean image information for image content to which a legal copyright has been granted, and may be understood as a concept including first metadata for identifying the content of the original image content.

[0062] In step S420, the original image vector generation unit (210) according to an embodiment of the present disclosure can extract a frame set for the original image from the original image information. Specifically, the original image vector generation unit (210) can extract a plurality of frames according to a specified period for the original image information and can construct a frame set including the extracted plurality of frames. In one embodiment, the original image vector generation unit (210) can extract one frame per second from the original image information. For convenience of explanation, it will be assumed that frames A1, A2, A3, A4, B1, B2, B3, C1, C2, C3, D1, and D2 have been derived from the original image information, and accordingly, the frame set can be composed of [A1, A2, A3, A4, B1, B2, B3, C1, C2, C3, D1, D2]. Here, frames represented by the same alphabet signify that they are frames extracted from the same scene.

[0063] Meanwhile, in some embodiments, the original image vector generation unit (210) may form a frame set by extracting a scene frame representing a specific scene from the original image information. When a frame is sampled from the original image information according to the above-described premise, the frame set formed by extracting a scene frame representing a specific scene may be composed of [A1, B1, C1, D1].

[0064] In step S430, the original image vector generation unit (210) according to an embodiment of the present disclosure may remove color information for each of the plurality of frames included in the frame set for the original image information. Referring to FIG. 5, the frame extracted from the original image information in step S420 as in FIG. 5 (a) may be converted to grayscale by removing color information as in FIG. 5 (b).

[0065] In step S440, the original image vector generation unit (210) according to an embodiment of the present disclosure performs a Fourier transform on each of the plurality of frames included in the frame set converted to grayscale, and can separate the frequency domain based thereon. In some embodiments, a Fast Fourier Transform (FFT) may be used for the Fourier transform. Referring to FIG. 5, the frame converted to grayscale shown in FIG. 5 (b) can be derived in a form in which the frequency domain is separated by the Fourier transform as in FIG. 5 (c).

[0066] In step S450, the original image vector generation unit (210) according to an embodiment of the present disclosure may rearrange the frequency components derived from step S440 around the low-frequency components, extract a center region, and generate vector information based thereon. Specifically, as shown in FIG. 5 (c), the frame of the frequency domain derived by the Fourier transform may form a pattern in which low-frequency components are skewed to the corners constituting the frame, and the original image vector generation unit (210) may move the low-frequency components skewed to the corners constituting the frame to the center of the frame as shown in FIG. 5 (d). In addition, the original image vector generation unit (210) may extract a pre-set center region as shown in FIG. 5 (e) to extract the low-frequency components moved to the center. In one embodiment, the size of the center area may be set to 30x30, and in order to extract the center area, the original image vector generation unit (210) may extract the center area by performing a crop operation on (d) of FIG. 5, in which the low-frequency component is moved to the center of the frame. In one embodiment, the original image vector generation unit (210) may convert the extracted center area into a one-dimensional vector, and the converted vector value may be provided to a database (220, see FIG. 2) as vector information for the corresponding frame.

[0067] As assumed in step S420, if the frame set for the original image is composed of [A1, A2, A3, A4, B1, B2, B3, C1, C2, C3, D1, D2], vector information for each of the multiple frames included in the frame set can be derived, and [a1, a2, a3, b1, b2, b3, c1, c2, c3, d1, d2], composed of vector information corresponding to each frame, can be provided to the database (220) as vector information for the image.

[0068] FIG. 6 is a flowchart illustrating a method for generating vector information for a query image according to an embodiment of the present disclosure. Steps S610 to S650 shown in FIG. 6 may correspond to step S350 of FIG. 3 described above. In the embodiment, regarding the method for generating vector information for a query image, the method for generating vector information for an original image described above may be applied, and below, details that overlap with the principles described in FIG. 4 and 5 described above will be omitted.

[0069] In step S610, a query image vector generation unit (230, see FIG. 2) of an electronic device (130, see FIG. 1) according to an embodiment of the present disclosure may obtain query image information from a second external device (120, see FIG. 1). In the embodiment, the query image information may refer to image information regarding a target image content for which illegal copying is to be verified, and may be understood as image information regarding streaming image content, including URL information regarding streaming image content.

[0070] In step S620, the query image vector generation unit (230) according to an embodiment of the present disclosure may extract a frame set for the query image from the query image information. Specifically, the query image vector generation unit (230) may extract a plurality of frames according to a specified period for the query image information and may form a frame set including the extracted plurality of frames. Alternatively, in some embodiments, the query image vector generation unit (230) may form a frame set by extracting a scene frame representing a specific scene from the query image information.

[0071] In step S630, the query image vector generation unit (230) according to an embodiment of the present disclosure may remove color information for each of the plurality of frames included in the frame set for the query image information. The query image vector generation unit (230) may convert the extracted frames into grayscale.

[0072] In step S640, the query image vector generation unit (230) according to an embodiment of the present disclosure performs a Fourier transform on each of the plurality of frames included in the frame set converted to grayscale, and can separate the frequency domain based thereon. In some embodiments, a Fast Fourier Transform may be used for the Fourier transform. The query image vector generation unit (230) can separate the frequency domain for the corresponding frame using the Fourier transform.

[0073] In step S650, the query image vector generation unit (230) according to an embodiment of the present disclosure may rearrange the frequency components derived from step S640 around low-frequency components, extract a center region, and generate vector information based thereon. In the embodiment, the size of the center region may be pre-set, and the query image vector generation unit (230) may convert the extracted center region into a one-dimensional vector, and the converted vector value may be used as vector information for the corresponding frame.

[0074] FIG. 7 is a block diagram for explaining an electronic device (700) according to an embodiment of the present disclosure.

[0075] The electronic device (700) illustrated in FIG. 7 may correspond to the electronic device (130, see FIG. 1) described above in FIG. 1, and with reference to FIG. 7, the electronic device (700) according to an embodiment of the present disclosure may include a transceiver (710), a processor (720), and a memory (730).

[0076] The electronic device (700) is connected to a first external device (110, see FIG. 1) and a second external device (120, see FIG. 2) through a transceiver (710) and can exchange data.

[0077] The processor (720) may perform an operation performed by at least one device described through FIGS. 1 to 6 described above, or perform at least one method described through FIGS. 1 to 6. Additionally, the processor (720) may execute a program to perform an operation performed by at least one device described through FIGS. 1 to 6 described above, or to perform at least one method described through FIGS. 1 to 6, and may process information and control an electronic device (700) to perform an operation performed by at least one device described through FIGS. 1 to 6 described above or at least one method described through FIGS. 1 to 6.

[0078] The memory (730) may be volatile memory or non-volatile memory. Additionally, the memory (730) may store the code of a program executed by the processor (720).

[0079] FIG. 8 is a table for explaining the results of a performance comparison of an electronic device (130, see FIG. 1) according to an embodiment of the present disclosure.

[0080] The table illustrated in FIG. 8 is a table for comparing the performance of the illegal image identification method according to the present invention with various feature extraction methods for identifying illegal images. FIG. 8 compares the performance of the illegal image identification method (FFT) according to the present invention with existing feature extraction methods such as AKAZE, SURF, SIFT, and ORB. According to the illegal image identification method (FFT) according to the present invention, the amount of feature information used to identify illegal images can be reduced by approximately 8 to 93 times, thereby creating the effect of reducing computational costs and enabling rapid image identification even in real-time streaming environments.

[0081] Meanwhile, the embodiments disclosed in this specification may be implemented in the form of a recording medium that stores instructions executable by a computer. The instructions may be stored in the form of program code and, when executed by a processor, may generate a program module to perform the operations of the disclosed embodiments. The recording medium may be implemented as a computer-readable recording medium. A computer-readable recording medium may include all types of recording media that store instructions decipherable by a computer. Examples include ROM, RAM, magnetic tape, magnetic disk, flash memory, optical data storage devices, etc.

[0082] The above descriptions are specific embodiments for carrying out the present disclosure. The present disclosure will include not only the embodiments described above, but also embodiments that can be simply modified or easily modified. Furthermore, the present disclosure will include technologies that can be easily modified and implemented using the embodiments described above. Accordingly, the scope of the present disclosure should not be limited to the embodiments described above, but should be defined by the claims set forth below as well as equivalents to the claims of the present disclosure. Explanation of the symbols

[0083] 10 : System 110: First external device 120 : Second external device 130 : Electronic device

Claims

Claim 1 A method of operation for an electronic device for identifying illegally copied video, comprising: a step of obtaining query video information for a target video content to be checked for illegal copying; a step of deriving first vector information corresponding to the query video information using a Fourier transform; a step of requesting comparison target information from a database based on first metadata included in the query video information; and a step of identifying whether the target video content is illegally copied based on the comparison target information and the first vector information. Claim 2 A method of operation according to claim 1, wherein the step of deriving first vector information corresponding to the query image information using the Fourier transform comprises: a step of deriving a query image frame based on the query image information; a step of removing color information for the query image frame; a step of separating the frequency domain by performing the Fourier transform on the query image frame from which the color information has been removed; and a step of deriving the first vector information based on the frequency domain. Claim 3 In claim 2, the step of deriving the first vector information based on the frequency domain comprises: a step of rearranging the frequency domain around a low-frequency component; a step of extracting a center region for a query image frame rearranged around the low-frequency component; and a step of deriving the first vector information based on the center region. Claim 4 In claim 3, the above-mentioned central area is derived by cropping based on a preset size. Claim 5 A method of operation according to claim 1, wherein the database stores second vector information corresponding to the original image content, and the second vector information is derived from the original image information for the original image content using the Fourier transform. Claim 6 In claim 5, the comparison target information comprises the second vector information derived from the original image information, which includes the second metadata corresponding to the first metadata. Claim 7 In claim 6, the step of identifying whether the target video content is illegally copied includes a method of operation that includes a step of confirming the similarity between the first vector information and the second vector information. Claim 8 In claim 1, the operation method wherein the Fourier transform is a Fast Fourier Transform (FFT). Claim 9 An electronic device for identifying illegally copied video, comprising a transceiver, a memory for storing instructions, and at least one processor, wherein the at least one processor connected to the transceiver and the memory comprises: acquiring query video information for a target video content to be checked for illegal copying, deriving vector information corresponding to the query video information using a Fourier transform, requesting comparison target information from a database based on metadata included in the query video information, and identifying whether the target video content is illegally copied based on the comparison target information and the vector information. Claim 10 A computer program stored on a computer-readable storage medium, wherein, when executed on at least one processor, the computer program performs the following operations to identify illegally copied images, the operations comprising: an operation of obtaining query image information for a target image content to be verified for illegal copying; an operation of deriving vector information corresponding to the query image information using a Fourier transform; an operation of requesting comparison target information from a database based on metadata included in the query image information; and an operation of identifying whether the target image content is illegally copied based on the comparison target information and the vector information.

Images