Electronic device and operation method thereof

Abstract

Disclosed is an electronic device, the electronic device according to the present disclosure comprising: a communication unit; a memory for storing at least one instruction; and at least one processor for executing the at least one instruction stored in the memory, wherein the at least one processor: transmits a fingerprint obtained from output content to a server through a communication unit at first transmission intervals; receives identification information obtained through automation content recognition (ACR) performed on the basis of the fingerprint, from the server through the communication unit; and changes the transmission intervals of the fingerprint obtained from the output content on the basis of the received identification information.

Description

Electronic device and method of operation thereof

[0001] The disclosed various embodiments relate to an electronic device and a method of operating the same, and more particularly, to an electronic device capable of performing more accurate automatic content recognition of content being viewed, and a method of operating the same.

[0002] With the advancement of technology, Automatic Content Recognition technology, which can recognize content displayed on electronic devices such as televisions, is being developed.

[0003] Fingerprinting technology, a type of automatic content recognition technology, is a technology that identifies content displayed on television by extracting fingerprints from the content and matching them to a database.

[0004] Electronic devices can extract fingerprints from currently displayed content and transmit them to a server performing ACR. The server can compare the fingerprints received from the electronic device with a database to identify the content and transmit the identification information to the electronic device. Based on the identification information received from the server, the electronic device can recognize in real time what content is displayed on the screen, thereby providing context-sensitive content.

[0005] An electronic device according to an embodiment may include a communication unit, a memory storing at least one instruction, and at least one processor executing the at least one instruction stored in the memory.

[0006] In an embodiment, the at least one processor may transmit a fingerprint obtained from the output content to the server through the communication unit every first transmission cycle.

[0007] In an embodiment, the at least one processor may receive identification information obtained through ACR (Automation Content Recognition) performed based on the fingerprint from the server through the communication unit.

[0008] In an embodiment, the at least one processor may change the transmission cycle of a fingerprint obtained from the output content based on the received identification information.

[0009] Figure 1 is a diagram explaining the content recognition rate according to the query cycle.

[0010] Figure 2 is a diagram explaining the content recognition rate according to the query cycle.

[0011] Figure 3 illustrates an ACR system according to an embodiment.

[0012] Figure 4 is a block diagram of an electronic device according to an embodiment.

[0013] FIG. 5 illustrates an electronic device and a server performing ACR according to an embodiment.

[0014] Figure 6 is an internal block diagram of an electronic device according to an embodiment.

[0015] FIG. 7 is a diagram explaining the content recognition rate according to the query cycle when performing ACR according to an embodiment.

[0016] FIG. 8 is a diagram explaining the content recognition rate according to the query cycle when performing ACR according to an embodiment.

[0017] Figure 9 is a flowchart illustrating an operation method of an electronic device according to an embodiment.

[0018] FIG. 10 is a flowchart illustrating a method for predicting whether output content is target content, according to an embodiment.

[0019] Fig. 11 is a flowchart illustrating an operating method according to an embodiment.

[0020] According to an embodiment, a method of operating an electronic device may include a step of transmitting a fingerprint obtained from output content to a server at each first transmission cycle.

[0021] According to an embodiment, a method of operating an electronic device may include a step of receiving identification information obtained through Automation Content Recognition (ACR) performed based on the fingerprint from the server.

[0022] According to an embodiment, a method of operating an electronic device may include a step of changing a transmission cycle of a fingerprint obtained from the output content based on the received identification information.

[0023] A recording medium according to an embodiment may be a computer-readable recording medium having recorded thereon a program that can perform a method of operating an electronic device, including a step of transmitting a fingerprint obtained from output content to a server at every first transmission cycle, by a computer.

[0024] In an embodiment, the recording medium may be a computer-readable recording medium having recorded thereon a program that can perform a method of operating an electronic device, including a step of receiving identification information obtained through ACR (Automation Content Recognition) performed based on the fingerprint from the server.

[0025] In an embodiment, the recording medium may be a computer-readable recording medium having recorded thereon a program that can perform a method of operating an electronic device, including a step of changing a transmission cycle of a fingerprint obtained from the output content based on the received identification information, by a computer.

[0026] In this disclosure, the expression “at least one of a, b or c” may refer to “a”, “b”, “c”, “a and b”, “a and c”, “b and c”, “all of a, b and c”, or variations thereof.

[0027] Below, embodiments of the present disclosure are described in detail with reference to the attached drawings so that those skilled in the art can easily implement the present disclosure. However, the present disclosure may be implemented in various different forms and is not limited to the embodiments described herein.

[0028] The terms used in this disclosure are described as currently common terms, taking into account the functions mentioned herein. However, these terms may mean various other terms depending on the intentions of those skilled in the art, precedents, the emergence of new technologies, etc. Therefore, the terms used in this disclosure should not be interpreted solely based on their names, but rather based on the meanings of the terms and the overall content of this disclosure.

[0029] Additionally, the terminology used in this disclosure is for the purpose of describing specific embodiments only and is not intended to limit the present disclosure.

[0030] Throughout the specification, when a part is said to be "connected" to another part, this includes not only the cases where it is "directly connected" but also the cases where it is "electrically connected" with another element in between.

[0031] As used herein, and particularly in the claims, the terms "above" and "above" and similar referents may refer to both the singular and the plural. Furthermore, unless the order of steps in a method according to the present disclosure is explicitly specified, the steps described may be performed in any appropriate order. The present disclosure is not limited by the order in which the steps are described.

[0032] The appearances of phrases such as “in some embodiments” or “in an embodiment” in various places throughout this specification are not necessarily all referring to the same embodiment.

[0033] Some embodiments of the present disclosure may be represented by functional block configurations and various processing steps. Some or all of these functional blocks may be implemented by various hardware and / or software configurations that perform specific functions. For example, the functional blocks of the present disclosure may be implemented by one or more microprocessors or by circuit configurations for a given function. Furthermore, for example, the functional blocks of the present disclosure may be implemented in various programming or scripting languages. The functional blocks may be implemented by algorithms that execute on one or more processors. Furthermore, the present disclosure may employ conventional techniques for electronic configuration, signal processing, and / or data processing. Terms such as “mechanism,” “element,” “means,” and “configuration” may be used broadly and are not limited to mechanical and physical configurations.

[0034] Additionally, the connecting lines or connecting members between components depicted in the drawings are merely exemplary representations of functional connections and / or physical or circuit connections. In an actual device, connections between components may be represented by various functional connections, physical connections, or circuit connections that may be replaced or added.

[0035] Additionally, terms such as “part”, “module”, etc. described in the specification mean a unit that processes at least one function or operation, which may be implemented as hardware or software, or a combination of hardware and software.

[0036] Additionally, the term “user” in the specification means a person who uses an electronic device, and may include a consumer, evaluator, viewer, administrator, or installer.

[0037] The present disclosure will be described in detail with reference to the attached drawings below.

[0038] The present disclosure may relate to artificial intelligence (AI) systems and / or applications of AI systems utilizing machine learning algorithms.

[0039] Figure 1 is a diagram explaining the content recognition rate according to the query cycle.

[0040] Electronic devices can capture content displayed on the screen and extract fingerprints from the captured images.

[0041] Electronic devices can transmit their fingerprints to a server that performs Automatic Content Recognition (ACR) technology.

[0042] The server can compare the fingerprint received from the electronic device with fingerprints stored in a database to search for a matching fingerprint. The server can transmit identification information about the found fingerprint to the electronic device.

[0043] Figure 1 illustrates an electronic device transmitting a fingerprint to a server at a regular query cycle.

[0044] An electronic device can send queries based on a query cycle. For example, in FIG. 1, the electronic device can transmit a fingerprint to the server every m seconds.

[0045] Assume that the content displayed by an electronic device is Program A. At time t1, which is a point in time according to the query cycle, the electronic device can transmit a fingerprint extracted from the screen to a server. The server can then search for the fingerprint transmitted at time t1 in a database, obtain identification information indicating that the content corresponding to the fingerprint is Program A, and transmit this information to the electronic device. The electronic device can then recognize that the identified content displayed on the screen is Program A based on the identification information received from the server.

[0046] After m seconds have elapsed and the time point t2, which is the query cycle time, arrives, the electronic device can transmit the fingerprint extracted from the screen to the server. The server can perform ACR on the fingerprint transmitted at time t2 and transmit identification information to the electronic device indicating that the content corresponding to the fingerprint transmitted at time t2 is Program A.

[0047] Afterwards, assume that the electronic device terminates the output of program A, outputs advertisement B, and then outputs another program C.

[0048] At time t3, which is a point in time according to the query cycle, the electronic device can transmit a fingerprint to the server. The server can perform ACR on the fingerprint transmitted at time t3 to obtain identification information indicating that the fingerprint transmitted at time t3 corresponds to Program C and transmit this information to the electronic device. Based on the identification information received from the server, the electronic device can recognize that the identification content displayed on the screen corresponds to Program C.

[0049] Unlike program A, advertisement B's content output duration is shorter than the query period of m seconds. In this case, the electronic device cannot identify advertisement B as being displayed on the screen because it does not extract fingerprints while advertisement B is being displayed.

[0050] In this way, if the query cycle is longer than the content output time, there is a problem of the recognition rate for the content being low.

[0051] Figure 2 is a diagram explaining the content recognition rate according to the query cycle.

[0052] In FIG. 2, the electronic device can transmit a fingerprint to the server according to a certain query cycle. For example, the electronic device can send a query to the server every n seconds.

[0053] n seconds may be a shorter time than m seconds, which is the query period described in Figure 1.

[0054] In FIG. 2, while the electronic device is outputting program A, at a point in time corresponding to the query cycle, a fingerprint extracted from the captured screen can be transmitted to the server. The electronic device can transmit the fingerprint to the server at points t1, t2, t3, and t4 at n-second intervals corresponding to the query cycle.

[0055] The server can search the database for fingerprints received from the electronic device at times t1, t2, t3, and t4, and transmit identification information to the electronic device indicating that the fingerprints received at times t1, t2, t3, and t4 correspond to program A.

[0056] The electronic device can recognize that the identification content displayed on the screen is program A from the identification information received from the server.

[0057] Afterwards, assume that the electronic device terminates the output of program A and outputs advertisement B.

[0058] At time t5, which corresponds to a query cycle, the electronic device can transmit a fingerprint to the server. The server can perform ACR on the fingerprint transmitted at time t5 to obtain identification information indicating that the content corresponding to the fingerprint transmitted at time t5 is advertisement B, and transmit this information to the electronic device.

[0059] After outputting advertisement B, the electronic device may output program C and then transmit a fingerprint to the server again at times t6 to t9. The server may perform ACR on the fingerprint transmitted at times t6 to t9 to obtain identification information that the content corresponding to the fingerprint transmitted at times t6 to t9 is program C, and transmit this to the electronic device.

[0060] In this way, if the query cycle is shorter than the output time of Advertisement B, even content that is output for a short time, such as Advertisement B, can be identified, thereby increasing the content recognition rate.

[0061] However, if the query cycle is short, the electronic device sends queries to the server more frequently, so there is a problem that the server has to process more queries.

[0062] That is, when an electronic device transmits a query as illustrated in FIG. 1, the server receives three queries from the electronic device during the same time period, so only three queries need to be processed. However, when an electronic device transmits a query as illustrated in FIG. 2, the server receives nine queries from the electronic device during the same time period, so nine queries need to be processed. Therefore, the server's computational load increases, which leads to an increase in server costs.

[0063] Accordingly, a technology is required that can recognize even content that is output only for a short time without increasing server costs.

[0064] Figure 3 illustrates an ACR system according to an embodiment.

[0065] Referring to FIG. 3, an ACR system according to an embodiment may include an electronic device (100) and a server (200).

[0066] The electronic device (100) according to the embodiment may be an electronic device capable of outputting content.

[0067] In embodiments, the content may be in various forms, such as video, audio, subtitles, other additional information, etc., such as still images or moving images.

[0068] In an embodiment, the electronic device (100) may be a video display device capable of outputting images or videos through a display, and / or an audio device capable of outputting audio. The electronic device (100) may be stationary or mobile.

[0069] In an embodiment, the electronic device (100) may include at least one of a television, a desktop, a smartphone, a tablet personal computer, a game console, an audio device, a mobile phone, a video phone, an e-book reader, a laptop personal computer, a netbook computer, a digital camera, a personal digital assistant (PDA), a portable multimedia player (PMP), a camcorder, a navigation device, a wearable device, a smart watch, a home network system, a security system, and a medical device.

[0070] When the electronic device (100) is a video display device, the electronic device (100) may be implemented as a flat display device, a curved display device having a screen with a curvature, or a flexible display device whose curvature can be adjusted. The output resolution of the electronic device (100) may have various resolutions, such as, for example, HD (High Definition), Full HD, Ultra HD, or a resolution clearer than Ultra HD.

[0071] In an embodiment, if the electronic device (100) is a television (TV), the electronic device (100) may include a digital TV equipped with an operating system (OS) and an Internet connection function.

[0072] The electronic device (100) can output various types of content provided by content providers.

[0073] A content provider may refer to a terrestrial broadcaster, cable broadcaster, satellite broadcaster, IPTV (Internet Protocol Television) service provider, OTT (Over the Top) service provider, or server operator that provides various types of content to consumers.

[0074] Content can take many forms, including video, including still images or moving images, audio, subtitles, and other additional information.

[0075] Content can include visual content like movies and dramas, audible content like music, gaming content, and artistic content introducing or guiding works of art like famous paintings or sculptures. Visual content can include various types of programs and advertisements.

[0076] In an embodiment, the electronic device (100) can receive and output various contents generated by a content provider through an external device. The external device can be implemented as a source device of various forms, such as a PC, a set-top box, a Blu-ray disc player, a mobile phone, a game console, a home theater, an audio player, a USB, etc.

[0077] An external device can be connected to the electronic device (100) via a wired or wireless communication network, such as HDMI, and provide various contents to the electronic device (100). The electronic device (100) can receive VOD (Video On Demand) contents provided by an IPTV service provider or an OTT service provider via a set-top box and output the same. The VOD service is a service that provides a user with desired video at a desired time through a communication network connection, and can include various types of contents provided by an OTT service provider or an IPTV service provider. The IPTV service provider or an OTT service provider can provide not only VOD contents but also real-time broadcast programs.

[0078] In an embodiment, when an operating system is installed in the electronic device (100), the electronic device (100) can stream and output various types of VOD content created by an OTT service provider using the operating system installed therein in addition to real-time broadcasting programs.

[0079] In an embodiment, the electronic device (100) can connect to the Internet and provide web surfing services, social network services, etc. In addition, the electronic device (100) can function as a communication center that can check news, weather, email, etc. in real time.

[0080] In an embodiment, the electronic device (100) can execute various types of applications. The electronic device (100) may have various types of apps (applications) installed by default. Alternatively, the electronic device (100) may, under user control, access the Internet, search for apps requested by the user, and install them. The electronic device (100) can execute apps to provide various services.

[0081] In an embodiment, the electronic device (100) may be wirelessly connected to an external device, such as a set-top box, through a wireless network that follows a communication standard such as Bluetooth, WLAN (Wireless LAN) (Wi-Fi), Wibro (Wireless broadband), Wimax (World Interoperability for Microwave Access), CDMA, or WCDMA, and may receive a video signal from the external device.

[0082] In an embodiment, the electronic device (100) may be connected to an external device via a wired cable to receive a video signal from the external device or transmit a video signal to the external device. The wired cable may include a port capable of simultaneously transmitting a video signal and an audio signal, such as a High-Definition Multimedia Interface (HDMI), a Universal Serial Bus (USB), a Display Port (DP), or Thunderbolt™. Alternatively, the wired cable may include a port for separately transmitting a video signal and an audio signal.

[0083] In an embodiment, the electronic device (100) may be implemented as an electronic device that does not include a video display device such as a display. For example, if the electronic device (100) is an external device itself, such as a set-top box, a satellite broadcast receiving device, an Internet receiving device that receives content from an OTT (Over The Top) service provider, the electronic device (100) may be of a form that does not include a video display device.

[0084] In this case, the electronic device (100) can be connected to the video display device via a wire, and transmit a signal input from an external source to the video display device. For example, the electronic device (100) can be connected to the video display device, such as a monitor, using a port that can simultaneously transmit video signals and audio signals, such as USB (Universal Serial Bus), HDMI (High-Definition Multimedia Interface), DP (Display Port), Thunderbolt™, etc. Alternatively, the electronic device (100) can be connected to the video display device using a port that separately transmits video signals and audio signals.

[0085] The electronic device (100) can be controlled by a control device. The control device may be a device used to control the electronic device (100), such as a remote controller. A user can control various functions of the electronic device (100) using the control device.

[0086] The control device may be a dedicated control device for the electronic device (100). Alternatively, in an embodiment, the control device may be an electronic device, such as a smartphone or an AI speaker, whose main function is to perform operations other than controlling the electronic device (100). A user may control the electronic device (100) using the control device by installing a remote control app or the like on the control device. In this case, the control device may include at least one module capable of performing Wi-Fi, Bluetooth, or infrared communication. The control device may transmit and receive data with the electronic device (100) using a Wi-Fi, Bluetooth, or infrared communication module.

[0087] The control device may have an input unit. The input unit may receive user input for controlling the electronic device (100). The input unit included in the control device may include a plurality of keys. The keys may take various forms, such as physical buttons that receive a user's push operation, a jog shuttle, or touch buttons displayed on a touchpad that detects touch.

[0088] At least one of the electronic device (100) or the control device may include a microphone capable of receiving an acoustic signal. If the control device includes a microphone, the microphone may receive an analog signal, digitize the analog signal, and transmit the analog signal to the electronic device (100). In an embodiment, the control device may receive an acoustic signal through the microphone, digitize the analog signal, and transmit the digitized acoustic signal to the electronic device (100) using a data transmission communication method such as Bluetooth or Wi-Fi.

[0089] In an embodiment, the electronic device (100) and the server (200) may provide an automatic content recognition function. Automatic Content Recognition (ACR) is an identification technology that recognizes content played on the electronic device (100). Through the automatic content recognition technology, the electronic device (100) can quickly obtain detailed information about the output content without text-based input or search.

[0090] The electronic device (100) can acquire viewer measurement indicators indicating viewer behavior or tendencies in real time through automatic content recognition technology. The electronic device (100) can analyze the viewer measurement indicators to select customized advertisements suited to the viewer and provide them to the viewer.

[0091] In an embodiment, the electronic device (100) can capture a screen currently being output through the screen and extract a fingerprint from the captured screen to perform automatic content recognition technology.

[0092] In an embodiment, the electronic device (100) can identify the point at which advertising content is output during a live broadcast. The electronic device (100) can identify that advertising content is output when inserted between different programs during a live broadcast, or at the midpoint of the same program.

[0093] In an embodiment, the electronic device (100) can identify that advertising content is output before, after, or during content output, even when outputting VOD content provided by an IPTV service provider or an OTT service provider rather than a real-time broadcast.

[0094] In an embodiment, the electronic device (100) may extract a fingerprint from the output content at regular intervals, for example, at a first query interval, and transmit the fingerprint to the server (200). The first query interval may also be referred to as a first transmission interval.

[0095] In an embodiment, the electronic device (100) can request an ACR result from the server (200) by transmitting a fingerprint to the server (200).

[0096] In an embodiment, the electronic device (100) can predict whether the output content is target content.

[0097] In the present disclosure, predicting whether the output content is target content by the electronic device (100) may mean that the electronic device (100) infers, identifies, or determines whether the output content is target content.

[0098] In an embodiment, the target content is the content to be identified, which may include shorter video content than general content, such as advertising content.

[0099] In an embodiment, the target content may be content of a transition section inserted in the middle of a program of a certain length or longer.

[0100] In an embodiment, the target content may include, in addition to advertising content, short content, which is short-form video content, breaking news content, transition screens such as cut scenes displayed when chapters or episodes change in game content, intro screens included in various dramas or movie content, prologue content, epilogue content, closing credits, or end credits.

[0101] In an embodiment, the electronic device (100) can predict whether the content currently output through the screen is target content using various methods.

[0102] In an embodiment, the electronic device (100) can guess whether the output content is target content based on at least one of metadata for the content, a frame included in the content, and a target content pattern.

[0103] In an embodiment, the electronic device (100) may obtain metadata about the content to predict whether the output content is target content.

[0104] In an embodiment, the electronic device (100) can obtain SCTE-35 packet data from metadata for content.

[0105] SCTE-35 is a message that can be included in a source MPEG-2 transport stream, a broadcast standard signal inserted for dynamic advertisement insertion in real-time streaming content. SCTE-35 can be inserted into the transport stream and transmitted along with media content.

[0106] In an embodiment, the electronic device (100) can identify a cue-out event or a cue-in event by parsing the SCTE-35 packet when the metadata for the content includes SCTE-35 packet data.

[0107] A cue out event may indicate the point at which advertising content begins, and a cue in event may indicate the point at which advertising content output ends.

[0108] In an embodiment, the electronic device (100) may detect at least one of a cue out event indicating a point where an advertisement content, which is one of target contents, starts, a cue in event indicating a point where the advertisement content ends, and / or an advertisement interval event indicating a time interval from a cue out point to a cue in point in a SCTE-35 packet.

[0109] In an embodiment, the electronic device (100) may utilize EPG (Electronic Program Guide) data among the metadata for content. EPG data is information about broadcast programs and may be received from a server providing the broadcast program, a content provider server, etc. The broadcast schedule may include information on the time at which the program is output.

[0110] In an embodiment, the electronic device (100) can obtain information on the output time of a program using EPG data, and identify a point at which advertising content is likely to be output, such as a point before the start of a program, a point after the end of a program, or a point in the middle of a program.

[0111] In an embodiment, the electronic device (100) may use EPG data to predict the interval between the times at which different programs are output as the target content output time. Alternatively, the electronic device (100) may determine that advertising content may be output at the midpoint, 1 / 3 point, 1 / 4 point, etc. of the time at which a program is output, and may predict the midpoint, 1 / 3 point, or 1 / 4 point of the program output time as the target content output time.

[0112] In an embodiment, the electronic device (100) can predict whether the output content is target content based on the amount of change and / or distribution of information in the frames included in the content. The output content may be content output through the screen / screen / display of the electronic device (100). Hereinafter, the content output from the electronic device (100) will be referred to as output content.

[0113] In an embodiment, the information of the frame may include at least one of color, brightness, luminance, and contrast of the frame.

[0114] In an embodiment, the electronic device (100) may transmit content to an external server and receive an analysis result based on at least one of the amount of change and distribution of frame information included in the content from the external server. The electronic device (100) may receive the analysis result from the external server and, based on the analysis result, predict whether the output content is target content.

[0115] In an embodiment, the electronic device (100) may also use artificial intelligence technology to predict whether the output content is target content.

[0116] In an embodiment, the electronic device (100) can predict whether the output content is target content by using a deep learning model that has learned a content pattern from an image.

[0117] In an embodiment, the content pattern may represent the timing of the content output or the unique characteristics of the content. In an embodiment, the neural network may be trained to acquire the patterns of the target content by learning the timing of advertisement insertion during a program or the screen characteristics of the advertisement content.

[0118] In an embodiment, the electronic device (100) inputs content currently output on the screen into a neural network, and can predict whether the output content is target content by using whether a pattern corresponding to the target content is detected from the neural network.

[0119] Typically, content such as advertisements or breaking news is displayed on screens multiple times a day. In an example, a neural network can recognize repeated detections of the same fingerprint as a target content pattern and predict that the displayed content is the target content.

[0120] In an embodiment, the electronic device (100) can use metadata about the content, frames included in the content, and target content patterns together to predict whether the output content is target content.

[0121] In an embodiment, the electronic device (100) may obtain a score related to whether the output content corresponds to the target content based on each of metadata for the content, frames included in the content, and target content patterns.

[0122] In an embodiment, the electronic device (100) may obtain a score related to whether each item of metadata for the content, a frame included in the content, and a target content pattern corresponds to the target content.

[0123] In an embodiment, the electronic device (100) can apply an item-specific weight to an item-specific score, and predict whether the output content is target content based on a weighted sum of the item-specific scores to which the item-specific weights are applied.

[0124] In an embodiment, the electronic device (100) can predict at least one of a target content start time, a target content section, and a target content end time from the output content.

[0125] In an embodiment, if the electronic device (100) predicts that the output content is content at the start of the target content, it can extract a fingerprint from the output content.

[0126] In an embodiment, if the electronic device (100) predicts that the output content is target content, it can initialize a previously stacked fingerprint.

[0127] In an embodiment, the electronic device (100) can transmit a preset number of fingerprints from an initialized fingerprint stack to the server (200).

[0128] In an embodiment, the electronic device (100) may transmit a fingerprint to the server (200) at each first query cycle, and may request an ACR result from the server (200) by transmitting a fingerprint to the server (200) even before the query transmission time according to the first query cycle, if the output content is predicted to be the target content.

[0129] In an embodiment, when the server (200) receives a fingerprint from the electronic device (100), it can perform ACR using the received fingerprint.

[0130] In an embodiment, the server (200) can variably operate the linkage cycle with the electronic device (100) depending on whether target content is detected in the electronic device (100).

[0131] In an embodiment, the server (200) can compare the received fingerprint with fingerprints stored in a database to detect a matching fingerprint.

[0132] In an embodiment, the server (200) may include a fingerprint DB. In addition, in an embodiment, the server (200) may include a metadata DB in which metadata corresponding to fingerprints are stored.

[0133] In an embodiment, the server (200) can search for a fingerprint received from an electronic device (100) in a fingerprint DB and obtain metadata corresponding to the matching fingerprint from the metadata DB.

[0134] In an embodiment, metadata corresponding to a fingerprint may include identifying information indicating content identified by the fingerprint.

[0135] In an embodiment, the identification information may include a unique ID for the content. The unique ID for the content may include various additional information about the content, such as the type of content, the title or program name of the content, and the name of the content provider, such as the broadcaster, production company, or distributor that produced the content.

[0136] In an embodiment, the server (200) may obtain identification information for a fingerprint from a metadata DB and transmit it to the electronic device (100).

[0137] In an embodiment, if the electronic device (100) does not receive content identification information from the server (200) that the output content is target content, the electronic device (100) may continue to transmit the fingerprint to the server (200) according to the first query cycle.

[0138] In an embodiment, when the server (200) does not receive content identification information indicating that the output content is target content, it may include at least one of the following: before the server (200) receives the identification information indicating that the output content is target content, or when the content identification information received from the server (200) is not content identification information for the target content.

[0139] If the content identification information received from the server (200) is not content identification information for the target content, it may include a case where content identification information is received indicating that the output content does not correspond to the target content.

[0140] In an embodiment, the electronic device (100) may transmit a fingerprint to the server (200) according to the first transmission cycle, which is the default cycle, if the prediction made by the electronic device (100) is incorrect, i.e., if the identification information received from the server (200) is content identification information indicating that the output content is not the target content.

[0141] In an embodiment, when the electronic device (100) receives content identification information from the server (200) indicating that the output content is target content, i.e., when the output content currently being output on the screen is target content as predicted by the electronic device (100), the electronic device (100) may change the first query period to a second query period. In an embodiment, the second query period may be a period faster than the first query period. The second query period may also be referred to as a second transmission period.

[0142] In an embodiment, when the electronic device (100) changes the query cycle to a second query cycle, it can send a query according to the second query cycle.

[0143] In an embodiment, the electronic device (100) may transmit a fingerprint to the server (200) according to the second query cycle and receive an ACR result from the server (200).

[0144] In an embodiment, when the electronic device (100) receives content identification information from the server (200) that is not content identification information for the target content while the current query period is the second query period, for example, when receiving content identification information indicating that the output content does not correspond to the target content, the electronic device (100) may change the second query period back to the first query period.

[0145] In an embodiment, if the electronic device (100) identifies that the output content is content at the end of the target content, it can reset and initialize previously stacked fingerprints and transmit a preset number of fingerprints from the initialized fingerprint stack to the server (200).

[0146] In an embodiment, the electronic device (100) may change the query period from the second query period to the first query period if it identifies that the output content is content at the end point of the target content.

[0147] In this way, according to an embodiment, the electronic device (100) can recognize / predict in advance the time at which target content, such as advertising content, will be output, and variably change the query transmission time accordingly.

[0148] According to an embodiment, since the electronic device (100) normally transmits a query for a fingerprint to the server (200) in the first query cycle, the amount of data calculation of the server (200) can be prevented from unnecessarily increasing.

[0149] According to an embodiment, the electronic device (100) can predict whether output content is target content, and if the output content is predicted to be target content, can extract a new fingerprint and transmit it to the server (200). Accordingly, the electronic device (100) can quickly receive ACR results for content predicted to be target content from the server (200), thereby accurately recognizing target content output for a short period of time without missing it.

[0150] According to an embodiment, when the electronic device (100) receives a result from the server (200) that the output content is target content, the electronic device (100) can correspondingly change the query period from the first query period to a second query period that is faster than the first query period, thereby improving the content recognition rate for the target content.

[0151] Figure 4 is a block diagram of an electronic device (100) according to an embodiment.

[0152] The electronic device (100) of FIG. 4 may be an example of the electronic device (100) of FIG. 3.

[0153] In an embodiment, the electronic device (100) may include a communication unit (130), a memory (120), and a processor (110).

[0154] In an embodiment, the communication unit (130) can connect the electronic device (100) to a peripheral device, an external device, a mobile terminal, a server (200), etc., under the control of the processor (110). The communication unit (130) can include at least one communication module capable of performing wireless communication.

[0155] In an embodiment, the communication unit (130) may transmit a fingerprint to the server (200) via a communication network. In an embodiment, the communication unit (130) may transmit a fingerprint to the server (200) every first transmission cycle or every second transmission cycle.

[0156] In an embodiment, the communication unit (130) may receive identification information obtained by performing ACR based on a fingerprint from the server (200).

[0157] In an embodiment, the memory (120) can store at least one instruction.

[0158] In an embodiment, the memory (120) may store at least one program executed by the processor (110). Additionally, the memory (120) may store data input to or output from the electronic device (100).

[0159] The memory (120) may include at least one type of storage medium among a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (e.g., SD or XD memory, etc.), a RAM (Random Access Memory), a SRAM (Static Random Access Memory), a ROM (Read-Only Memory), an EEPROM (Electrically Erasable Programmable Read-Only Memory), a PROM (Programmable Read-Only Memory), a magnetic memory, a magnetic disk, and an optical disk.

[0160] In an embodiment, the memory (120) may store one or more instructions for obtaining a fingerprint from the output content.

[0161] In an embodiment, the memory (120) may store one or more instructions for transmitting a fingerprint to the server (200) through the communication unit (1300) every first transmission cycle. In an embodiment, the memory (120) may store one or more instructions for receiving identification information obtained through ACR (Automation Content Recognition) performed based on a fingerprint from the server (200) through the communication unit (130).

[0162] In an embodiment, the memory (120) may store one or more instructions for changing the transmission cycle of a fingerprint obtained from content output based on received identification information.

[0163] In an embodiment, the memory (120) may store one or more instructions for predicting / judging / inferring / identifying whether the output content is target content.

[0164] In an embodiment, the memory (120) may store one or more instructions for identifying at least one of a target content start time, a target content section, and a target content end time in the output content.

[0165] In an embodiment, the memory (120) may store one or more instructions for predicting whether output content is target content based on at least one of metadata for the content, a frame included in the content, and a target content pattern.

[0166] In an embodiment, the memory (120) may store one or more instructions for predicting whether output content is target content from metadata of at least one of SCTE-35 packet data and EPG (Electronic Program Guide) data.

[0167] In an embodiment, the memory (120) may store one or more instructions for predicting whether the output content is target content based on at least one of the variation and distribution of at least one of color, brightness, luminance, and contrast of a frame included in the content.

[0168] In an embodiment, the memory (120) may store one or more instructions for predicting whether output content is target content using a neural network that has learned a target content pattern from an image.

[0169] In an embodiment, the memory (120) may store one or more instructions for obtaining an item-specific score indicating a likelihood that the output content is the target content based on at least one of metadata about the content, a frame included in the content, and a target content pattern.

[0170] In an embodiment, the memory (120) may store weights for each item.

[0171] In an embodiment, the memory (120) may store one or more instructions for applying item-specific weights to item-specific scores.

[0172] In an embodiment, the memory (120) may store one or more instructions for obtaining a weighted sum of item-by-item scores with item-by-item weights applied.

[0173] In an embodiment, the memory (120) may store one or more instructions for predicting whether the output content is target content based on a weighted sum of item-by-item scores to which item-by-item weights are applied.

[0174] In an embodiment, the memory (120) may store one or more instructions for extracting a fingerprint from the output content if the output content is predicted to be target content.

[0175] In an embodiment, the memory (120) may store one or more instructions for initializing a fingerprint stack and transmitting a preset number of fingerprints from the initialized fingerprint stack to the server (200) if the output content is predicted to be target content.

[0176] In an embodiment, one or more instructions for setting a query period may be stored in the memory (120).

[0177] In an embodiment, the memory (120) may store one or more instructions for changing the first query period to a second query period when receiving content identification information from the server (200) that the output content is target content.

[0178] In an embodiment, the memory (120) may store one or more instructions for changing the second transmission cycle back to the first transmission cycle based on at least one of: identifying that the output content corresponds to the end point of the target content, or receiving content identification information that the output content does not correspond to the target content.

[0179] A processor (110) according to an embodiment controls the overall operation of an electronic device (100). A processor (101) according to an embodiment may control signal flow between internal components of the electronic device (100) and perform a function of processing data. In an embodiment, the processor (110) may control the electronic device (100) to function by executing one or more instructions stored in a memory (120).

[0180] In an embodiment, the processor (110) may include single core, dual core, triple core, quad core, and multiples thereof.

[0181] In an embodiment, the processor (110) may be one or more. For example, the processor (101) may include multiple processors. In this case, the processor (101) may be implemented as a main processor and a sub processor.

[0182] Additionally, the processor (110) may include at least one of a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and a VPU (Video Processing Unit). Alternatively, in an embodiment, the processor (110) may be implemented in the form of a SoC (System On Chip) that integrates at least one of a CPU, a GPU, and a VPU. Alternatively, in an embodiment, the processor (110) may further include an NPU (Neural Processing Unit).

[0183] In an embodiment, the processor (110) may control at least one component included in the electronic device (100) so that an intended operation is performed by executing at least one instruction stored in the memory (120). Accordingly, even if the processor (110) performs predetermined operations as an example, it may mean that the processor (110) controls at least one component included in the electronic device (100) so that the predetermined operations are performed.

[0184] In an embodiment, a fingerprint obtained from content output from at least one processor (110) may be transmitted to a server (200) through a communication unit (130) at every first transmission cycle.

[0185] In an embodiment, at least one processor (110) may receive identification information obtained through ACR (Automation Content Recognition) performed based on a fingerprint from a server (200).

[0186] In an embodiment, at least one processor (110) may change the first transmission cycle to a second transmission cycle if the content to be output is identified as target content based on the received identification information.

[0187] In an embodiment, at least one processor (110) may change the transmission cycle of a fingerprint obtained from the output content based on the received identification information.

[0188] In an embodiment, at least one processor (110) can predict whether the output content is target content.

[0189] In the present disclosure, at least one processor (110) predicting whether output content is target content may mean that the processor (110) determines, infers, or identifies whether output content is target content.

[0190] In the present disclosure, the processor (110) predicting / inferring / judging / identifying whether the output content is target content is performed before the server (200) identifies what content the output content is through ACR, and can be distinguished from the server (200) obtaining identification information about the output content through ACR.

[0191] In an embodiment, if at least one processor (110) predicts / identifies that the output content is target content, it can initialize a fingerprint stack and transmit a preset number of fingerprints from the fingerprint stack to the server (20) via the communication unit (130).

[0192] In an embodiment, at least one processor (110) can identify at least one of a target content start time, a target content section, and a target content end time in the output content.

[0193] In an embodiment, at least one processor (110) can predict / identify whether the output content is target content based on at least one of metadata for the content, a frame included in the content, and a target content pattern.

[0194] In an embodiment, at least one processor (110) can predict whether the output content is target content from metadata of at least one of SCTE-35 packet data and EPG (Electronic Program Guide) data.

[0195] In an embodiment, at least one processor (110) can predict whether the output content is target content based on at least one of the amount of change and distribution of information of a frame included in the content.

[0196] In an embodiment, the information of the frame may include at least one of color, brightness, luminance, and contrast of the frame.

[0197] In an embodiment, at least one processor (110) can predict whether output content is target content by using a neural network that has learned a target content pattern from an image.

[0198] In an embodiment, at least one processor (110) may obtain a score related to whether the output content corresponds to the target content based on at least one of metadata about the content, a frame included in the content, and a target content pattern. The score related to whether the output content corresponds to the target content may include, for example, a score indicating a likelihood that the output content is the target content.

[0199] In an embodiment, at least one processor (110) can predict whether the output content is target content based on a score related to whether the output content corresponds to the target content.

[0200] For example, at least one processor (110) may obtain a score related to whether the output content corresponds to the target content for each item of metadata for the content, a frame included in the content, and a target content pattern, apply an item-specific weight to the score for each item, and use the sum of the item-specific scores with the item-specific weights applied to predict whether the output content is the target content.

[0201] In an embodiment, at least one processor (110) may extract a new fingerprint from the output content if it predicts that the output content is target content.

[0202] In an embodiment, when at least one processor (110) identifies that the output content is target content, it can initialize a fingerprint stack, extract a preset number of fingerprints from the initialized fingerprint stack, and transmit the extracted fingerprints to the server (200).

[0203] In an embodiment, at least one processor (110) may extract a fingerprint every default cycle.

[0204] In an embodiment, at least one processor (110) may extract a fingerprint from the output content for each first query period and transmit it to the server (200) when the default period is the first query period.

[0205] In an embodiment, when at least one processor (110) receives content identification information from the server (200) indicating that the output content is target content, it may change the first query period to a second query period. The second query period may be determined to be shorter than the first query period, and may be appropriately determined considering the output time of the target content, which is short content. For example, if the target content is output for an average of 10 seconds, the second query period may be determined to be shorter than this, that is, 8 seconds.

[0206] In an embodiment, at least one processor (110) may change the second query period to the first query period based on at least one of identifying that the output content is content at the end point of the target content and receiving content identification information from the server (200) that the output content does not correspond to the target content.

[0207] FIG. 5 illustrates an electronic device (100) and a server (200) performing ACR according to an embodiment.

[0208] Referring to FIG. 5, the electronic device (100) may include an ACR module (501).

[0209] In an embodiment, the ACR module (501) may be included within the processor (110) disclosed in FIG. 4, or may be included within the electronic device (100) separately from the processor (110), and may operate under control from the processor (110).

[0210] In an embodiment, the ACR module (501) may include suitable logic, circuitry, interfaces, and / or code operable to provide automatic content recognition functionality for the electronic device (100).

[0211] In an embodiment, a module may refer to a functional and structural combination of hardware for implementing the technical concepts of the present disclosure and software for operating the hardware. For example, a module may refer to a logical unit of a given code and hardware resources for executing the given code, and is not necessarily limited to physically connected code or a single type of hardware.

[0212] In an embodiment, the ACR module (501) may include a fingerprint extraction unit (511), a target content prediction unit (513), and a query unit (515).

[0213] The fingerprint extraction unit (511) according to the embodiment can extract a fingerprint from the screen currently being output through the screen.

[0214] In an embodiment, the fingerprint extraction unit (511) may perform an operation of extracting a fingerprint by sampling content displayed on the screen of the electronic device (100) at a given rate. For example, the fingerprint extraction unit (511) may sample content displayed on the screen 10 times per second, extract a fingerprint, and stack the extracted fingerprint.

[0215] In an embodiment, the fingerprint extraction unit (511) can extract a fingerprint from a captured image using various methods.

[0216] For example, the fingerprint extraction unit (511) can extract a fingerprint from an image using a multi-level quantization technique. This method extracts features from each frame included in the video, and then applies a multi-level quantization technique to compress the extracted features to obtain a fingerprint. However, this is merely an example and is not limited thereto.

[0217] The target content prediction unit (513) according to the embodiment may be a module that performs an operation of predicting whether the output content is target content.

[0218] In an embodiment, the target content prediction unit (513) can predict whether the output content is target content in various ways.

[0219] In an embodiment, the target content may be, but is not limited to, advertising content, and the target content may include short content with a short content output time, content in a transition section, intro content or cut scenes of a game, closing credits of a movie, etc.

[0220] In an embodiment, the target content prediction unit (513) can identify at least one of the target content start time, the target content section, and the target content end time in the output content in various ways.

[0221] In an embodiment, the target content prediction unit (513) can use metadata to predict whether the output content is target content.

[0222] In an embodiment, the electronic device (100) may obtain metadata indicating information about the media signal, either together with the media signal corresponding to the content, or as a separate signal from the media signal.

[0223] In an embodiment, the electronic device (100) can receive a transport stream in which metadata about the content is multiplexed together with the content. The electronic device (100) can obtain metadata together with video packets and audio packets from the transport stream.

[0224] In an embodiment, the target content prediction unit (513) may parse a SCTE-35 packet when a SCTE-35 packet is detected from metadata included in a transport stream.

[0225] In an embodiment, the payload syntax of the SCTE-35 packet may detect an indicator indicating a signal to switch to an advertisement or an indicator indicating a signal to switch to a program.

[0226] An indicator signal indicating a switch to an advertisement may be called a cue-out event, and an indicator signal indicating a switch to a program may be called a cue-in event.

[0227] In an embodiment, the target content prediction unit (513) can identify a cue-out event or a cue-in event by parsing a SCTE-35 packet.

[0228] In an embodiment, the target content prediction unit (513) can recognize the section from the cue-out point to the cue-in point as an advertisement section using the cue (CUE) data provided by SCTE-35.

[0229] In an embodiment, the target content prediction unit (513) may transmit to the query unit (515) at least one of a cue-out event, a cue-in event, and / or an event indicating a time interval from a cue-out point to a cue-in point.

[0230] In an embodiment, the target content prediction unit (513) may obtain metadata separately from the content media. In an embodiment, the target content prediction unit (513) may receive EPG (Electronic Program Guide) information, which is metadata indicating additional information about a broadcast program, from a broadcasting station server or a content provider server, together with or separately from the broadcast program. EPG information is program guide information and may include information about the broadcast signal, i.e., the time at which the content is output on the broadcast channel.

[0231] In an embodiment, if the content is broadcast content, the target content prediction unit (513) can parse the start time of the broadcast content from EPG information and determine a point in time before the start of the program, a point in time after the start of the program, a point in time of half or a point in time of the total program time, etc., as a section of the advertisement content, which is one of the target contents.

[0232] In an embodiment, the target content prediction unit (513) can analyze a frame to predict whether the content is target content.

[0233] Unlike general content, transitional content inserted between content to create the effect of screen transitions, such as opening or ending credits or intro videos in a movie, often has uniform brightness and color across the entire frame.

[0234] For example, the content of the transition section has a small amount of variation or entropy in the frame, such as when the entire frame is white or black, or when text is included in only some areas of the same solid color frame.

[0235] In an embodiment, the target content prediction unit (513) can process a frame included in the content to obtain a change in at least one of color, brightness, luminance, and contrast of the frame.

[0236] In an embodiment, the target content prediction unit (513) can obtain deviations in color, contrast, brightness, etc. for each pixel or block by comparing each pixel included in a frame with surrounding pixels, or by comparing a block including a predetermined number of pixels with an adjacent block or a block of an adjacent frame.

[0237] Alternatively, in an embodiment, the target content prediction unit (513) may compare the current frame with another frame to obtain the amount of change in color, contrast, brightness, etc. between frames.

[0238] In an embodiment, the target content prediction unit (513) can produce a gradient of color, brightness, or other values ​​between pixels, blocks, or frames in a frame. The gradient is a slope obtained through differentiation and can represent a change in color or brightness for each pixel or block.

[0239] In an embodiment, the target content prediction unit (513) can obtain entropy from the relationships between pixels constituting a frame. A high entropy of a frame indicates a high degree of disorder among the pixels, which may indicate a case where there is little regularity among the pixels included in the frame. Conversely, a low entropy of a frame may indicate a case where there is a high degree of regularity among the pixels.

[0240] In an embodiment, if the target content prediction unit (513) determines that a frame is a single color based on the amount of change in color, brightness, luminance, etc., or entropy, it can recognize that the frame is a frame included in the content of the transition section.

[0241] In an embodiment, the target content prediction unit (513) calculates the gradient or entropy of the frame, and if the gradient or entropy is below a threshold, it can predict that the content is content of the transition section.

[0242] In an embodiment, the target content prediction unit (513) may transmit an event indicating that content of the transition section has been detected to the query unit (515).

[0243] In an embodiment, the target content prediction unit (513) may also use artificial intelligence (AI) technology to predict whether content is target content.

[0244] Artificial intelligence technology consists of machine learning (deep learning) technology that uses algorithms that classify / learn the characteristics of input data on their own, and element technologies that use machine learning algorithms to simulate the cognitive and judgment functions of the human brain.

[0245] AI technology can be implemented using algorithms. Here, an algorithm or set of algorithms for implementing AI technology is called a neural network. A neural network can receive input data, perform operations for analysis and classification, and output result data. For a neural network to accurately output result data corresponding to the input data, it must be trained.

[0246] Here, 'training' may mean training a neural network so that the neural network can discover or learn on its own how to input various data into the neural network, how to analyze the input data, how to classify the input data, and / or how to extract features necessary for generating result data from the input data.

[0247] Training a neural network means applying a learning algorithm to a large number of training data sets to create an artificial intelligence model with desired characteristics. This learning may occur on the server (200) itself, where the artificial intelligence is executed, or through a separate server / system.

[0248] Here, a learning algorithm is a method for training a target device (e.g., a robot) using a large amount of learning data, thereby enabling the target device to make decisions or predictions on its own. Examples of learning algorithms include supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning. Unless otherwise specified, the learning algorithms in the embodiments are not limited to the aforementioned examples.

[0249] A set of algorithms that output output data corresponding to input data through a neural network, software that executes the set of algorithms, and / or hardware that executes the set of algorithms can be called an 'AI model' (or, 'artificial intelligence model' or neural network model, neural network).

[0250] In an embodiment, the target content prediction unit (513) can obtain output data corresponding to input data through an AI model.

[0251] In an embodiment, the neural network utilized by the target content prediction unit (513) may be trained to process input data according to predefined operating rules or AI models to obtain target content patterns from the input data. The predefined operating rules or AI models may be created using specific algorithms. Furthermore, the AI ​​models may be trained using specific algorithms.

[0252] In an embodiment, the target content pattern may include the timing at which the target content is displayed. It is generally assumed that for programs longer than one hour, advertisements are inserted midway through the program, and that advertisements tend to be inserted at the one-third and two-thirds points of the program. The neural network can learn the timing at which advertisements are displayed relative to the program duration, program start time, and program end time.

[0253] In an embodiment, when information about the output content and the output content currently being output on the screen is input to a neural network that has completed learning, the neural network can obtain a result as to whether the output content is content at the time when the advertisement content is output.

[0254] In an embodiment, the target content pattern may include unique characteristics of the target content. For example, a neural network may learn advertising content that includes the title of the program to be played in the upper right corner of the frame. After training, the neural network receives output content as input and analyzes the input content. Based on the presence of the program title in the upper right corner of the frame, the neural network can determine that the frame is advertising content.

[0255] In an embodiment, in the case of advertising content, since the same image is frequently repeated, the neural network may identify the content as advertising content when the same image is repeated.

[0256] In an embodiment, the neural network may be stored on an external computing device separate from the electronic device (100) or server (200).

[0257] In an embodiment, the electronic device (100) can transmit an output image to a computing device in real time. The computing device can receive the output image from the electronic device (100) and input it into a neural network to obtain a result as to whether the content is target content.

[0258] The computing device can transmit the output results obtained from the neural network to the electronic device (100).

[0259] In an embodiment, the target content prediction unit (513) may receive an output result obtained using a neural network from a computing device.

[0260] In an embodiment, when the target content prediction unit (513) receives an output result indicating that the output content is the target content, it can transmit an event indicating that the target content has been detected to the query unit (515).

[0261] However, this is not limited thereto, and in an embodiment, the artificial intelligence model mentioned above may be combined with the electronic device (100).

[0262] The electronic device (100) can use on-device AI technology to predict whether output content is target content. That is, the electronic device (100) can quickly obtain recommendation results by independently collecting and calculating information using a neural network equipped in the electronic device (100) without going through a cloud server. The electronic device (100) can input output content into the neural network and directly obtain a result from the neural network regarding whether the output content is target content.

[0263] In an embodiment, the target content prediction unit (513) can predict whether the output content is target content based on at least one of metadata for the content, a frame included in the content, and a target content pattern.

[0264] In the embodiment, the target content prediction unit (513) uses metadata about the content to predict whether the output content is the target content, and the result is called the first result value.

[0265] In the embodiment, the target content prediction unit (513) analyzes the frame included in the content and predicts whether the output content is the target content, and the result is called the second result value.

[0266] In the embodiment, the result of the target content prediction unit (513) predicting whether the output content is the target content using the target content pattern is referred to as the third result value.

[0267] In an embodiment, the target content prediction unit (513) obtains at least one of the first result value, the second result value, and the third result value, and based on this, can predict whether the output content is the target content.

[0268] In an embodiment, the target content prediction unit (513) can predict that the output content is the target content if at least one of the first result value, the second result value, and the third result value is greater than or equal to the reference value.

[0269] In an embodiment, the target content prediction unit (513) can predict that the output content is the target content if two of the first result value, the second result value, and the third result value are each equal to or greater than a reference value.

[0270] In an embodiment, the target content prediction unit (513) can predict that the output content is the target content if all of the first result value, the second result value, and the third result value are equal to or greater than the reference value.

[0271] In an embodiment, the target content prediction unit (513) may obtain a final result value by using a weighted sum of the first result value, the second result value, and the third result value.

[0272] In an embodiment, the target content prediction unit (513) may obtain item-specific weights. The item-specific weights may be weights assigned to each of the first, second, and third result values. The item-specific weights may have weight values ​​between 0 and 1 and may be a weight matrix containing components assigned to each item.

[0273] For example, the first result value, the second result value, and the third result value may be (r1, r2, r3), and the weights added to each item may be (w1, w2, w3).

[0274] The weight assigned to each item can be determined in a variety of ways.

[0275] For example, if the target content prediction unit (513) obtains a first result value for whether the output content is target content using SCTE-35, the weight w1 given to the first result value may have a high value of 0.9 because the accuracy of SCTE-35 is high.

[0276] On the other hand, if the output content is predicted to be the target content by analyzing the frames included in the content and finding that the frames are monochrome, the weight w2 assigned to the second result value may have a value of 0.5, since a monochrome frame cannot always be determined to be the target content.

[0277] In an embodiment, the target content prediction unit (513) can obtain a weighted item-by-item score (w1r1, w2r2, w3r3) by applying an item-by-item weight to each item. The target content prediction unit (513) can obtain a weighted sum by adding up the results of the item-by-item weights, and can predict whether the output content is the target content based on whether the weighted sum is greater than or equal to a reference value. That is, the target content prediction unit (513) can identify the output content as the target content if the result value of w1r1 + w2r2 + w3r3 is greater than or equal to a certain reference value.

[0278] In an embodiment, if the target content prediction unit (513) predicts that the output content is the target content, it can transmit an event that the output content is the target content to the query unit (515).

[0279] In an embodiment, the target content prediction unit (513) can identify at least one of a target content start time, a target content section, and a target content end time in the output content, and transmit an event notifying this to the query unit (515).

[0280] The query unit (515) according to the embodiment may be a module that performs an operation of requesting an ACR result corresponding to a fingerprint from the server (200).

[0281] In an embodiment, the query unit (515) can transmit a fingerprint to the server (200).

[0282] In an embodiment, the query unit (515) can transmit a fingerprint to the server (200) at each default cycle. If the default cycle is the first query cycle, the query unit (515) can extract a fingerprint stored in the fingerprint extraction unit (511) at each first query cycle and transmit it to the server (200).

[0283] In an embodiment, the query unit (515) may extract a predetermined number of fingerprints most recently stacked among the fingerprints stacked in the fingerprint extraction unit (511) for each first query cycle and transmit them to the server (200).

[0284] In an embodiment, the query unit (515) transmits a fingerprint to the server (200) according to the query cycle, and at the same time, when receiving an event from the target content prediction unit (513) that the output content is the target content, the query unit (515) can transmit a fingerprint to the server (200) accordingly.

[0285] In an embodiment, the query unit (515) may receive an event from the target content prediction unit (513) indicating that the output content is the target content. Receiving an event indicating that the output content is the target content means that the currently output content is new content different from the previously output content, and that the new content is likely to be the target content. Therefore, it is no longer meaningful to use the existing fingerprint.

[0286] In an embodiment, when the query unit (515) receives an event that the output content is the target content, it can notify the fingerprint extraction unit (511) of this, so that the fingerprint extraction unit (511) can initialize the existing fingerprint.

[0287] Alternatively, in an embodiment, the target content prediction unit (513) may transmit an event indicating that content of the transition section has been detected to the fingerprint extraction unit (511) as well as the query unit (515), thereby causing the fingerprint extraction unit (511) to initialize an existing fingerprint.

[0288] In an embodiment, when the query unit (515) receives an event indicating that the output content is the start point of the target content, or an event indicating that the output content is content belonging to a target content section, the query unit (515) may transmit only a certain number of fingerprints from among the new fingerprint stacks initialized and stored in the fingerprint extraction unit (511) to the server (200).

[0289] In an embodiment, when the query unit (515) transmits a new fingerprint to the server (200) based on an event that the output content is the target content, it may transmit identification information indicating that the output content is the target content to the server (200).

[0290] A server (200) according to an embodiment may include suitable logic, circuitry, interfaces, and / or code that may be operable to support automatic content recognition operations in an electronic device (100).

[0291] In an embodiment, the server (200) may provide content automatic recognition applications and / or services to the electronic device (100). The server (200) may provide one or more content automatic recognition technologies and may also provide synchronization with the electronic device (100). The server (200) may support multiple different fingerprinting technologies for content automatic recognition.

[0292] The server (200) may include a search unit (521), a fingerprint DB (523), and a metadata DB (525).

[0293] The server (200) extracts fingerprints from various images, similar to the electronic device (100), and stores them in the fingerprint DB (523). When the server (200) stores the extracted fingerprints in the fingerprint DB (523), it may also store metadata information corresponding to the extracted fingerprints in the metadata DB (525).

[0294] When the search unit (521) receives a fingerprint from the electronic device (100), it can analyze the received fingerprint based on the fingerprint DB (523). The search unit (521) searches the fingerprint DB (533) for a fingerprint that matches the fingerprint received from the electronic device (100), and if there is a matching fingerprint, it can obtain metadata information of the corresponding fingerprint from the metadata DB (525). The metadata information of the fingerprint may include content identification information.

[0295] In an embodiment, when the server (200) receives a fingerprint from the query unit (515), if it also receives identification information indicating that the output content is the target content, the server (200) may search only for fingerprints corresponding to the target content, rather than searching the entire fingerprint DB (523) when searching for a fingerprint matching the received fingerprint. In this case, there is an effect of reducing the resources of the server (200) in that the search range in which the server (200) performs ACR is reduced.

[0296] The server (200) can transmit content identification information obtained by performing ACR using a fingerprint to the electronic device (100).

[0297] In an embodiment, when the query unit (515) receives content identification information from the server (200) that the output content is target content, the query unit (515) may change the first query cycle to a second query cycle that is faster than the first query cycle.

[0298] In an embodiment, when the query period is changed to a second query period, the query unit (515) can extract a fingerprint for each second query period and transmit it to the server (200).

[0299] In an embodiment, when the query unit (515) receives an event indicating that target content output has ended from the target content prediction unit (513), it can transmit a fingerprint to the server (200) accordingly.

[0300] In an embodiment, the target content prediction unit (513) may detect a cue-in event indicating a point at which an advertisement content, which is one of the target contents, ends from a SCTE-35 packet obtained from metadata, or, if it identifies the current point in time as a program transition section between different programs using EPG data, it may transmit an event indicating that the target content has ended to the query unit (515).

[0301] Alternatively, if the target content prediction unit (513) predicts that the output content is not the target content based on at least one of the frames included in the content and the target content pattern, it may transmit an event indicating that the target content has ended to the query unit (515).

[0302] For example, the target content prediction unit (513) can combine the first result value, the second result value, and the third result value together, and if the final result value is not greater than a certain standard value, identify that the output content is not the target content, and transmit an event indicating that the target content has ended to the query unit (515).

[0303] In an embodiment, the ACR module (501) may initialize a new fingerprint stack when it identifies that the target content has ended, since it is no longer meaningful to use the existing fingerprint.

[0304] In an embodiment, the query unit (515) may extract only a certain number of fingerprints from the initialized fingerprint stack and transmit them to the server (200).

[0305] In an embodiment, when the query unit (515) transmits a new fingerprint to the server (200) in response to an event indicating that the target content has ended, it may transmit identification information indicating that the target content has ended along with the fingerprint to the server (200).

[0306] In an embodiment, when the query unit (515) receives an event indicating that the target content has ended, the query period may be changed from the second query period back to the first query period, which is the default query period, and a fingerprint may be transmitted to the server (200) for each first query period.

[0307] In an embodiment, the search unit (521) searches for a fingerprint matching the fingerprint received from the electronic device (100) in the fingerprint DB (523), and if there is a matching fingerprint, the metadata information of the corresponding fingerprint can be obtained from the metadata DB (525). The metadata information of the fingerprint may include content identification information.

[0308] In an embodiment, when the server (200) receives a fingerprint from the query unit (515) and also receives identification information indicating that the target content has ended, when searching for a fingerprint matching the received fingerprint, the server (200) can search for content corresponding to the fingerprint by searching the entire fingerprint DB (523).

[0309] The server (200) can transmit content identification information obtained by performing ACR to the query unit (515).

[0310] Figure 6 is an internal block diagram of an electronic device (100) according to an embodiment.

[0311] The electronic device (100) of FIG. 6 may be an example of the electronic device (100) of FIGS. 3 to 5. Hereinafter, any description overlapping with the description of FIGS. 3 to 5 will be omitted.

[0312] Referring to FIG. 6, the electronic device (100) may further include, in addition to the processor (110), memory (120), and communication unit (130), a tuner unit (610), a communication unit (130), a detection unit (630), an input / output unit (640), a video processing unit (650), a display unit (660), an audio processing unit (670), an audio output unit (680), and a user input unit (690).

[0313] The tuner unit (610) can select and tune only the frequency of the channel to be received by the electronic device (100) from among many radio wave components through amplification, mixing, resonance, etc. of broadcast content received via wire or wireless. The content received through the tuner unit (610) is decoded and separated into audio, video, and / or additional information. The separated audio, video, and / or additional information can be stored in the memory (120) under the control of the processor (110).

[0314] Communication unit (130) Communication unit (130) The communication unit (130) may include at least one of a wireless LAN module (621), a Bluetooth module (622), and a wired Ethernet (623) in accordance with the performance and structure of the electronic device (100).

[0315] The wireless LAN module (621) can transmit and receive Wi-Fi signals with peripheral devices according to the Wi-Fi communication standard.

[0316] The Bluetooth module (622) can receive Bluetooth signals transmitted from peripheral devices according to the Bluetooth communication standard. The Bluetooth module (622) can be a BLE (Bluetooth Low Energy) communication module and can receive BLE signals. The Bluetooth module (622) can continuously or temporarily scan BLE signals to detect whether a BLE signal is received.

[0317] The detection unit (630) detects the user's voice, the user's image, or the user's interaction, and may include a microphone (631), a camera unit (632), a light receiving unit (633), and a sensing unit (634). The microphone (631) may receive an audio signal including the user's uttered voice or noise, and may convert the received audio signal into an electrical signal and output it to the processor (110).

[0318] The camera unit (632) includes a sensor (not shown) and a lens (not shown), and can capture an image formed on the screen and transmit it to the processor (110).

[0319] The optical receiver (633) can receive an optical signal (including a control signal). The optical receiver (633) can receive an optical signal corresponding to a user input (e.g., touch, press, touch gesture, voice, or motion) from a control device such as a remote control or a mobile phone.

[0320] The sensing unit (634) can detect the state around the electronic device and transmit the detected information to the communication unit (130) or processor (110).

[0321] The input / output unit (640) can receive video (e.g., a moving image signal or a still image signal), audio (e.g., a voice signal or a music signal), and additional information from a device external to the electronic device (100) under the control of the processor (110).

[0322] The input / output unit (640) may include one of an HDMI port (High-Definition Multimedia Interface port, 641), a component jack (component jack, 642), a PC port (PC port, 643), and a USB port (USB port, 644). The input / output unit (640) may include a combination of an HDMI port (641), a component jack (642), a PC port (643), and a USB port (644).

[0323] The video processing unit (650) processes image data to be displayed by the display unit (660) and can perform various image processing operations such as decoding, rendering, scaling, noise filtering, frame rate conversion, and resolution conversion for the image data.

[0324] The display unit (660) can display content received from a broadcasting station, an external server, an external storage medium, etc., on the screen. The content is a media signal and may include a video signal, an image, a text signal, etc.

[0325] When the display unit (660) is implemented as a touch screen, the display unit (660) can be used as an input device such as a user interface in addition to an output device. For example, the display unit (660) can include at least one of a liquid crystal display, a thin film transistor-liquid crystal display, an organic light-emitting diode, a flexible display, a 3D display (4D display), and an electrophoretic display. In addition, depending on the implementation form of the display unit (660), two or more display units (660) can be included.

[0326] The audio processing unit (670) performs processing on audio data. The audio processing unit (670) can perform various processing such as decoding, amplification, and noise filtering on audio data.

[0327] The audio output unit (680) can output audio included in content received through the tuner unit (610) under the control of the processor (110), audio input through the communication unit (130) or input / output unit (640), and audio stored in the memory (120). The audio output unit (680) can include at least one of a speaker (681), a headphone (682), or an S / PDIF (Sony / Philips Digital Interface: output terminal) (683).

[0328] The user input unit (690) can receive user input for controlling the electronic device (100). The user input unit (690) can include various types of user input devices, including, but not limited to, a touch panel that detects a user's touch, a button that receives a user's push operation, a wheel that receives a user's rotation operation, a keyboard, a dome switch, a microphone for voice recognition, a motion detection sensor that senses motion, etc. When a remote control or other mobile terminal controls the electronic device (100), the user input unit (690) can receive a control signal received from the mobile terminal.

[0329] FIG. 7 is a diagram explaining the content recognition rate according to the query cycle when performing ACR according to an embodiment.

[0330] Referring to FIG. 7, it is assumed that the content currently output by the electronic device (100) is program A.

[0331] In an embodiment, the electronic device (100) may send a query to the server (200) according to the first query cycle.

[0332] The server (200) can search the database for the fingerprints transmitted at time t1 and time t2, obtain identification information that the content corresponding to the fingerprint is program A, and transmit this to the electronic device (100).

[0333] The electronic device (100) can identify that the identification content displayed on the screen is program A from the identification information received from the server.

[0334] In an embodiment, the electronic device (100) transmits a query according to the first query cycle and can identify that the output content is target content.

[0335] In an embodiment, if the electronic device (100) identifies that the output content is target content at time t3, it can transmit a query to the server (200) at time t3.

[0336] In an embodiment, the electronic device (100) can initialize a fingerprint stack and transmit a preset number of fingerprints from the initialized fingerprint stack to the server (200).

[0337] In an embodiment, the server (200) may search for a fingerprint received from the electronic device (100) at time t3, obtain identification information that the content corresponding to the fingerprint is advertisement B, and transmit the same to the electronic device (100).

[0338] In an embodiment, when the electronic device (100) receives identification information from the server (200) that the output content is advertisement B, which is one of the target contents, the electronic device (100) may change the query period to a second query period.

[0339] In an embodiment, the electronic device (100) may transmit a query to the server (200) at time t4 and time t5, respectively, according to the second query cycle.

[0340] In an embodiment, the server (200) can search for a fingerprint received from the electronic device (100) at time t5, obtain identification information that the content corresponding to the fingerprint is program C, and transmit the same to the electronic device (100).

[0341] In an embodiment, when the electronic device (100) receives identification information from the server (200) that the output content is a general program C and not the target content, the electronic device (100) may change the query cycle back to the first query cycle. Thereafter, the electronic device (100) may transmit a query to the server (200) at time t6 according to the first query cycle.

[0342] In this way, according to the embodiment, the electronic device (100) can increase the recognition rate for target content output for a short period of time by transmitting a fingerprint to the server (200) to perform ACR when the output content is identified as target content.

[0343] In addition, according to an embodiment, the electronic device (100) normally maintains the query cycle as the first query cycle, and changes the query cycle to the second query cycle only when the output content is identified as target content, thereby minimizing the amount of computation or increase in resources that the server (200) must process.

[0344] FIG. 8 is a diagram explaining the content recognition rate according to the query cycle when performing ACR according to an embodiment.

[0345] Referring to FIG. 8, while outputting program A, the electronic device (100) can send a query to the server (200) at times t1 and t2 according to the first query cycle.

[0346] The electronic device (100) receives identification information about the content identified at each point in time as a result of performing ACR for a query transmitted from the server (200) at points in time t1 and t2, and can identify that the content currently displayed on the screen is program A.

[0347] In an embodiment, the electronic device (100) transmits a query according to the first query cycle and can identify that the output content is target content.

[0348] In an embodiment, when the electronic device (100) identifies that the output content is target content at time t3, it can initialize a fingerprint stack and transmit a preset number of fingerprints from the initialized fingerprint stack to the server (200).

[0349] In an embodiment, the server (200) may search for a fingerprint received from the electronic device (100) at time t3, obtain identification information that the content corresponding to the fingerprint is advertisement B, and transmit the same to the electronic device (100).

[0350] In an embodiment, when the electronic device (100) receives identification information from the server (200) that the output content is advertisement B that is the target content, the electronic device (100) may change the query period from the first query period to the second query period.

[0351] In an embodiment, the electronic device (100) may transmit a query to the server (200) at time t4 according to the second query cycle, and receive identification information from the server (200) that the content output at time t4 is advertisement B.

[0352] In an embodiment, the electronic device (100) may identify that the target content has ended at time t5. In an embodiment, when the electronic device (100) identifies that the target content has ended, the electronic device (100) may initialize a fingerprint stack and transmit a preset number of fingerprints from the initialized fingerprint stack to the server (200).

[0353] The server (200) can search for a fingerprint received from the electronic device (100) at time t5, obtain identification information that the content corresponding to the fingerprint is program C, and transmit it to the electronic device (100).

[0354] In an embodiment, the electronic device (100) may change the query period back to the first query period in response to identifying that the target content has ended or receiving identification information from the server (200) that the output content is a general program C and not the target content. The electronic device (100) may transmit a query to the server (200) at time t6 according to the first query period.

[0355] In this way, according to the embodiment, the electronic device (100) can further improve the recognition rate for target content output for a short period of time by transmitting a fingerprint to the server (200) to perform ACR when the output content is identified as the start point of the target content and when the output content is identified as the end point of the target content.

[0356] FIG. 9 is a flowchart illustrating an operation method of an electronic device (100) according to an embodiment.

[0357] In an embodiment, the electronic device (100) may transmit a fingerprint to the server (200) every first transmission cycle (step 910).

[0358] In an embodiment, the server (200) may receive a fingerprint from the electronic device (100) and perform ACR based on the received fingerprint. In an embodiment, the server (200) may search a database for the received fingerprint, identify content corresponding to the fingerprint, and generate identification information indicating the content corresponding to the fingerprint and transmit the generated identification information to the electronic device (100).

[0359] In an embodiment, the electronic device (100) may receive identification information obtained through ACR performed based on a fingerprint from the server (200) (step 920).

[0360] In an embodiment, the electronic device (100) can change the transmission cycle of the fingerprint based on the identification information (step 930).

[0361] In an embodiment, if the electronic device (100) identifies that the output content is target content from the identification information, it can change the first transmission cycle to the second transmission cycle.

[0362] In an embodiment, if the electronic device (100) identifies from the identification information that the output content is not the target content, the first transmission cycle can be maintained as is.

[0363] FIG. 10 is a flowchart illustrating a method for predicting whether output content is target content, according to an embodiment.

[0364] In an embodiment, the electronic device (100) can predict whether the output content is target content.

[0365] In an embodiment, the electronic device (100) can obtain a first result value of whether the output content is target content by using metadata (step 1010).

[0366] In an embodiment, the electronic device (100) may obtain metadata for content and obtain at least one of SCTE-35 packet data and EPG (Electronic Program Guide) data from the metadata.

[0367] In an embodiment, the electronic device (100) can obtain at least one of the point in time when target content is output, the point in time when target content is ended, and the section in which target content is output, by using at least one of SCTE-35 packet data or EPG data.

[0368] In an embodiment, the electronic device (100) can analyze the frame to obtain a second result value as to whether the output content is target content (step 1020).

[0369] In an embodiment, the electronic device (100) may image-process a frame included in the content to obtain at least one of a change amount and distribution of at least one of color, brightness, luminance, and contrast of the frame.

[0370] In an embodiment, the electronic device (100) can predict whether the output content is target content based on the image processing result for the frame.

[0371] In an embodiment, the electronic device (100) can obtain a third result value as to whether the output content is the target content by using the target content pattern (step 1030).

[0372] In an embodiment, the target content pattern may include at least one of a pattern for when the target content is output, or a characteristic of the target content frame.

[0373] In an embodiment, the electronic device (100) can predict whether the output content is the target content by using a neural network that has learned the target content pattern from the image.

[0374] Fig. 11 is a flowchart illustrating an operating method according to an embodiment.

[0375] In an embodiment, the electronic device (100) can identify whether the current point in time is a point in time according to a query cycle (step 1110).

[0376] In an embodiment, if the electronic device (100) identifies that the current point in time is a point in time according to the query cycle, it can transmit a query to the server (200) (step 1130).

[0377] In an embodiment, the electronic device (100) can predict whether the output content is target content (step 1120).

[0378] In an embodiment, if the electronic device (100) identifies that the output content is target content, it may transmit a query to the server (200) (step 1130).

[0379] In an embodiment, the server (200) can perform ACR with a fingerprint (step 1140).

[0380] In an embodiment, the server (200) can search for a fingerprint matching a fingerprint received from an electronic device (10) in a fingerprint DB.

[0381] In an embodiment, the server (200) can obtain content identification information by searching for metadata corresponding to a fingerprint searched in the fingerprint DB in the metadata DB (step 1150).

[0382] In an embodiment, the server (200) may transmit content identification information to the electronic device (100).

[0383] In an embodiment, the electronic device (100) can receive content identification information from the server (200) and predict whether the output content is target content (step 1160).

[0384] In an embodiment, when the electronic device (100) receives content identification information indicating that the output content is target content, the electronic device may change the query period (step 1170).

[0385] For example, if the current query cycle is the first query cycle, the electronic device (100) may change the first query cycle to the second query cycle in response to receiving identification information that the currently output content is the target content.

[0386] In an embodiment, the electronic device (100) can identify whether the current point in time is a point in time according to the second query cycle (step 1110), and if the current point in time is a point in time according to the second query cycle, can transmit a query to the server (200) (step 1130).

[0387] The method of operating the electronic device (100) according to some embodiments and the electronic device (100) may also be implemented in the form of a recording medium including computer-executable instructions, such as program modules executed by a computer. Computer-readable media may be any available media that can be accessed by a computer, and includes both volatile and nonvolatile media, removable and non-removable media. Furthermore, computer-readable media may include both computer storage media and communication media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information, such as computer-readable instructions, data structures, program modules, or other data. Communication media typically includes computer-readable instructions, data structures, program modules, or other data in a modulated data signal, such as a carrier wave, or other transport mechanism, and includes any information delivery media.

[0388] In addition, the electronic device and the operating method thereof according to the embodiment of the present disclosure described above may be implemented as a computer program product including a computer-readable recording medium / storage medium having recorded thereon a program for implementing an operating method of the electronic device, the method including a step of transmitting a fingerprint obtained from output content to a server at every first transmission cycle, a step of receiving identification information obtained through Automation Content Recognition (ACR) performed based on the fingerprint from the server, and a step of changing a transmission cycle of the fingerprint obtained from the output content based on the received identification information.

[0389] A device-readable storage medium may be provided in the form of a non-transitory storage medium. Here, the term "non-transitory storage medium" simply means a tangible device that does not contain signals (e.g., electromagnetic waves). This term does not distinguish between cases where data is permanently stored in the storage medium and cases where data is temporarily stored. For example, a "non-transitory storage medium" may include a buffer in which data is temporarily stored.

[0390] According to one embodiment, the method according to various embodiments disclosed in the present document may be provided as included in a computer program product. The computer program product may be traded as a product between a seller and a buyer. The computer program product may be distributed in the form of a machine-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or may be distributed online (e.g., downloaded or uploaded) through an application store or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product (e.g., a downloadable app) may be temporarily stored or temporarily generated in a machine-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or an intermediary server.

Claims

1. In an electronic device (100), Communication Department (130); A memory (120) storing at least one instruction; and At least one processor (110) for executing at least one instruction stored in the memory, At least one processor above A fingerprint obtained from the output content is transmitted to the server (200) through the communication unit at every first transmission cycle, Receive identification information obtained through ACR (Automation Content Recognition) performed based on the above fingerprint from the server through the communication unit, An electronic device that changes the transmission cycle of a fingerprint obtained from the output content based on the received identification information.

2. In the first paragraph, at least one processor, An electronic device that changes the first transmission cycle to a second transmission cycle when the output content is identified as target content based on the received identification information.

3. In the second paragraph, at least one processor When the output content is identified as target content, the fingerprint stack is initialized, An electronic device that transmits a preset number of fingerprints from the fingerprint stack to the server through the communication unit.

4. In the third paragraph, the electronic device, wherein the at least one processor obtains at least one of a target content start time, a target content section, and a target content end time from the output content.

5. In the fourth paragraph, the electronic device changes the second transmission cycle to the first transmission cycle based on at least one of identifying that the output content corresponds to the end point of the target content and receiving content identification information that the output content does not correspond to the target content.

6. In any one of the third to fifth paragraphs, the at least one processor An electronic device that identifies whether the outputted content is the target content based on at least one of metadata about the content, a frame included in the content, and a target content pattern.

7. In the 6th paragraph, the at least one processor Obtain a score related to whether the output content corresponds to the target content based on at least one of metadata for the content, a frame included in the content, and the target content pattern; An electronic device that identifies whether the output content is the target content based on the obtained score.

8. An electronic device according to claim 6 or 7, wherein the metadata for the content includes at least one of SCTE-35 packet data and EPG (Electronic Program Guide) data.

9. In any one of paragraphs 6 to 8, the at least one processor Identifying whether the output content is the target content based on at least one of the amount of change and distribution of information of the frame included in the content, An electronic device, wherein the information of the above frame includes at least one of color, brightness, luminance, and contrast of the frame.

10. In any one of paragraphs 6 to 9, the at least one processor An electronic device that uses a neural network that learns content patterns from images to identify whether the output content is the target content.

11. In the method of operating an electronic device, A step of transmitting a fingerprint obtained from the output content to the server at each first transmission cycle; A step of receiving identification information obtained through ACR (Automation Content Recognition) performed based on the above fingerprint from the server; and An operating method of an electronic device, comprising a step of changing a transmission cycle of a fingerprint obtained from the output content based on the received identification information.

12. A method of operating an electronic device, further comprising a step of changing the first transmission cycle to a second transmission cycle when the output content is identified as target content based on the received identification information in the 11th paragraph.

13. In the 12th paragraph, if the output content is identified as target content, a step of initializing a fingerprint stack; and A method of operating an electronic device, further comprising the step of transmitting a preset number of fingerprints from the fingerprint stack to the server.

14. A method of operating an electronic device, further comprising a step of obtaining at least one of a target content start time, a target content section, and a target content end time from the output content in the 13th paragraph.

15. A computer-readable recording medium having recorded thereon a program for performing the method of any one of claims 11 to 14 on a computer.

Images