Prediction of future viewing of video segments to optimize system resource utilization

Abstract

Devices, computer program products, and methods are provided for improved management of system resources in matching systems. For example, examples can improve the efficiency of system resource utilization by managing the duration for which data associated with a video segment is retained based on data that takes into account the popularity of the identified video segment. The identified popularity can be determined by algorithms that consider the number of viewers watching the video segment, the video segment's rating, metrics derived from a remote source, or any other factors that can indicate the likelihood that the video segment will be viewed.

Description

[0001] This application is a divisional application of the invention patent application filed on July 15, 2016, with international application number PCT / US2016 / 042611, national application number 201680052207.0, entitled "Prediction of future viewing of video segments to optimize system resource utilization".

[0002] Cross-references to related applications This application claims the benefit of U.S. Provisional Application No. 62 / 193,331, filed July 16, 2015, the entire contents of which are incorporated herein by reference. This application is also incorporated in whole by reference to: U.S. Patent Application Serial No. US14 / 089,003, filed November 25, 2013; U.S. Provisional Application No. 61 / 182,334, filed May 29, 2009; U.S. Provisional Application No. 61 / 290,714, filed December 29, 2009; U.S. Application No. 12 / 787,748, filed May 27, 2015; and U.S. Application No. 12 / 788,721, filed May 27, 2015. Technical Field

[0003] This disclosure relates to improving the management of system resources for identifying content displayed by media systems (e.g., television systems, computer systems, or other electronic devices capable of connecting to the Internet). In some examples, various technologies and systems are provided for determining the likelihood that content will be displayed on a media system in the future. Background Technology

[0004] Media systems (such as television systems, computer systems, or other electronic devices capable of connecting to the internet) can display content for users viewing the media system. In some examples, the content displayed on the media system may be unknown to the media system and any applications running on it. For example, the media system could display anything it provides. In such examples, the identification of the content may not be sent to the media system. In other examples, the identification of the content may be lost in the time interval between when the content is sent by the remote system and when the content is received by the media system. Because it is unknown what content is being displayed on the media system, the opportunity to display additional content similar to or related to that content becomes difficult.

[0005] One solution for identifying content displayed on a media system is to use a matching system that matches the content displayed on the media system with reference content. However, in order to match content, the matching system must contain a large amount of reference content. Furthermore, the matching system must not only have a large amount of reference content, but it must also have correct reference content. Therefore, there is a need in the art to manage the reference content used by the matching system. Summary of the Invention

[0006] Devices, computer program products, and methods are provided for improved management of system resources in matching systems. For example, an example could demonstrate improved efficiency in system resource utilization by managing the duration for which data associated with a video segment is retained, based on data that takes into account the popularity of the identified video segment. The popularity of the identified segment can be determined by an algorithm that takes into account the number of viewers who have watched the video segment, the video segment's rating, metrics derived from social media, or any other factors that may indicate the likelihood that the video segment will be watched.

[0007] In some implementations, apparatus, computer program products, and methods are provided for improving the management of system resources in a matching system. For example, one method may include receiving a reference dataset associated with a media segment and storing the reference dataset in a reference database. In some examples, the reference dataset may be received by a server. The server may be configured to identify unidentified media segments by matching unidentified datasets with the reference dataset. In this example, the unidentified dataset may be associated with an unidentified media segment. In some examples, the reference dataset may include pixel data or audio data of frames from the media segment.

[0008] The method may further include receiving popularity data indicating the popularity of a media segment. In some examples, the popularity data includes at least one or more of the following: view information of the media segment, rating information of the media segment, or information associated with a published message on a remote source. In this example, the published message may be associated with the media segment. In some examples, the information associated with a published message on a remote source includes at least one or more of the following: the number of published messages on the remote source, the number of positive messages published, the number of negative messages published, the number of positive indications for the published message, the number of republished messages, or the number of views of the published message. In this example, the published message is associated with the media segment. In some examples, view information may include the number of times the media segment has been identified by the server.

[0009] The method may further include determining the deletion time of the reference dataset. In some examples, the deletion time can be determined using popularity data. The method may further include deleting the reference dataset from the reference database. In some examples, the reference dataset may be deleted after the deletion time has expired. In some examples, determining the deletion time may include calculating the average of two or more values ​​of the popularity data. In this example, each value of the popularity data may be associated with a weight.

[0010] In some implementations, the method may further include adding the reference dataset to the backup database when the reference dataset is deleted from the reference database. In this implementation, the server may be configured to search the backup database after the reference database for a dataset that matches the unidentified dataset.

[0011] The features, aspects and advantages of this disclosure will be most readily understood when the following description is read with reference to the accompanying drawings, in which the same numerical designations denote the same components or parts throughout the drawings. Attached Figure Description

[0012] The following is a detailed description of illustrative examples with reference to the accompanying drawings: Figure 1 This is a block diagram of an example of a matching system used to identify video content being viewed through a media system.

[0013] Figure 2 An example of a matching system for identifying unknown data points is shown.

[0014] Figure 3 This is a block diagram of an example video capture system.

[0015] Figure 4 This is a block diagram of an example system for collecting video content presented by a display.

[0016] Figure 5 An example of a matching system for optimizing resource utilization is shown.

[0017] Figure 6 This is a diagram illustrating an example of the relative weights used to determine whether to retain data from the reference dataset.

[0018] Figure 7 This is a flowchart illustrating an example of a process for optimizing resource utilization in a matching system.

[0019] Figure 8 It is a graph showing the location of points and the path points around them.

[0020] Figure 9 It is a graph that shows the set of points located at a certain distance from the query point.

[0021] Figure 10 It is a graph showing the possible point values.

[0022] Figure 11 It is a diagram showing a space divided into rings with exponentially increasing widths.

[0023] Figure 12 It is a graph showing the self-intersecting paths and query points.

[0024] Figure 13 It is a graph showing the locations of three consecutive points and the path points around them. Detailed Implementation

[0025] In the following description, specific details are set forth for purposes of explanation in order to provide a thorough understanding of examples of the invention. However, it will be apparent that various examples may be practiced without these specific details. The accompanying drawings and description are not intended to be limiting.

[0026] The following description provides exemplary examples only and is not intended to limit the scope, applicability, or configuration of this disclosure. Rather, the subsequent description of the exemplary examples will provide those skilled in the art with a description of the possibilities for implementing the exemplary examples. It should be understood that various changes can be made to the function and arrangement of the elements without departing from the spirit and scope of the invention as set forth in the appended claims.

[0027] Specific details are given in the following description to provide a thorough understanding of the examples. However, those skilled in the art will understand that these examples can be implemented without these specific details. For example, circuits, systems, networks, processes, and other components may be illustrated as block diagrams to avoid obscuring the examples with unnecessary detail. In other instances, to avoid obscuring these examples, well-known circuits, processes, algorithms, structures, and techniques may be shown without unnecessary detail.

[0028] Additionally, it should be noted that the examples can be described as processes, depicted as flowcharts, diagrams, data flow diagrams, structure diagrams, or block diagrams. Although a flowchart can describe operations as a sequential process, many operations can be performed in parallel or simultaneously. Furthermore, the order of operations can be rearranged. A process terminates upon completion of its operations, but may have additional steps not included in the accompanying drawings. A process can correspond to a method, function, program, subroutine, subroutines, etc. When a process corresponds to a function, its termination can correspond to returning the function to the calling function or the main function.

[0029] The terms "machine-readable storage medium" or "computer-readable storage medium" include, but are not limited to, portable or non-portable storage devices, optical storage devices, and various other media capable of storing, containing, or carrying instructions and / or data. Machine-readable or computer-readable storage media may include non-transitory media in which data can be stored and exclude transient electronic signals propagated via carrier waves and / or wirelessly or via wired connections. Examples of non-transitory media may include, but are not limited to, magnetic disks or magnetic tapes, optical storage media such as optical discs (CDs) or digital versatile discs (DVDs), flash memory, memory, or memory devices. Computer program products may include code and / or machine-executable instructions representing any combination of procedures, functions, subroutines, programs, routines, subroutines, modules, software packages, classes, or instructions, data structures, or program statements. Code segments can be coupled to another code segment or hardware circuitry by passing and / or receiving information, data, arguments, parameters, or memory contents. Information, arguments, parameters, data, or other information can be passed, forwarded, or transmitted using any suitable means, including memory sharing, messaging, token passing, network transmission, or other transmission technologies.

[0030] Furthermore, the example can be implemented using hardware, software, firmware, middleware, microcode, hardware description languages, or any combination thereof. When implemented as software, firmware, middleware, or microcode, the program code or code segments (e.g., a computer program product) that perform the necessary tasks can be stored on a machine-readable medium. The processor can then perform the necessary tasks.

[0031] The systems depicted in some of the accompanying figures can be provided in various configurations. In some examples, the system can be configured as a distributed system, where one or more components of the system span one or more network distributions within a cloud computing system.

[0032] As described in further detail below, certain aspects and features of this disclosure relate to identifying unknown video segments by comparing unknown data points with one or more reference data points. The systems and methods described herein improve the efficiency of storing and searching large datasets for identifying unknown video segments. For example, the systems and methods allow for the identification of unknown data segments while reducing the density of the large datasets required to perform the identification. This technique can be applied to any system that harvests and manipulates large amounts of data. Illustrative examples of such systems include automated content-based search systems (e.g., automatic content recognition for video-related applications or other suitable applications), MapReduce systems, Bigtable systems, pattern recognition systems, face recognition systems, classification systems, computer vision systems, data compression systems, cluster analysis, or any other suitable systems. Those skilled in the art will recognize that the techniques described herein can be applied to any other system that stores data compared with unknown data. For example, in the case of automatic content recognition (ACR), the systems and methods reduce the amount of data that must be stored to enable a matching system to search for and find relationships between unknown data groups and known data groups.

[0033] For illustrative purposes only and not as a limitation, some examples described herein use automatic audio and / or video content recognition systems. However, those skilled in the art will recognize that other systems can use the same techniques.

[0034] A major challenge for ACR systems and other systems that utilize large amounts of data is managing the sheer volume of data required for system operation. Another challenge includes the need to build and maintain a database of known content as a reference for matching incoming content. Building and maintaining such a database involves collecting and extracting large amounts (e.g., hundreds, thousands, or more) of content (e.g., nationally distributed television programs and an even larger volume of local television broadcasts from many other potential content sources). Extraction can be performed using any available technique that reduces raw data (e.g., video or audio) to compressed, searchable data. With a 24 / 7 operating schedule and a sliding window of roughly two weeks' worth of content (e.g., television programs), the amount of data required to perform ACR can accumulate rapidly. Other systems that harvest and manipulate large amounts of data (such as the example systems described above) may face similar challenges.

[0035] The systems and methods described herein can allow for improved management of system resources in ACR systems. For example, examples can improve the efficiency of system resource utilization by managing the duration for which data associated with a video segment is retained, based on data that takes into account the popularity of the identified video segment. The identified popularity can be determined by algorithms that consider the accounts of viewers who watch the video segment, the video segment's rating, metrics derived from social media, or any other factors that can indicate the likelihood that the video segment will be viewed. Although the description herein may refer to video segments, other media segments, including audio segments, can also be used.

[0036] Figure 1 A matching system 100 capable of identifying unknown content is illustrated. In some examples, the unknown content may include one or more unknown data points. In such an example, the matching system 100 may match the unknown data points with reference data points to identify unknown video segments associated with the unknown data points. The reference data points may be included in a reference database 116.

[0037] Matching system 100 includes client device 102 and matching server 104. Client device 102 includes media client 106, input device 108, output device 110, and one or more background applications 126. Media client 106 (which may include a television system, computer system, or other electronic device capable of connecting to the Internet) can decode data associated with video program 128 (e.g., broadcast signals, data packets, or other frame data). Media client 106 can place the decoded content of each frame of the video into a video frame buffer in preparation for display or further processing of the pixel information of the video frame. Client device 102 can be any electronic decoding system capable of receiving and decoding video signals. Client device 102 can receive video program 128 and store video information in a video buffer (not shown). Client device 102 can process the video buffer information and generate unknown data points (which may be referred to as "clues"), as shown in the following reference. Figure 3 In more detail, media client 106 can send unknown data points to matching server 104 for comparison with reference data points in reference database 116.

[0038] Input device 108 may include any suitable device that allows requests or other information to be input to media client 106. For example, input device 108 may include a keyboard, mouse, voice recognition input device, wireless interface for receiving wireless input from a wireless device (e.g., a remote control, mobile device, or other suitable wireless device), or any other suitable input device. Output device 110 may include any suitable device capable of presenting or otherwise outputting information, such as a display, wireless interface for sending wireless output to a wireless device (e.g., a mobile device or other suitable wireless device), printer, or other suitable output device.

[0039] Matching system 100 can begin the process of identifying video segments by first collecting data samples from known video data sources 118. For example, matching server 104 collects data to establish and maintain a reference database 116 from various video data sources 118. Video data sources 118 may include media providers of television programs, movies, or any other suitable video sources. Video data from video data sources 118 may be provided as a radio broadcast, a cable television channel, a streaming source from the Internet, and from any other video data source. In some examples, as described below, matching server 104 may process video received from video data sources 118 to generate and collect reference video data points in reference database 116. In some examples, video programs from video data sources 118 may be processed by a reference video program extraction system (not shown), which may generate reference video data points and send them to reference database 116 for storage. The reference data points can be used as described above to determine information subsequently used for analyzing unknown data points.

[0040] Matching server 104 can store reference video data points for each video program received within a certain time period (e.g., several days, several weeks, several months, or any other suitable time period) in reference database 116. Matching server 104 can build and continuously or periodically update reference database 116 containing television program samples (e.g., including reference data points that may also be referred to as cues or cue values). In some examples, the collected data is a compressed representation of video information sampled from periodic video frames (e.g., every five video frames, every ten video frames, every fifteen video frames, or other suitable number of frames). In some examples, the number of data bytes per frame collected for each program source is (e.g., 25 bytes per frame, 50 bytes per frame, 75 bytes per frame, 100 bytes per frame, or any other number of bytes per frame). Any number of program sources can be used to obtain video, such as 25 channels, 50 channels, 75 channels, 100 channels, 200 channels, or any other number of program sources. Using the sample amount of data, the total data collected over a three-day, 24-hour period becomes very large. Therefore, reducing the number of actual reference data points helps reduce the storage load on the matching server 104.

[0041] Media client 106 may send communication 122 to matching engine 112 of matching server 104. Communication 122 may include a request to matching engine 112 to identify unknown content. For example, unknown content may include one or more unknown data points, and reference database 116 may include multiple reference data points. Matching engine 112 may identify unknown content by matching unknown data points against reference data in reference database 116. In some examples, unknown content may include unknown video data presented by a display (for video-based ACR), search queries (for MapReduce systems, Bigtable systems, or other data storage systems), unknown facial images (for facial recognition), unknown pattern images (for pattern recognition), or any other unknown data that can be matched against a database of reference data. Reference data points may be derived from data received from video data source 118. For example, data points may be extracted from information provided by video data source 118 and may be indexed and stored in reference database 116.

[0042] Matching engine 112 may send a request to candidate determination engine 114 to determine candidate data points from reference database 116. Candidate data points may be reference data points located at a predetermined distance from unknown data points. In some examples, the distance between reference data points and unknown data points may be determined by comparing one or more pixels of the reference data point (e.g., a single pixel, a value representing a set of pixels (e.g., mean, average, median, or other values), or other suitable number of pixels) with one or more pixels of the unknown data point. In some examples, a predetermined distance may be maintained between the reference data point and the unknown data point when the pixel at each sample location falls within a specific range of pixel values.

[0043] In an illustrative example, the pixel value of a pixel may include a red value, a green value, and a blue value (in the red-green-blue (RGB) color space). In this example, a first pixel (or the value representing a first group of pixels) can be compared with a second pixel (or the value representing a second group of pixels) by comparing the corresponding red, green, and blue values ​​separately, ensuring that the value is within a certain range (e.g., between 0 and 5). For example, a first pixel can be matched with a second pixel when (1) the red value of the first pixel is within 5 values ​​(positive or negative) of the red value of the second pixel in the range of 0-255, (2) the green value of the first pixel is within 5 values ​​(positive or negative) of the green value of the second pixel, and (3) the blue value of the first pixel is within 5 values ​​(positive or negative) of the blue value of the second pixel. In this example, candidate data points are reference data points that approximately match unknown data points, resulting in the identification of multiple candidate data points (related to different media segments) for unknown data points. Candidate determination engine 114 can return candidate data points to matching engine 112.

[0044] For candidate data points, matching engine 112 can add tokens to a bin associated with the candidate data point and assigned to an identified video segment from which the candidate data point was derived. The corresponding tokens can be added to all bins corresponding to the identified candidate data points. When matching server 104 receives more unknown data points (corresponding to unknown content being viewed) from client device 102, a similar candidate data point determination process can be performed, and tokens can be added to the bins corresponding to the identified candidate data points. Only one of these bins corresponds to the unknown video content segment being viewed; the others correspond to candidate data points that match due to similar data point values ​​(e.g., having similar pixel color values) but do not correspond to the actual video content segment being viewed. The bin for the unknown video content segment being viewed will have more tokens assigned to it than other bins for video content segments that are not being viewed. For example, as more unknown data points are received, a larger number of reference data points corresponding to that bin are identified as candidate data points, resulting in more tokens being added to that bin. Once a bin contains a certain number of tokens, matching engine 112 can determine that the video segment associated with that bin is currently being displayed on client device 102. A video segment can include an entire video program or a portion of a video program. For example, a video segment can be a video program, a scene from a video program, one or more frames from a video program, or any other part of a video program.

[0045] Figure 2 Components of a matching system 200 for identifying unknown data are shown. For example, a matching engine 212 may use a database of known content (e.g., known media segments, information stored in a database for comparison searches, known faces or patterns, etc.) to perform matching processing for identifying unknown content (e.g., unknown media segments, search queries, images of faces or patterns, etc.). For example, matching engine 212 receives unknown data content 202 (which may be referred to as a "clue") to be matched against reference data points 204 in a reference database. Unknown data content 202 may also be received by or sent from matching engine 212 to candidate determination engine 214. Candidate determination engine 214 may perform search processing to identify candidate data points 206 by searching the reference data points 204 in the reference database. In one example, the search processing may include a nearest neighbor search process that generates a set of neighboring values ​​(at a certain distance from the unknown value of unknown data content 202). Candidate data point 206 is input to matching engine 212 for matching processing to generate a matching result 208. Depending on the application, the matching result 208 may include video data presented by the display, search results, faces identified using facial recognition, patterns identified using pattern recognition, or any other results.

[0046] When identifying candidate data points 206 for an unknown data point (e.g., unknown data content 202), the candidate identification engine 214 determines the distance between the unknown data point and reference data points 204 in the reference database. Reference data points located at a certain distance from the unknown data point are identified as candidate data points 206. In some examples, the distance between the reference data point and the unknown data point can be determined by comparing one or more pixels of the reference data point with one or more pixels of the unknown data point, as described above. Figure 1 As described. In some examples, reference data points can be spaced a certain distance from unknown data points when the pixels at each sample location are within a specific value range. As mentioned above, candidate data points are reference data points that approximately match the unknown data points, and due to the approximately matching, multiple candidate data points (related to different media segments) are identified for the unknown data point. The candidate determination engine 114 can return the candidate data points to the matching engine 112.

[0047] Figure 3 An example of a video extraction and capture system 400 including a storage buffer 302 with a decoder is shown. The decoder may be part of a matching server 104 or a media client 106. The decoder may not operate with or require a physical television display panel or device. The decoder can decode and, when needed, decrypt digital video programs into an uncompressed bitmap representation of the television program. To establish a reference database of reference video data (e.g., reference database 316), the matching server 104 may acquire one or more arrays of video pixels read from the video frame buffer. The array of video pixels is referred to as a video block. A video block can be of any shape or pattern, but for the purposes of this specific example, it is described as a 10×10 pixel array comprising ten pixels horizontally and ten pixels vertically. Also for the purposes of this example, it is assumed that there are 25 pixel block locations extracted from within the video frame buffer, uniformly distributed within the boundaries of the buffer.

[0048] Example allocation of pixel blocks (e.g., pixel block 304) in Figure 3 As shown in the diagram. As described above, a pixel block may include a pixel array, such as a 10×10 array. For example, pixel block 304 includes a 10×10 pixel array. A pixel may include color values, such as red, green, and blue values. For example, pixel 306 with red-green-blue (RGB) color values ​​is shown. The color values ​​of a pixel may be represented by an eight-bit binary value for each color. Other suitable color values ​​that can be used to represent the color of a pixel include luminance and chromaticity (Y, Cb, Cr) values ​​or any other suitable color values.

[0049] The average (or average in some cases, the average value) of each pixel block is taken, and the resulting data record is created and tagged with a timecode (or timestamp). For example, to find the average value for a 10×10 pixel block array, in this case, for a total of 600 bits of pixel information per frame, each of the 25 display buffer positions produces 24 bits of data. In one example, the average value of pixel block 304 is calculated and shown as pixel block average value 308. In an illustrative example, the timecode may include an “epoch time”, which represents the total elapsed time (in fractions of a second) since midnight on January 1, 1970. For example, the value of pixel block average value 308 is combined with timecode 412. Ephemeral time is a recognized convention in computing systems, including, for example, Unix-based systems. Information about the video program (called metadata) is attached to the data record. Metadata may include any information about the program, such as the program identifier, program time, program length, or any other information. The data record, including the average value of the pixel block, the timecode, and the metadata, forms a “data point” (also known as a “thread”). Data point 310 is an example of a reference video data point.

[0050] The process of identifying unknown video segments begins with steps similar to creating a reference database. For example, Figure 4 A video extraction and capture system 400 is shown, including a memory buffer 402 with a decoder. The video extraction and capture system 400 may be part of a client device 102 that processes data presented by a display (e.g., on an internet-connected television monitor such as a smart TV, mobile device, or other television viewing device). The video extraction and capture system 400 may utilize a similar process to generate unknown video data points 410, as used by the system 300 that creates reference video data points 310. In one example, a media client 106 may send the unknown video data point 410 to a matching engine 112 for a matching server 104 to identify the video segment associated with the unknown video data point 410.

[0051] like Figure 4As shown, video block 404 may include a 10×10 pixel array. Video block 404 can be extracted from video frames displayed on the screen. Multiple such pixel blocks can be extracted from a video frame. In an illustrative example, if 25 such pixel blocks are extracted from a video frame, the result will be points representing locations in 75-dimensional space. An average (or mean value) can be calculated for each color value of the array (e.g., RGB color values, Y, Cr, Cb color values, etc.). Data records (e.g., unknown video data point 410) are formed from the average pixel values, and the current time is appended to the data. One or more unknown video data points can be sent to matching server 104 using the techniques described above to match data from reference database 116.

[0052] Figure 5 An example of a matching system 500 for optimizing resource utilization is shown. The matching system 500 can be similar to... Figure 1 Matching system 100 is described in the text. For example, matching system 500 may include client device 510 (which may be the same as or similar to client device 102). Client device 510 may receive frames of video segments from a remote source. In some examples, the frames may be in the form of broadcast signals. In other examples, the frames may be in the form of data (e.g., frame data) sent to client device 510 in packets using a network (e.g., the Internet). In some examples, the frames may include pixel values ​​used by client device 510 to present the frame to a user on a display.

[0053] In some examples, pixel values ​​can be sent directly from client device 510 to matching server 520 (which may operate similarly to matching server 104). In other examples, using the techniques described above, a subset of pixel values ​​can be sent to matching server 520 (e.g., pixel clues can be sent to matching server 520). In still other examples, pixel values ​​(or a subset thereof) can be summarized by client device 510, and the summarized pixel values ​​can then be sent to matching server 520.

[0054] Matching server 520 may include matching engine 530 (which may perform in the same or similar manner as matching engine 112) to identify unidentified video segments displayed on client device 510. To identify unidentified video segments, matching engine 530 may receive pixel values ​​(or pixel cue points, average pixel values, or other representations of frames) associated with one or more frames from client device 510. Matching engine 530 may also receive or obtain reference data to match the pixel values. In some examples, reference data may be received from reference database 550. In such examples, reference database 550 may send one or more reference datasets, which may be stored in reference database 550, to matching engine 530. In other examples, matching engine 530 may use a database query to obtain one or more reference datasets from reference database 550. In some examples, matching engine 530 may send a request to candidate determination engine 540 to determine candidate data points from reference database 550. As described above, candidate data points may be reference data points located at a defined distance from one or more data points in a frame. Candidate determination engine 540 can return candidate data points to matching engine 530.

[0055] As described above, for each of the candidate data points, the matching engine 530 can add a token to the bin corresponding to the identified video segment. The identified video segment can be associated with one or more candidate data points, which include the candidate data points that cause the token to be added to the bin. This process can be repeated for each new frame sent from the client device 510 to the matching engine 530. Once the bin contains a certain number of tokens, the matching engine 530 can determine that the video segment associated with the bin is currently being displayed by the client device 510.

[0056] Once the matching engine 530 identifies a video segment, it can send a signal to the result engine 580 (which can perform similarly to the result engine 120), thereby indicating the identified video segment. The result engine 580 can determine the action to be performed in response to the identified video segment. In some examples, the result engine 580 can identify a background application corresponding to the identified video segment. The background application can be stored on the client device 510. In some examples, the background application can be an application associated with at least a portion of the video segment. The background application can run on the client device 510 to display content on the client device 510. The displayed content can be an overlay on content already displayed on the client device 510 or can replace content displayed on the client device 510.

[0057] The result engine 580 can send signals to the client device 510 to cause the identified background application to perform actions. For example, the background application can display content on the client device 510. The content can be related to the identified video segment. Other actions that the identified background application can perform include, but are not limited to, replacing one commercial message with another commercial message aimed at a specific audience, providing additional information about the identified video segment, or providing an opportunity to interact with the identified video segment or other people watching the identified video segment.

[0058] In some examples, reference datasets in reference database 550 may be managed by retention engine 560. Management of reference datasets may include determining when to remove or delete reference datasets from reference database 550. In some examples, when a reference dataset associated with a video segment is removed, one or more other reference datasets also associated with the video segment may be removed. In some examples, one or more other reference datasets are not removed. Further details regarding the removal or deletion of reference datasets are described below.

[0059] In some examples, reference datasets removed or deleted from reference database 550 may be added to a backup database (not shown). In some examples, the backup database may be located on matching server 520. In other examples, the backup database may be located remotely from matching server 520. In such examples, the backup database may be located in the cloud (e.g., one or more remote servers associated with matching server 520). Candidate determination engine 540 may be configured to search for matching candidate datasets in the backup database after reference database 550 has been searched and no matches have been found or a sufficient number of matches have been found.

[0060] Those skilled in the art will recognize that there may be more than one backup database. Zero or more backup databases may be located in the same location as the matching server 520, and zero or more backup databases may be located far from the matching server 520. In some examples, each additional backup database may retain a reference dataset for a longer period than the previous backup database. When the reference dataset is completely removed from the database associated with the matching server 520, the reference dataset will no longer be able to be compared with the unidentified dataset by the candidate determination engine 540. In some examples, each backup database may be managed by a retention engine 560. In other examples, each backup database may be managed by a separate retention engine. Backup databases may use the same duration as the reference database (essentially doubling the lifespan of the reference dataset), or they may use a different duration based on the same, similar, or different process used to determine the duration of the reference database.

[0061] Those skilled in the art will recognize that the use of multiple backup databases will depend on the resources available to the enterprise managing the backup databases, and on efficiency considerations that would allow the candidate determination engine to search an increasing number of backup databases. In some examples, separate processes may be used to search each backup database, allowing multiple backup databases to be searched with minimal impact on the normal operation of the candidate determination engine 540. In some examples, separate processes may be run in parallel, allowing the multiple backup databases to be searched at least partially in parallel. For example, a first database may be searched until a certain percentage of the first database has been found, at which point a search of the second database can begin in parallel with the first database.

[0062] The retention engine 560 can determine the length of time a reference dataset is retained in the reference database 550 (e.g., referred to as deletion time, retention time, removal time, or any other term indicating the time when a reference dataset is removed or deleted from the reference database 550). The retention engine 560 can use popularity data associated with the video segment (e.g., statistics, numbers, information, or other popularity data) to determine the length of time a reference dataset is retained.

[0063] In some examples, the time length can be different for each reference dataset or any combination thereof, for reference datasets associated with different video segments. Those skilled in the art will recognize that many time length configurations can be applied to the reference datasets.

[0064] In some examples, the duration can be extended whenever the matching server 520 experiences a specific number of views on a video segment (e.g., 1 view per day, 10 views per day, 1 view per week, 10 views per week on average, or any other suitable number of views). Views on a video segment can be determined when the matching server 520 recognizes that the media system is displaying the video segment, as will be discussed below. In some examples, the duration can be re-evaluated when it expires.

[0065] In some examples, the duration can be consistent across all video segments of a content item (e.g., a movie, performance, TV series, etc.). For example, all reference datasets for all video segments of a content item could be retained for one day, one week, or another duration. In some examples, the duration can be consistent across all reference datasets associated with a single video segment. In this example, the reference dataset can be retained even if it is not matched by matching server 520, as long as another reference dataset associated with the same video segment is matched by matching server 520.

[0066] In some examples, popularity data may be correlated with the popularity of identified video segments. Popularity data may include viewing information, which may indicate the number of views of video segments from viewing information source 572. In some examples, the number of views may be correlated with a specific time period. In some examples, the time period may be the day the video segment was broadcast (e.g., the same day as the viewing data) or a slightly later time period after the day the video segment was broadcast (delayed viewing data). Those skilled in the art will recognize that other time periods may be used.

[0067] In some examples, the viewing information source 572 can be a third-party server (e.g., on the internet, such as using the Internet 570). Examples of viewing information can include any type of audience measurement system, including but not limited to Nielsen / NetRatings, Nielsen BuzzMetrics, Nielsen Media Research (NMR) comScore, Wakoopa and Hitwise, Visible Measures, GfK, Sightcorp, TruMedia, Quividi, relEYEble, stickyPiXEL, Cognitec, goCount, and CognoVision, or any other form of audience measurement system.

[0068] In some examples, matching engine 530 can be used to determine viewing information. For instance, when a video segment is identified by matching engine 530 (when a certain number of tokens are reached, as described above), matching server 520 can store a record of when the video segment has been displayed by the media system. In this example, matching server 520 can keep track of the number of times the media system has displayed the video segment. Therefore, for the purpose of viewing information, the number of times the media system has displayed the video segment can be used as the number of views of the video segment.

[0069] Those skilled in the art will also recognize that more than one source of viewing information can be used to determine the duration. For example, viewing information from matching engine 530 and viewing information from viewing information source 572 can be used. As another example, viewing information from a first viewing information source and viewing information from a second viewing information source can be used.

[0070] Popularity data may also include rating information from rating sources 574 for video clips. Rating information can indicate the desirability, popularity, and / or demand of a video clip. For example, rating information could be the percentage of viewers who liked the video clip. The percentage of viewers who liked the video clip could be determined by sources on networks that provide video clip ratings (such as Internet 570). For example, a rating could be a rating given to a video clip by one or more reviewers on a website, a mobile application, etc. In some examples, one or more reviewers could be from at least one or more groups of professional reviewers and entertainment reviewers. Rating source 574 could be a third-party server (e.g., on the internet, such as using Internet 570). Examples of rating sources 574 could include Rotten Tomatoes, Metacritic, Everyone's a Critic, Reviewer, MovieAttractions, Flixster, FilmCrave, Flickchart, blogs, and systems such as "Fluctuation of Expectations Curve" (which uses an automated system to review video clips). Those skilled in the art will also recognize that more than one rating information or source can be used to determine the duration of the time. In an illustrative example, rating information from professional commentators and rating information from groups of entertainment commentators could be used.

[0071] Popularity data may also include information associated with one or more posts (e.g., published messages) on one or more remote sources (e.g., comment sources or social media sources, such as websites and mobile applications). One or more posts may be associated with video clips. Social media sources may be third-party servers (e.g., on the web, such as those using the Internet). Examples of social media sources may include, but are not limited to, Facebook, YouTube, Twitter, LinkedIn, Pinterest, Google Plus+, Instagram, Reddit, and Rotten Tomatoes.

[0072] Examples of information associated with one or more posts may include, but are not limited to, at least one or more of the following: multiple posts from one or more remote sources; the number of posts from one or more remote sources that are positive in nature for the video clip (based on natural language parsing of one or more posts); the number of posts from one or more remote sources that are negative in nature for the video clip (also based on natural language parsing of one or more posts); the number of positive indications from one or more remote sources (e.g., "likes" on Facebook); the number of reposts of posts associated with the video clip on one or more remote sources; the number of views of pages associated with the video clip on one or more remote sources; or any combination thereof. In some examples, a rating (e.g., 1 to 100) may be determined from one or more posts, based on the words used in one or more posts to identify how much the video clip is liked. In such examples, certain words may receive certain point scores. In some examples, the number of times the video clip or the actors associated with the video program are mentioned on social media platforms.

[0073] Those skilled in the art will also recognize that the length of time can be determined using more than one piece of information associated with one or more posts on one or more remote sources. For example, information such as the number of posts from one or more remote sources and the number of positive indications from one or more remote sources can be used.

[0074] In some examples, the duration can be determined by calculating a representative value of the popularity data (e.g., average, mean, median, or some combination thereof). The representative value can indicate the probability that the matching system will (or is likely to) need the reference dataset in the future, justifying the overhead of retaining the reference dataset in the reference database 550. In some examples, the average can indicate the likelihood that a video segment will appear on the media system (or be associated with a matching server or not) within a specific amount of time (e.g., within a day, a week, a month, or any other suitable time period). When calculating the average, the retention engine 560 can identify and use available popularity data. When calculating the average, if the video segment is not included in the source, the reference dataset associated with the video segment may receive zero for that source.

[0075] In some examples, popularity data can be normalized so that all popularity data are based on similar or identical proportions. For example, a rating from 0 to 1 can be multiplied by 100 to correspond to a rating from 0 to 100. As another example, the maximum number for each type of popularity data can be determined by analyzing the type of popularity data or pre-identified by the user. In this example, the number associated with the popularity data type can be divided by the maximum number so that all numbers are a percentage of the maximum. In some examples, numbers greater than the maximum can be allowed to exceed 1 by a certain percentage, or the numbers can be capped at 1.

[0076] In some examples, weights can be applied to popularity data so that some parts of the popularity data have a greater impact on the average than others. In illustrative examples, weighting popularity data could include: direct mentions of video clips or actors associated with video clips could have a higher weight than mentions of video clips in social media communications. Another weighting factor could be the number of words used in messages related to video clips. Yet another weighting factor could be several specific actors associated with video clips referenced in messages. Figure 6 This is a diagram illustrating an example of the relative weights used to determine whether to retain popularity data in a reference dataset. Those skilled in the art will recognize that the weights can be within a range, such as from 0 to 100, from 0 to 1, or other suitable ranges of weight values. The sum of all weights can be equal to 100, 1, or other values ​​at the top of the range. Weights can be multiplied by some portion of the popularity data to enhance or de-enhance the popularity data. Other methods of weighting data can also be used. Although Figure 6 A specific order of relative weights is shown, but those skilled in the art will recognize that different orders and different popularity data can be used.

[0077] In one example, the highest weight (e.g., 50%) can be associated with one or more popularity metrics from the popularity data. Popularity metrics can be viewing information from the matching server (box 620). Viewing information can indicate the number of times a video segment has been identified by the matching server as having been viewed by a media system associated with the matching system. As mentioned above, this number can increase whenever the matching server identifies a video segment as being displayed by a media system.

[0078] In the same example, a moderate weight (e.g., 30%) can be associated with one or more popularity metrics of the popularity data. For example, the first popularity metric of the popularity data could be a list of the top 20 shows for the day of viewing information (box 630). This list could include the number of times the top 20 shows were viewed. The viewing information for the day could be received from a second source (e.g., via the internet from viewing information source 572). Since the viewing information for the day does not come from the matching server but from the second source, the weight of the viewing information for the day could be less than that of the viewing information from the matching server (box 620). The second popularity metric of the popularity data could be social media popularity information (box 650). Social media popularity information could be the number of posts about video clips on one or more social media sources. For example, a wall post on Facebook could mention the name of a video clip. A wall post could be counted as a post about a video clip.

[0079] In the same example, a low weight (e.g., 20%) can also be associated with one or more popularity metrics in the popularity data. For example, the first popularity metric in the popularity data could be a second social media popularity metric (box 640). The second social media popularity metric could be the number of positive indications from one or more social media sources. The second popularity metric in the popularity data could be the number of positive indications from one or more remote sources (box 660). Those skilled in the art will recognize that other weight configurations can be used. Furthermore, any combination of popularity data can be used to calculate the weights.

[0080] Weights can be applied to popularity data or the average of popularity data to create a weighted average of the popularity data for a video segment (box 670). For example, weights can be applied by multiplying by the average data associated with the weights.

[0081] The totals associated with a video segment (e.g., the average, weighted average, or sum of popularity data) can be used to determine the length of time to retain one or more reference datasets associated with the video segment. In some examples, the totals within a certain range can be used to calculate the length of time. In some examples, a range (e.g., 0 to 50, or some other range accepted by the retention engine) can be predetermined based on a specific percentage of the maximum possible total, based on the total number of views through the matching system, or some other method of determining the range. In an illustrative example of the percentage of the maximum possible total, the maximum number of each popularity data point used to calculate the total is summed together to determine the maximum possible total. Each range can then represent a certain percentage of the maximum possible total. For example, the first range could be all totals, which fall between 0% and 50% of the maximum possible total.

[0082] In an illustrative example, when the total number is within a first range, the reference dataset associated with the video segments associated with the total number may retain a first time period (e.g., one day). When the total number is within a second range, the reference dataset associated with the video segments associated with the total number may retain a second time period (e.g., one week). Those skilled in the art will recognize that these ranges, time periods, and the number of ranges may differ from those shown above.

[0083] The process of determining the time length can be dynamically configured by the retention engine 560. For example, the range can be manipulated during normal processing to see if different ranges provide an effective balance between the number of matches and the size of the reference database. The effectiveness of the time length determination can be based on the determined number of matches and the average time between matches in the reference database (where the average time is a positive integer such as nanoseconds). For example, an efficiency metric can be calculated by dividing the determined number of matches by the average time. The efficiency metric can be optimized by changing the range, time period, and number of ranges in small increments until an optimized efficiency metric is found. Small increments can be tested by rematching past popularity data with different retention strategies (e.g., various configurations of the range, time period, and number of ranges) to see if the efficiency metric increases or decreases.

[0084] In some examples, the duration can be determined based on viewing statistics, such as the average number of days a video segment is viewed. This data is available from companies like AC Nielsen and Rentrack. The data can then be correlated with popularity data (potentially weighted by genre, such as sports, reality TV, crime series, etc.). The correlated data can then provide a direct time measure relative to popularity.

[0085] In other examples, the duration can be determined based on a weighted average tracking value. For instance, when the average value falls below a predetermined limit (e.g., below 50%), video segment data can stop being retained for a predetermined period of time.

[0086] Figure 7 This is a flowchart illustrating an example of a process 700 for optimizing resource utilization in a matching system. Process 700 is shown as a logic flowchart, where operations represent a sequence of operations that can be implemented in hardware, computer instructions, or a combination thereof. In the context of computer instructions, the operation represents a computer-executable instruction stored on one or more computer-readable storage media that performs the operation when executed by one or more processors. Typically, computer-executable instructions include routines, programs, objects, components, data structures, etc., that perform a specific function or implement a specific data type. The order in which the operations are described is not intended to be construed as limiting, and any number of the operations can be combined in any order and / or in parallel to implement the process.

[0087] Furthermore, process 700 can be executed under the control of one or more computer systems configured with executable instructions, and can be implemented as code (e.g., executable instructions, one or more computer programs, or one or more application programs) that executes jointly on one or more processors, implemented in hardware, or a combination of both. As described above, for example, in the form of a computer program comprising multiple instructions executable by one or more processors, the code can be stored on a machine-readable storage medium. The machine-readable storage medium can be non-transitory.

[0088] At step 710, process 700 includes receiving a reference dataset associated with a media segment. In some examples, the reference dataset may be received by a server. The server may be configured to identify an unidentified media segment by matching an unidentified dataset with the reference dataset. In some examples, an unidentified dataset may be associated with an unidentified media segment. In some examples, the reference dataset may include pixel data or audio data of frames from the media segment.

[0089] At step 720, process 700 includes storing the reference dataset in a reference database.

[0090] At step 730, process 700 includes receiving popularity data indicating the popularity of a media segment. In some examples, the popularity data includes at least one or more of the following: viewing information for the media segment, rating information for the media segment, or information associated with a published message on a remote source. In such examples, a published message may be associated with a media segment. In some examples, the information associated with a published message on a remote source may include at least one or more of the following: the number of published messages on the remote source, the number of positive messages published, the number of negative messages published, the number of positive indications for the published message, the number of republished messages, or the number of views for the published message. In some examples, a published message may be associated with a media segment. In some examples, viewing information may include the number of times the media segment has been identified by the server.

[0091] At step 740, process 700 includes determining the deletion time for the reference dataset. In some examples, the deletion time can be determined using popularity data. In some examples, determining the deletion time may include calculating the average of two or more values ​​of the popularity data. In this example, each value of the popularity data may be associated with a weight.

[0092] At step 750, process 700 includes deleting the reference dataset from the reference database. In some examples, the reference dataset may be deleted after a deletion period has expired.

[0093] In some implementations, process 700 may further include adding the reference set to the backup database when the reference dataset is deleted from the reference database. In this implementation, the server may be configured to search the backup database after the reference database for a dataset that matches the unidentified dataset.

[0094] As discussed above, a video matching system can be configured to identify a media content stream when it includes unscheduled media segments. Further as discussed above, identifying the media content stream may include identifying media content played by a media display device before or after the unscheduled media segments. The above is about... Figure 1 The process for identifying media content is discussed. Specifically, a video content system can use samples (e.g., graphic and / or audio samples) obtained from the display mechanism of a media content device and generate cues from these samples. A video matching system can then match the cues against a reference database containing cues of known content.

[0095] Video matching systems can further include various methods for improving the efficiency of finding potential matches or "candidates" in a database. The database can contain a large number of cues, and therefore the video matching system can include algorithms for finding candidate cues to match with cues generated from the display apparatus of the media content device. Locating candidate cues may be more efficient than other methods used to match cue values ​​with values ​​in a database, such as matching cue values ​​with every entry in the database.

[0096] Nearest neighbor and path tracing are examples of techniques that can be used to locate candidate queues in a reference database. An example of using fuzzy cues to track video transmission is given below, but the general concepts can be applied to any domain of selecting matching candidates from a reference database.

[0097] A method for efficient video tracking is presented. Given a large number of video segments, the system must be able to identify in real time which segment a given query video input comes from and at what time offset. The segment and offset together are referred to as the position. This method is called video tracking because it must be able to efficiently detect and adapt to pauses, fast forwards, rewinds, sudden switches to other segments, and switches to unknown segments. Before being able to track real-time video, a database is processed. Visual cues (a few pixel values) are extracted from the frames every fraction of a second and placed into a specialized data structure (note that this can also be done in real time). Video tracking is performed by continuously receiving cues from the input video and updating a set of certainties or estimates about its current position. Each cue either agrees with or disagrees with the estimate, and they are adjusted to reflect new evidence. If the confidence in this is high enough, the video position is assumed to be the correct position. This can be done efficiently by tracking only a small fraction of possible "suspicious" positions.

[0098] Methods for video tracking are described, but mathematical structures are used to explain and investigate them. The purpose of introducing these mathematical structures is to provide the reader with the necessary tools for translating between these two domains. Video signals, or video data, consist of consecutive frames. Each frame can be considered a still image. Each frame is a raster of pixels. Each pixel consists of three intensity values ​​corresponding to the red, green, and blue (RGB) that make up the pixel's color. In the terminology used in this paper, a cue is a list of the RGB values ​​of a subset of pixels in a frame, along with their corresponding timestamps. The number of pixels in a cue is significantly smaller than the number of pixels in a frame, typically between 5 and 15. As an ordered list of scalar values, cue values ​​are actually a vector. This vector is also referred to as a point.

[0099] Although these points exist in a high dimension, typically between 15 and 150, they can be imagined as points in two dimensions. In fact, the illustration will be given as a two-dimensional drawing. Now, consider the progression of the video and its corresponding cue points. Typically, small temporal changes result in small changes in pixel values. Pixels can be viewed as slightly "moving" between frames. After these tiny movements from frame to frame, the cue follows a path in space, like beads on a curved line.

[0100] In this analogical language, in video tracking, we receive the position of a bead in space (clue point) and search for a portion of the line the bead follows (path). This is significantly more difficult for two reasons. First, the bead doesn't follow the line precisely, but rather maintains a varying, unknown distance from it. Second, the lines are all tangled. These statements are more precise in Section 2. The algorithm described below accomplishes this task in two conceptual steps. When a clue is received, the algorithm finds all points on all known paths that are sufficiently close to the clue point; these points are called suspects. This is done efficiently using probabilistic point locations in the isosphere algorithm. These suspects are added to a history data structure, and the probability of each of them indicating the true location is calculated. This step further includes removing less likely suspect locations. This history update process ensures, on the one hand, that only a small portion of history is retained, and on the other hand, that possible locations are never deleted. A general algorithm is given in Algorithm 1, and in Figure 10 As shown in the image. The next section begins with a description of the Probability Point Location in Equal Spheres (PPLEB) algorithm from Section 1. The PPLEB algorithm is used to efficiently execute line 5 of Algorithm 1 above. The ability to quickly perform the search for suspects is crucial to the applicability of this method. In Section 2, a possible statistical model for executing lines 6 and 7 is described. The described model is a natural choice for the setup. It also shows how it can be used very efficiently.

[0101] Section 1 - Location of Probability Points in Isospheres The next section describes a simple algorithm for performing Probabilistic Point Locations in Isospheres (PPLEB). In the traditional PLEB (Point Locations in Isospheres), the algorithm starts with a set of n points x in a specific sphere of radius r and radius lR d. The algorithm is given O(n) preprocessing time to produce an efficient data structure. Then, given a query point x, the algorithm needs to return all points x such that The set of points makes Geometrically located within a sphere of radius r surrounding the query x (see...) Figure 9 The relationship is called x, which is close to x or is x, and x is a neighbor.

[0102] The PPLEB problem and the nearest neighbor search problem are two similar problems that have received much attention in academia. In fact, these are among the earliest problems studied in computational geometry. Many different methods cater to cases where the environment dimension is small or invariant. These partition the space in different ways and recursively search each part. These methods include KD-trees, cover trees, and others. Although they are very efficient in low dimensions, they tend to perform poorly when the environment dimension is high. This is known as the "curse of dimensionality." Various methods attempt to address this problem while overcoming the curse of dimensionality. The algorithm used in this paper employs a simpler and faster version of the algorithm and can rely on Local Sensitive Hashing.

[0103] Section 1.1 Locally Sensitive Hash In locality-sensitive hashing schemes, a family of hash functions H is designed such that: In other words, if x and y are close to each other, the probability that x and y are mapped to the same value h is significantly higher.

[0104] For clarity, let's first discuss all incoming vectors having the same length r' and r'> The simplified case. The reason for the latter condition will become clear later. First, define a random function u ∈ U, which is separated between x and y by the angle between x and y. Let From unit ball S d-1 Let u(x) = sign (a uniformly selected random vector). It's easy to verify. Furthermore, for any points x, y, x', y' on the circle, such that... , Define p using the following equation: The family of functions H is defined as the cross product of t independent copies of u, i.e., h(x) = [u1(x), ..., u]. t (x)]. Intuitively, one would expect that if h(x) = h(y), then x and y are likely close to each other. Let's quantify this. First, calculate the expected number n of false positives. fp These are cases where h(x) = h(y) but ||xy|| > 2r. Find n fp A value t not exceeding 1, that is, a prediction is unlikely to be wrong. Suppose h(x) and h(y) are neighbors, now calculate the probability that h(x) = h(y): Note that 2p < 1 must be achieved, which requires r' > This might not sound like a very high probability of success. In fact, 1 / It is significantly less than ½. The next section will explain how to increase this probability to ½.

[0105] Node Search Algorithm 1.2 Each function h maps each point in space to a bucket. The bucket function for point x... The hash function h is defined as follows: The maintained data structure is a bucket function [Bh1,…,Bh...]. m m=O ( ) Example. When searching for point x, the function returns... Based on the previous section, there are two expected outcomes: In other words, although there is at least a 50% probability of finding each of x's neighbors, it is impossible to find many non-neighbors.

[0106] Section 1.3 handles different radius input vectors The previous section only dealt with searching vectors of the same length, i.e., r'. This section describes how to use this construct as building blocks to support searches with different radii. For example... Figure 11 As can be seen, the space is divided into several rings with exponentially increasing widths. (From R...) i The ring i represents all points x. i , making This achieves two objectives. First, if x i and x j If they belong to the same ring, then Secondly, any search can be performed in at most 1 / This is performed within a ring. Furthermore, if the longest vector in the dataset is r', the total number of rings in the system is O(log(r' / r)).

[0107] Section 2 Path Tracing Problem In the path tracing problem, a fixed path in space is given along with the positions of particles in a time-point sequence. The terms "particle," "clue," and "point" are used interchangeably. The algorithm needs to output the positions of the particles on the path. This is made more difficult by several factors: the particles only approximately follow the path; the path can be discontinuous and intersect multiple times; and both the particle and path positions are given as a time-point sequence (each time point is different).

[0108] It is important to note that this problem can be simulated by tracking particles on any number of paths. This can be done by simply connecting the paths into a long path and interpreting the resulting positions as positions on a single path.

[0109] More precisely, let path P be a parametric curve. The curve parameter will be referred to as time. The points on the path that we know are at any time point t. i Given, that is, given n pairs (t) i ,P(t) i The particle follows the path, but its position is given at different points in time, such as... Figure 12 As shown in the diagram. Furthermore, m pairs (t') are also given. j ,x(t' j ), where x (t' j ) is time t' j The position of the particles.

[0110] Section 2.1 Likelihood Estimation Because particles do not follow the path precisely, and because the paths can intersect multiple times, it is usually impossible to definitively identify the particle's actual position on the path. Therefore, probability distributions are calculated for all possible path positions. If a position probability is significantly probable, the particle's position is assumed to be known. The next section describes how to do this efficiently.

[0111] If a particle follows a path, the time difference between the particle's timestamp and the offset of the corresponding point on P should be relatively constant. In other words, if x(t') is currently at offset t on the path, it should be close to P(t). Also, τ seconds ago, it should have already been at offset t-τ. Therefore, x(t'-τ) should be close to P(t-τ) (note that if the particle intersects the path and x(t') is temporarily close to P(t), then x(t'-τ) and P(t-τ) are unlikely to also be close). Define the relative offset as Δ = t - t'. Note that as long as the particle follows the path, the relative offset Δ remains constant. That is, x(t') is close to P(t'+ t'). ).

[0112] The maximum likelihood relative shift is obtained through calculation: In other words, the most likely relative offset is the one to which the particle's history is most probable. However, this equation cannot be solved without a statistical model. The model must quantify: how closely x follows the path; the probability of x jumping between positions; and how smooth the path and particle curves are between measurement points.

[0113] Section 2.2 Time Discount Position Building The statistical model used to estimate the likelihood function is now described. This model assumes that the deviation of a particle from its path is normally distributed with a standard deviation *ar*. It also assumes some non-zero probability that at any given time point, a particle will suddenly switch to another path. This is reflected in the exponential discount over past points. Besides being a reasonable choice for modeling perspective, this model has the advantage of being efficiently updatable. For some constant time unit 1, the likelihood function is set to be proportional to *f*, defined as follows: here <<1 is the proportionality constant and It is the probability that a particle will jump to a random position on the path within a given time unit.

[0114] Effectively update function f This can be achieved using the following simple observation. In addition, due to <<1, if Then the following situation will occur: This is an important property of the likelihood function, because the summation update can now be performed only on x(t'). j ) is executed on the neighbor of ) instead of on the entire path. Let S represent (t i , P(ti The set of )) makes The following equations occur: This is described in Algorithm 2.2 below. Item f is used as a sparse vector that also receives negative integer indices. The set S is the path x(t) on the path x(t). i The set of all neighbors of x(t) can be calculated quickly using the PPLEB algorithm. It is easy to verify that if x(t) i The number of neighbors of a given number is subject to some constant n. near If there is a restriction, then the number of non-zero values ​​in vector f is determined by factors that are only larger constants. Constraints. The final stage of this algorithm is if... If the value exceeds a certain threshold, a specific value δ will be output. 1: 2: When Input, proceed 3: 4: 5: Regarding ,conduct 6: 7: 8: End 9: Set all f values ​​below the threshold ∈ to 0. 10: End Figure 8 Given three consecutive point locations and the path points around them. Note that neither the lowest point nor the middle point alone is sufficient to identify the correct part of the path. However, they can be identified together. Adding vertices increases the certainty that the particle is indeed the final (left) curve of the path.

[0115] exist Figure 9 Given a set of n (gray) points, the algorithm is given a query point (black) and returns a set of points (points within a circle) that are within a distance r from the query point. In a conventional setting, the algorithm must return all of these points. In a probabilistic setting, each such point should be returned with some constant probability.

[0116] Figure 10The values ​​of u(x1), u(x2), and u(x) are shown. Intuitively, if the dashed line passes between them, the function u assigns different values ​​to x1 and x2; otherwise, it assigns the same value. The dashed line passing in a random direction ensures that the probability of this happening is proportional to the angle between x1 and x2.

[0117] Figure 11 This illustrates how dividing space into multiple rings makes ring R i In radius 2r(1+ ) i and 2r(1+ ) i+1 Between them, it can be ensured that the lengths of any two vectors within the ring span multiple (1+ The factors are all the same, and any search is at most 1 / Execution within the ring.

[0118] Figure 12 The self-intersecting paths and query points (black) are shown. This demonstrates that without the history of a particle's position, it is impossible to know its location on the path.

[0119] Figure 13 Given three consecutive point locations and the path points around them. Note that x(t1) and x(t2) alone are insufficient to identify the correct portion of the path. However, they can be identified together. Adding x(t3) increases the certainty that the particle is indeed the final (left) curve of the path.

[0120] Substantial changes can be made to meet specific requirements. For example, custom hardware can be used, and / or specific elements can be implemented using hardware, software (including portable software such as applets), or both. Furthermore, connections to other access or computing devices, such as network input / output devices, can be utilized.

[0121] In the foregoing description, various aspects of the invention have been described with reference to specific examples; however, those skilled in the art will recognize that the invention is not limited thereto. The various features and aspects of the invention described above can be used alone or in combination. Furthermore, examples can be used in any number of settings and applications other than those described herein without departing from the broader spirit and scope of this specification. Therefore, the specification and drawings are to be considered illustrative rather than restrictive.

[0122] In the foregoing description, the methods have been described in a specific order for illustrative purposes. It should be understood that, in alternative examples, these methods may be performed in a different order than described. It should also be understood that the methods described above can be executed by hardware components or can be implemented in the order of machine-executable instructions that can be used to cause a machine (such as a general-purpose or special-purpose processor or logic circuits programmed with these instructions) to execute the method. These machine-executable instructions can be stored on one or more machine-readable media, such as CD-ROMs or other types of optical discs, floppy disks, ROMs, RAMs, EPROMs, EEPROMs, magnetic or optical cards, flash memory, or other types of machine-readable media suitable for storing electronic instructions. Alternatively, the method can be executed by a combination of hardware and software.

[0123] When a component is described as being configured to perform certain operations, such configuration can be achieved, for example, by designing electronic circuitry or other hardware that performs the operations, by programming the electronic circuitry that performs the operations (e.g., a microprocessor or other suitable electronic circuitry), or any combination thereof.

[0124] While illustrative examples of this application have been described in detail herein, it should be understood that the inventive concept may be embodied and adopted in other ways, and the appended claims are intended to be construed as including such variations, in addition to being limited by the prior art.

Claims

1. A computer-implemented method, comprising: A server receives a reference dataset associated with a media segment, wherein the server is configured to identify an unidentified media segment by matching an unidentified dataset with the reference dataset, and wherein the unidentified dataset is associated with the unidentified media segment. The reference dataset is stored in a reference database; Receive popularity data indicating the popularity of the media segment, wherein the popularity data includes values ​​associated with at least one or more of the following: viewing information of the media segment, rating information of the media segment, or information associated with a posting message on a remote source, and wherein the posting message is associated with the media segment; The deletion time of the reference dataset is determined by calculating a representative value of two or more values ​​of the popularity data, wherein the representative value is one of the average, mean, or median of the two or more values ​​of the popularity data; as well as The reference dataset is deleted from the reference database, wherein the reference dataset is deleted after the deletion time expires.

2. The computer-implemented method according to claim 1, further comprising: When the reference dataset is deleted from the reference database, the reference dataset is added to the backup database, wherein the server is configured to search the backup database after the reference database to find a dataset that matches the unidentified dataset.

3. The computer-implemented method according to claim 1, wherein the reference dataset includes pixel data or audio data of frames of the media segment.

4. The computer-implemented method of claim 1, wherein determining the deletion time of the reference dataset includes using corresponding weights associated with one or more values ​​of the popularity data.

5. The computer-implemented method of claim 1, wherein the information associated with the published message on the remote source includes at least one or more of the following: the number of published messages on the remote source, the number of positive messages published, the number of negative messages published, the number of positive indications to the published message, the number of republished messages, or the number of views of the published message, and wherein the published message is associated with the media segment.

6. The computer-implemented method of claim 1, wherein the viewing information includes the number of times the media segment has been identified by the server.

7. A system comprising: One or more processors; as well as A non-transitory computer-readable medium containing instructions that, when executed by the one or more processors, cause the one or more processors to perform operations, the operations including: A server receives a reference dataset associated with a media segment, wherein the server is configured to identify an unidentified media segment by matching an unidentified dataset with the reference dataset, and wherein the unidentified dataset is associated with the unidentified media segment. The reference dataset is stored in a reference database; Receive popularity data indicating the popularity of the media segment, wherein the popularity data includes values ​​associated with at least one or more of the following: viewing information of the media segment, rating information of the media segment, or information associated with a posting message on a remote source, and wherein the posting message is associated with the media segment; Determine the deletion time of the reference dataset, wherein the deletion time is determined by calculating a representative value of two or more values ​​of the popularity data, the representative value being one of the average, mean, or median of the two or more values ​​of the popularity data; and The reference dataset is deleted from the reference database, wherein the reference dataset is deleted after the deletion time expires.

8. The system of claim 7, further comprising instructions that, when executed by the one or more processors, cause the one or more processors to perform an operation, the operation comprising: When the reference dataset is deleted from the reference database, the reference dataset is added to the backup database, wherein the server is configured to search the backup database after the reference database to find a dataset that matches the unidentified dataset.

9. The system of claim 7, wherein the reference dataset comprises pixel data or audio data of frames of the media segment.

10. The system of claim 7, wherein determining the deletion time of the reference dataset includes using corresponding weights associated with one or more values ​​of the popularity data.

11. The system of claim 9, wherein the information associated with the published message on the remote source includes at least one or more of the following: the number of published messages on the remote source, the number of positive messages published, the number of negative messages published, the number of positive indications of the published message, the number of republished messages of the published message, or the number of views of the published message, and wherein the published message is associated with the media segment.

12. The system of claim 7, wherein the viewing information includes the number of times the media segment has been identified by the server.

13. A computer program product tangibly embodied in a non-transitory machine-readable storage medium, said non-transitory machine-readable storage medium comprising instructions that, when executed by one or more processors, cause said one or more processors to: A server receives a reference dataset associated with a media segment, wherein the server is configured to identify an unidentified media segment by matching an unidentified dataset with the reference dataset, and wherein the unidentified dataset is associated with the unidentified media segment. The reference dataset is stored in a reference database; Receive popularity data indicating the popularity of the media segment, wherein the popularity data includes values ​​associated with at least one or more of the following: viewing information of the media segment, rating information of the media segment, or information associated with a posting message on a remote source, and wherein the posting message is associated with the media segment; The deletion time of the reference dataset is determined by calculating a representative value of two or more values ​​of the popularity data, wherein the representative value is one of the average, mean, or median of the two or more values ​​of the popularity data; as well as The reference dataset is deleted from the reference database, wherein the reference dataset is deleted after the deletion time expires.

14. The computer program product of claim 13, further comprising instructions that, when executed by the one or more processors, cause the one or more processors to: When the reference dataset is deleted from the reference database, the reference dataset is added to the backup database, wherein the server is configured to search the backup database after the reference database to find a dataset that matches the unidentified dataset.

15. The computer program product of claim 13, wherein the reference dataset comprises pixel data or audio data of frames of the media segment.

16. The computer program product of claim 13, wherein determining the deletion time of the reference dataset includes using corresponding weights associated with one or more values ​​of the popularity data.

17. The computer program product of claim 13, wherein the information associated with the published message on the remote source includes at least one or more of the following: the number of published messages on the remote source, the number of positive messages published, the number of negative messages published, the number of positive indications to the published message, the number of republished messages, or the number of views of the published message, and wherein the published message is associated with the media segment.

18. The computer program product of claim 13, wherein the viewing information includes the number of times the media segment has been recognized by the server.

Images