System and method for identifying and characterizing signals contained in a data stream
Through machine learning models, modeling and threshold detection of Weibo counting time series, the problems of untimely and inaccurate detection of events in the prior art are solved, and more efficient and timely information transmission and wide dissemination of Weibo content are achieved.
Patent Information
- Application Number
- CN201780080757.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2016-12-29
- Filing Date
- 2017-10-27
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2038-09-20
AI Technical Summary
The prior art is difficult to accurately and promptly detect and notify ongoing events, such as news, which leads to inefficient access to user information, wasted computing resources and network bandwidth, and Weibo content fails to reach a wide range of audiences in a timely manner.
Through machine learning models, model the Weibo count time series, extract the predictable part and residual part, and determine the trigger threshold. When the residual Weibo count exceeds the threshold, notify the user and provide relevant content, and use the data analysis engine and related information. The sexual analysis engine processes query and Weibo data, optimizing detection of surge in event counts.
It has achieved earlier and more accurate detection of breaking news, reduced the number of information searches, saved computing resources and network bandwidth, improved the timeliness and breadth of Weibo content, and encouraged more people and organizations to use Weibo.
Smart Images

Figure CN110121729B_ABST
Abstract
Description
Technical Field
[0001] This specification relates to systems and methods for identifying and characterizing signals contained in data streams, such as signals contained in a time series of a data stream within a relevant time period associated with a subject. Background Technology
[0002] Individuals use devices to digitally record many aspects of their lives, as well as an increasing number of events and topics. Such individuals use a variety of devices, such as mobile phones, tablets, laptops, or desktop computers, and other sensors, such as cameras or wearable sensors, via the Internet of Things (IoT). Therefore, it becomes possible to understand events or perspectives as they unfold and are reflected in digital media. In fact, there is both a need and an opportunity to accurately and early detect unfolding events (such as breaking news) through digital media and to provide this information to users. Summary of the Invention
[0003] This specification describes techniques for identifying and characterizing signals contained in data streams, such as signals contained in a time series of microblog counts associated with a query within a relevant time period.
[0004] Generally, an innovative aspect of the subject matter described in this specification can be embodied in a method comprising the following actions: obtaining a historical temporal distribution of event counts associated with the subject over a relevant time period; extracting a predictable portion of the historical temporal distribution of the event counts to generate a residual event count temporal distribution that includes residual event counts over consecutive time periods; determining a residual trigger threshold based on the residual event count temporal distribution; and taking action when the residual event count exceeds the residual trigger threshold. This action may include providing a user with a notification of a spike in the event counts associated with the subject. In one embodiment, the event may be a microblog post, and the action may be forwarding data to display the microblog data as part of the search results.
[0005] Another innovative aspect of the subject matter described in this specification can be embodied in a method comprising the following actions: receiving a query; obtaining a time series of microblog counts associated with the query over a relevant time period; extracting a predictable portion of the microblog count time series to generate a residual microblog count time series over the relevant time period, the residual microblog count time series comprising residual microblog counts over consecutive time periods; determining a residual trigger threshold based on the residual microblog count time series; and forwarding data to display microblog content as part of the search results for a given query when the residual microblog count exceeds the residual trigger threshold.
[0006] The foregoing and other embodiments may each optionally include one or more of the following features, individually or in combination. In particular, one embodiment includes a combination of all of the following features. The method may include using a machine learning model to predict a predictable portion of a tweet count time series. The tweet count may be a count of tweets provided on the Twitter platform. As a result of the method described herein, the method stops inserting tweet content as part of the search results for a specified time after an excessive number of tweet counts fall below the threshold. The relevant time period for the tweet count time series is between 1 and 7 days. The residual trigger threshold is determined at least in part based on measurements of the median and variance of the residual time series. The method also includes: communicating to a user a confidence measure that the residual tweet count reflects an event that should be notified to the user, the confidence measure being at least in part based on the degree to which the residual tweet count exceeds the trigger threshold. Determining the residual trigger threshold may be at least in part based on the median and variance of the residual time series. The method may also include communicating to a user a confidence measure that the residual tweet count reflects an event that should be notified to the user, the confidence measure being at least in part based on the degree to which the residual tweet count exceeds the trigger threshold. This method may also include incorporating user interaction with the provided microblog content when determining whether to offer additional microblog content as part of the search results for a query. The method also includes limiting the microblog count time series to microblogs from specific locations.
[0007] Other embodiments of this aspect include corresponding computer systems, apparatuses, and computer programs recorded on one or more computer storage devices, each configured to perform the actions of these methods. For a system of one or more computers to be configured to perform a specific operation or action, this means that software, firmware, hardware, or a combination thereof are installed thereon, which, in operation, cause the system to perform the operation or action. For one or more computer programs to be configured to perform a specific operation or action, this means that the one or more programs include instructions that, when executed by a data processing apparatus, cause the apparatus to perform the operation or action.
[0008] The subject matter described in this specification can be implemented in specific embodiments to achieve one or more of the following advantages. By receiving news of unfolding events earlier and more accurately, users can obtain information more efficiently and in a more timely manner. Depending on the context, timely receipt of unfolding news and the wisdom of the masses is highly advantageous. Furthermore, timely and accurate delivery of notifications of unfolding events can reduce the number of searches required to find information about those events, thereby saving computing resources and freeing up network bandwidth for greater efficiency. In addition, microblog users and other publishers benefit because their content can be immediately reached a broad, positive, and appropriate audience. This encourages more people and organizations to use microblogs, and to use them faster and more accurately, which is generally beneficial for information and communication.
[0009] Details of one or more embodiments of the subject matter of this specification are set forth in the accompanying drawings and the following description. Other features, aspects, and advantages of the subject matter will become apparent from the specification, drawings, and claims. Attached Figure Description
[0010] Figure 1 It is a schematic diagram of a system used to identify and characterize signals contained in a data stream.
[0011] Figure 2 It is a flowchart of a method for identifying and characterizing signals contained in a data stream.
[0012] Figure 3 It is a flowchart of an alternative method for identifying and characterizing signals contained in a data stream.
[0013] Figure 4 Two charts show the time series data of event counts for events that match the query.
[0014] Figure 5 Two graphs are shown, illustrating the event count time series data that matches another query, and these graphs reveal how, when using... Figure 2 When using this method, avoid triggering slower increases.
[0015] Figure 6 This is an example of an event carousel embedded in a search results page that responds to a query.
[0016] Similar reference numerals and names in various accompanying drawings indicate similar elements. Detailed Implementation
[0017] Determining when to notify users of ongoing events on search engine platforms or other online platforms is challenging. Such platforms should notify users accurately and as early as possible, providing context rather than delivering incorrect notifications.
[0018] The embodiments described in this specification provide a machine learning method that models the history of near real-time event counts (e.g., tweet counts) matching a given query to determine when a spike occurs. The advantage of this method is that it detects breaking news earlier and more accurately.
[0019] More specifically, when such a time series model is available, notifications triggered by a surge in near-real-time event counts (e.g., tweet counts) based on the original time series can be improved. As mentioned above, it is difficult to predict trending activities that should trigger actions, such as notifications to users. The embodiments described in this specification address this problem by first predicting what the data counts (e.g., tweet counts) would be under “normal” conditions; that is, the embodiments extract a predictable portion of the tweet count time series and then apply triggering logic based on how much the actual counts differ from their predicted counts. This approach can adjust for predictable time series fluctuations, such as the time of day. For example, this approach excludes date changes from contributing to triggering decisions, so that an expected increase in activity (e.g., in the morning) is not mistaken for a surge.
[0020] To build such a model, the embodiments described in this specification collect training data and use a regularized regression model to produce an interpretable predictive model. This predictive model provides an improved surge detection mechanism.
[0021] Figure 1 An example system 100 for detecting and characterizing signals in a data stream is illustrated. The system receives data such as microblog content (e.g., tweets and retweets) 102 from a data source such as microblog sources, which is fed into three different parts of the system: a data analysis engine 104, a user quality database 106, and a search index 108. The data analysis engine 104 generates time series data associated with topics or queries (e.g., microblogs). The user quality database 106 determines user quality scores and user locations for users who create microblogs. The search index 108 indexes the microblog content. The system also includes a relevance analysis engine 110.
[0022] In operation, the user inputs a query into the search engine using computing device 112. The query is received by the relevance analysis engine 110 (in some cases via the search engine front-end). At step A, the relevance analysis engine 110 forwards the query to the data analysis engine 104. At step B, the data analysis engine 104 returns the historical distribution of microblog counts to the relevance analysis engine 110, for example, a time series of microblog counts for the query over a relevant time period such as the past few days. The data analysis engine 104 may also return data about the location of relevant microblogs and associated tag data to the relevance analysis engine 110.
[0023] In some embodiments described in this specification, a microblog post (e.g., a tweet) is associated with a query when it contains a substantive query term or a synonym of a substantive query term. However, in one embodiment, if the query includes more than one substantive term, and a microblog post contains only one of those substantive terms, it is not considered associated with the query. For example, a microblog post that only mentions X will not be counted in the query [XY]. Some embodiments also eliminate non-substantive terms. Substantive terms can vary depending on the context. For example, the query “the who”, where the word “the” is highly substantive.
[0024] In some embodiments, queries from relevance analysis engine 110 to data analysis engine 104 consider only the text of the query and the text of the microblog. Responses from data analysis engine 104 notify relevance analysis engine 110 of multidimensional patterns in the relevant microblogs. Knowing these patterns, relevance analysis engine 110 issues a query to search index 108, which may associate microblogs (e.g., tweets) with the query because of any combination of the following: the microblog's timestamp, the country where the microblog was posted, tags in the microblog, entities mentioned in the microblog (e.g., Joe Celebrity), the sub-country location where the microblog was posted, the microblog username mentioned in the microblog, and words (single characters) or phrases in the microblog.
[0025] Based on the distribution data received from the data analysis engine 104, the relevance analysis engine 110 determines whether to take action (e.g., notify the user) or include the microblog content in the search results provided by the associated search engine in response to the query. If the relevance analysis engine 110 determines in response to the user-submitted query that the microblog content should be included in the search results, then the relevance analysis engine 110 sends a query to the search index 108 and receives the relevant microblog content as a return.
[0026] Figure 2This is a flowchart of an example process 200 for detecting and characterizing signals in a data stream (e.g., signals in a Weibo count time series). For convenience, process 200 will be described as being executed by a system of one or more computers located in one or more locations and appropriately programmed according to this specification. For example, a system for detecting and characterizing signals in a data stream (e.g., Figure 1 The system 100, when properly programmed, can execute method 200.
[0027] One embodiment of the method includes receiving a query (202), such as a query entered by a user into a search engine; obtaining (e.g., from a data analytics engine) a time series of microblog counts associated with the query over a relevant time period (204); extracting (e.g., at a relevance analysis engine) a predictable portion of the microblog count time series (206) to generate a residual time series comprising residual microblog counts over consecutive time periods; determining a residual trigger threshold (208) based on the residual time series (e.g., at the relevance analysis engine); and forwarding data (210, such as data representing microblog content as part of the search results for the query) when the residual microblog count exceeds the residual trigger threshold; and providing this data for displaying (210, such as via the relevance analysis engine) the microblog content as part of the search results for the query. In one embodiment, the microblog content is provided as part of the search results in a microblog carousel. In another embodiment, the microblog content is simply included in the search results.
[0028] Therefore, certain embodiments described in this specification are related to the delay between a real-world event (e.g., a news event) and the time it takes for the relevance analysis engine 110 to determine that the system should take an action, such as providing a notification to the user. The timeline may progress as follows: a news event occurs; 5 minutes later, the tweet count (e.g., the number of tweets) associated with a query about the news event begins to rise; 10 minutes later, the relevance analysis engine 110 determines that the system should take action (i.e., the relevance analysis engine determines that there is a “surge” in the tweet count for the relevant query relative to the predicted count); the associated search engine responds to the relevant query by displaying tweets as search results. The embodiments described in this specification shorten the time it takes for the relevance analysis engine to determine that the system should take action.
[0029] Figure 3This is a flowchart of an alternative method for identifying and characterizing signals contained in an event data stream. The method 300 shown includes: obtaining 302 a historical temporal distribution of event counts associated with a topic over a relevant time period; (e.g., at a relevance analysis engine) extracting 304 a predictable portion of the historical temporal distribution of the event counts to generate a residual event count temporal distribution that includes residual event counts over consecutive time periods; determining 306 a residual trigger threshold based on the residual event count temporal distribution (e.g., at the relevance analysis engine); and taking action 308 when the residual event count exceeds the residual trigger threshold (e.g., at the relevance analysis engine). In one embodiment, an event count is the number of microblogs (e.g., tweets) created within a time interval (bucket) that match a query. In this example, an “event” refers to the creation of a relevant microblog. However, an event can also be the creation of other forms of social media, academic articles, or other content reflecting an ongoing event.
[0030] As described above, the embodiments described in this specification collect training data and use a regularized regression model to produce an interpretable predictive model. When deriving the predictive model, the least absolute shrinkage and selection operator (LASSO) regression can be used. In statistics and machine learning, LASSO is a regression analysis method that performs both variable selection and regularization to improve the predictive accuracy and interpretability of the resulting statistical model. To derive the predictive model, a large number of time series of different queries can be collected over a period of time. Such a historical dataset (which includes attributes such as time series timestamps of tweet counts or global (query-independent) time series) is used as the training set to build a machine learning model that predicts the next segment of tweet counts for a given query.
[0031] Therefore, the embodiments described in this specification use predictive models to anticipate the predictable portion of near real-time event counts, such as predicting the predictable portion of a time series of microblog counts associated with a given query or topic.
[0032] Generally speaking, interpretability is desirable, but not essential. Models that are more difficult to interpret than LASSO can also be used in this context. Such less interpretable models often give more accurate predictions, but are more difficult to debug. For example, it may be possible to use neural networks instead.
[0033] A time series is a sequence of numerical values obtained over a continuous period of time, typically at equal intervals. In some implementations, microblog counts are collected at equal time intervals, which may be referred to as segments. The size of a segment is a trade-off between precision and recall. Larger segments give the system more confidence in the signal, but the system will identify count spikes later.
[0034] Implementation examples of the system Figure 1 The data analytics engine 104 obtains microblog count time series data, such as multi-day history with overlapping 60-minute intervals, to produce 30-minute intervals, where each 30-minute interval includes the count of microblogs (e.g., tweets) within a 30-minute period. In other words, the recorded counts are 60-minute counts, but written at 30-minute intervals to develop a microblog count time series with 30-minute intervals.
[0035] Then, in one embodiment of the system, a predictable portion of the time series (provided by the prediction model described above) is extracted from the Weibo count time series to generate a residual time series. Therefore, the residual time series comprises residual Weibo counts for consecutive time intervals (e.g., in 30-minute segments). This embodiment of the system then determines a trigger threshold based on the residual time series.
[0036] In one embodiment, the trigger threshold is equal to median' + x' * IQR', where median' = median (residual), IQR' = quartile interval (residual), x' is an adjustment parameter, residual = [residual(-1), residual(-2), ..., residual(-K)], residual(-i) = numerator(-i) - prediction_numerator(-i), i is in [1, ..., K]; the number of previous segments (num buckets ago): i = 1 is the most recent segment, i = 2 is the second most recent segment. The number of segments can range, for example, from 12 to 192 half-hour segments. In other embodiments, the segment size can vary, for example, from 1 minute to 2 hours. In a further embodiment, the quartile interval can be replaced by different measures of the variability of microblog counts.
[0037] In some embodiments, the adjustment parameter x' is a constant. The adjustment parameter is set so that the system periodically triggers genuine events (but rarely spam, such as advertisements for budget hotels), and thus the system triggers close to the actual time of the event. Furthermore, the trigger can be a variety of actions, such as notifying the user in response to a query or including relevant microblog content in search results. In one embodiment, the system balances false positives (indicating an event is spiking on microblogs when such an event is not actually spiking) and false negatives (not indicating an event-related spiking is occurring on microblogs when such an event is actually spiking). If the system lowers the constant and thus the threshold, the system will trigger more aggressively (e.g., notifications or including microblog content in search results). The adjustment parameter can be set using human raters and historical data. Using a repository of historical data, a “replay time” with a given adjustment parameter can be used to see when the system triggers a given query, such as a notification. Then, based on an understanding of the actual timing and context of the event in question, it can be considered whether the adjustment parameter is causing the system to trigger too early or too late. A single tuning parameter can be used for hundreds or thousands of queries, and all resulting triggers are sent to a human evaluator. The human evaluator can point out inaccurate triggers and how they should be adjusted. In some embodiments, the constant is set lower for motion queries and higher for others.
[0038] One embodiment of the system includes Weibo posts in search results whenever the model tells it that the Weibo count is surging, and for an additional number of hours (e.g., 2 hours) after the last surge in Weibo count.
[0039] Figure 4 Two graphs are shown showing time-series data of event counts for events (e.g., social media data such as Twitter) that match a query (e.g., for a query of "New York City train shutdown"). The top graph shows a trigger threshold that is equal to (median + iqr_multiplier * iqr), where the median is the median count of tweets matching the specified query over a specified recent time period (e.g., the past few days), iqr is its quartile interval, and iqr_multiplier is a constant. The bottom graph uses... Figure 2 The residual method is shown. From Figure 4 It can be seen from this that Figure 2 The method provided earlier and more detection of the surge in Weibo counts associated with the query “New York City train shutdown”.
[0040] Figure 5Two graphs are shown illustrating the event count time series data for events matching a query (e.g., social media data such as Twitter), where the graphs reveal how the event count time series data changes when used with the query. Figure 2 The method avoids triggering slower increases. The chart above again shows the method using a trigger threshold equal to (median + iqr_multiplier * iqr). From Figure 5 As can be seen in the chart below, if the increase in the count is predictable, then Figure 2 The method may not trigger, for example, a notification to the user or inclusion of the Weibo content in search results, while the method used in the chart above will trigger under certain circumstances, even if the increase in Weibo count is predictable.
[0041] Once triggered (e.g., including Weibo content in search results), Figure 1 One embodiment of the relevance analysis engine 110 forwards data representing microblog content as part of the search results to the search engine frontend, which in turn forwards it to the user device. Figure 6 This is an example of a social media carousel embedded in a search results page as a result of operations in some embodiments.
[0042] Some embodiments of the present invention do not require a query to initiate the process of detecting a surge in near real-time content associated with a topic. As long as the topic of interest is obtained in some way, embodiments of the systems and methods described herein can be used to accurately notify a user of an event when content about that event is surging. This accurate notification can be reflected in application metrics, such as user engagement metrics.
[0043] The embodiments can also restrict the time series of Weibo counts to specific locations. Weibo users often maintain public archives that include their locations. Furthermore, the embodiments can use Weibo user locations and queries to identify relevant tags. For example, if an earthquake occurred in San Francisco and a user searches for San Francisco, the system can expand the retrieval of Weibo content to include content associated with relevant tags, such as #sfearthquake.
[0044] In addition to Weibo counts, near real-time event counts can also include various types of data, including social media counts and other publications, such as academic or news publications. Other types of near real-time data can be used in addition to or in place of Weibo data.
[0045] Embodiments of the subject matter and functional operation described in this specification may be implemented in digital electronic circuits, tangibly embodied computer software or firmware, computer hardware (including the structures disclosed in this specification and their structural equivalents), or in a combination of one or more of these. Embodiments of the subject matter described in this specification may be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a tangible, non-transitory storage medium for execution by or control of the operation of a data processing apparatus. The computer storage medium may be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or a combination of one or more of these. Alternatively or additionally, program instructions may be encoded on artificially generated propagation signals, such as machine-generated electrical, optical, or electromagnetic signals, which are generated to encode information for transmission to a suitable receiver device for execution by the data processing apparatus.
[0046] The term "data processing apparatus" refers to data processing hardware and encompasses a variety of devices, apparatuses, and machines for processing data, including, for example, programmable processors, computers, or multiple processors or computers. The apparatus may also be or further include special-purpose logic circuitry, such as FPGAs (field programmable gate arrays) or ASICs (application-specific integrated circuits). In addition to hardware, the apparatus may optionally include code that creates an execution environment for computer programs, such as code constituting processor firmware, protocol stacks, database management systems, operating systems, or combinations thereof.
[0047] A computer program can also be referred to or described as a program, software, software application, application, module, software module, script, or code. It can be written in any form of programming language, including compiled or interpreted languages, or declarative or procedural languages; and it can be deployed in any form, including as a standalone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A program may, but does not necessarily, correspond to a file in a file system. A program may be stored as part of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), as a single file dedicated to the program in question, or as multiple coordinating files (e.g., a file storing one or more modules, subroutines, or portions of code). A computer program can execute on one computer or on multiple computers deployed to be located at one site or distributed across multiple sites and interconnected via a data communication network.
[0048] The processes and logic flows described in this specification can be executed by one or more programmable computers that execute one or more computer programs to perform functions by manipulating input data and generating output. The processes and logic flows can also be executed by special-purpose logic circuitry (e.g., FPGA or ASIC), or by a combination of special-purpose logic circuitry and one or more programmable computers.
[0049] A computer suitable for executing computer programs can be based on a general-purpose or special-purpose microprocessor, or both, or any other type of central processing unit. Typically, the central processing unit receives instructions and data from read-only memory or random access memory, or both. The basic components of a computer are the central processing unit for executing or running instructions and one or more memory devices for storing instructions and data. The central processing unit and memory may be supplemented by or incorporated into special-purpose logic circuitry. Typically, a computer will also include one or more mass storage devices for storing data, such as disks, magneto-optical disks, or optical disks, or operatively coupled to receive data from or transfer data to one or more mass storage devices, or both. However, a computer does not require such devices. Furthermore, a computer can be embedded in another device, such as a mobile phone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a Global Positioning System (GPS) receiver, or a portable storage device, such as a universal serial bus (USB) flash drive, to name a few.
[0050] Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, such as semiconductor memory devices (e.g., EPROM (Erasable Programmable Read Only Memory), EEPROM (Electrically Erasable Programmable Read Only Memory), and flash memory devices); magnetic disks (e.g., internal hard disks or removable hard disks); magneto-optical disks; and CD-ROMs and DVD-ROMs.
[0051] To provide interaction with the user, embodiments of the subject matter described in this specification can be implemented on a computer having a display device for displaying information to the user, such as a CRT (cathode ray tube) or LCD (liquid crystal display), and a keyboard and pointing device, such as a mouse or trackball, through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback, such as visual, auditory, or tactile feedback; and input from the user can be received in any form, including sound, speech, or tactile input. Additionally, the computer can interact with the user by sending and receiving documents to and from the device used by the user; for example, by sending a webpage to a web browser on the user's device in response to a request received from a web browser. Furthermore, the computer can interact with the user by sending text messages or other forms of messages to a personal device (e.g., a smartphone), running a messaging application, and receiving response messages from the user in return.
[0052] Embodiments of the subject matter described in this specification can be implemented in a computing system that includes back-end components (e.g., as a data server), or middleware components (e.g., an application server), or front-end components (e.g., a client computer, web browser, or application with a graphical user interface through which a user can interact with embodiments of the subject matter described in this specification), or any combination of one or more such back-end, middleware, or front-end components. The components of the system can be interconnected via digital data communication (e.g., a communication network) of any form or medium. Examples of communication networks include local area networks (LANs) and wide area networks (WANs), such as the Internet.
[0053] A computing system may include clients and servers. Clients and servers are typically geographically separated and interact via a communication network. The client-server relationship is established by computer programs running on their respective computers and having a client-server relationship with each other. In some embodiments, the server transmits data (e.g., a Hypertext Markup Language (HTML) page) to a user device, for purposes such as displaying data to a user interacting with the device acting as a client and receiving user input from that user. Data generated at the user device (e.g., the result of user interaction) can be received from the device at the server.
[0054] In this specification, the term "database" will be used broadly to refer to any collection of data: the data does not need to be structured in any particular way, or not at all, and it can be stored on storage devices in one or more locations. Therefore, for example, an index database may include multiple collections of data, each of which can be organized and accessed differently.
[0055] Similarly, in this specification, the term "engine" will be used broadly to refer to a software-based system or subsystem capable of performing one or more specific functions. Typically, an engine will be implemented as one or more software modules or components installed on one or more computers in one or more locations. In some cases, one or more computers will be dedicated to a particular engine; in other cases, multiple engines may be installed and run on the same one or more computers.
[0056] While this specification contains numerous specific implementation details, these should not be construed as limiting the scope of any invention or the scope that may be claimed, but rather as descriptions of features specific to particular embodiments of a particular invention. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments. Furthermore, although the foregoing features may be described as operating in certain combinations, and even initially claimed in this way, in some cases, one or more features from the claimed combination may be removed from that combination, and the claimed combination may refer to a sub-combination or a variation of the sub-combination.
[0057] Similarly, although operations are described in a specific order in the accompanying drawings, this should not be construed as requiring these operations to be performed in the specific order shown or sequentially, or requiring all of the shown operations to obtain the desired result. In some cases, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system modules and components in the above embodiments should not be construed as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated into a single software product or packaged into multiple software products.
[0058] Specific embodiments of the subject matter have been described. Other embodiments are within the scope of the appended claims. For example, the actions listed in the claims can be performed in a different order and still achieve the desired result. As an example, the processes described in the figures do not necessarily require the specific order or sequential order shown to obtain the desired result. In some cases, multitasking and parallel processing may be advantageous.
Claims
1. A system for identifying and characterizing a signal contained in a data stream, comprising: One or more computers and one or more storage devices storing instructions on the one or more storage devices, the instructions, when executed by the one or more computers, are operable to cause the one or more computers to perform operations including: Receive inquiries; obtaining a historical microblog count time series counting the number of microblogs associated with the received query within each of a plurality of equal time intervals in a past time period; providing the historical microblog count time series to a machine learning prediction model, the machine learning prediction model being trained on a set of historical microblog count time series to output a predicted number of microblogs associated with a given query in a future time interval for a given input microblog count time series; receiving from the machine learning prediction model a predicted number of microblogs associated with the received query in a future time interval in response to providing the historical microblog count time series to the machine learning prediction model; Determining a residual trigger threshold based on the predicted number of microblogs associated with the received query in a future time interval, wherein determining the residual trigger threshold comprises: Determine the actual count of microblogs for the microblog time series; determining a time series of residuals associated with the received query, each corresponding residual in the time series of residuals being a difference between a corresponding actual count in the actual counts of the microblog time series and a corresponding predicted number of microblogs in a future time interval; and Determining the residual trigger threshold based on the time series of the residual; and When the number of microblogs associated with the received query exceeds the residual trigger threshold, data representing microblog content is provided for display as part of search results for the query, wherein determining the residual trigger threshold is based at least in part on measures of a median of the predicted number and a variance of the predicted number.
2. The system according to claim 1, wherein: The operations also include not including the microblog content as part of the search results for the query for a specified time after the excess microblog count drops below the threshold.
3. The system according to claim 1, wherein: Weibo count is the number of tweets.
4. The system according to claim 1, wherein: The operations also include combining the user interaction with the provided microblog content when determining whether to provide additional microblog content as part of the search results for the query.
5. The system according to claim 1, wherein: The operations also include limiting the microblog count time series to microblogs from a specific location.
6. A computer-implemented method for identifying and characterizing a signal contained in a data stream, comprising: Receive inquiries; obtaining a historical microblog count time series counting the number of microblogs associated with the received query within each of a plurality of equal time intervals in a past time period; providing the historical microblog count time series to a machine learning prediction model, the machine learning prediction model being trained on a set of historical microblog count time series to output a predicted number of microblogs associated with a given query in a future time interval for a given input microblog count time series; receiving from the machine learning prediction model a predicted number of microblogs associated with the received query in a future time interval in response to providing the historical microblog count time series to the machine learning prediction model; Determining a residual trigger threshold based on the predicted number of microblogs associated with the received query in a future time interval, wherein determining the residual trigger threshold comprises: Determine the actual count of microblogs for the microblog time series; determining a time series of residuals associated with the received query, each corresponding residual in the time series of residuals being a difference between a corresponding actual count in the actual counts of the microblog time series and a corresponding predicted number of microblogs in a future time interval; and Determining the residual trigger threshold based on the time series of the residual; and When the number of microblogs associated with the received query exceeds the residual trigger threshold, data representing microblog content is provided for display as part of search results for the query, wherein determining the residual trigger threshold is based at least in part on measures of a median of the predicted number and a variance of the predicted number.
7. The method according to claim 6, further comprising: After the excess microblog count drops below the threshold, the microblog content is not included as part of the search results for the query for a specified time.
8. The method according to claim 6, wherein: Weibo count is the number of tweets.
9. The method according to claim 6, wherein: The past time period is between 1 and 7 days.
10. The method according to claim 6, wherein: The method also includes communicating to a user a confidence measure that the residual microblog count reflects an event for which the user should be notified, the confidence measure being based at least in part on the extent to which the residual microblog count exceeds the trigger threshold.
11. The method according to claim 6, wherein: The method also includes combining the user interaction with the provided microblog content when determining whether to provide additional microblog content as part of the search results for the query.
12. The method according to claim 6, wherein: The method further includes limiting the microblog count time series to microblogs from a specific location.
13. The method according to claim 6, further comprising: Determine the median value of the microblog count during the relevant time period; determining a variability measure of the variability of the microblog counts over a relevant time period; determining a second trigger threshold based at least in part on the median value and the measure of variability; as well as If the microblog count exceeds the residual trigger threshold or the second trigger threshold, a carousel is displayed.
