A hot word mining method and system based on big data

By using a big data-based hot word mining system, which utilizes category vector generation and cross-heat assessment, the problem of traffic limitation in existing hot word mining methods has been solved, achieving accurate evaluation and effective exposure of hot words.

CN117828152BActive Publication Date: 2024-11-29淮安科畅科技信息有限公司
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202311622391.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-11-30
Publication Date
2024-11-29
Estimated Expiration
2043-11-30

AI Technical Summary

Technical Problem

Existing hot word mining methods cannot effectively assess popularity, resulting in important content not being accessible to more users and causing traffic throttling.

Method used

By using a big data-based hot word mining system, category vector generation, cross-heat assessment, and Newton's law of cooling are employed to evaluate the popularity of terms and make cross-judgments, thereby obtaining comprehensive hot words.

Benefits of technology

It improves the accuracy and reliability of hot word mining, reduces the suppression of popularity, and ensures that important content is effectively exposed.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117828152B_ABST
    Figure CN117828152B_ABST
Patent Text Reader

Abstract

The present invention relates to the field related to Internet hot word mining, and discloses a hot word mining method and system based on big data, including a retrieval pointing division module, a term vector generation module, a pointing interval coordination module and a hot word mining evaluation module; by taking the approach of classifying the subject content on the Internet, the term heat is evaluated under different categories, and the exposure of the terms between categories is cross-judged, so as to comprehensively consider and select more accurate hot terms. Compared with the hot word mining method in the prior art, the cross-heat judgment can effectively reduce the heat suppression behavior under a certain topic content, and the result is more reliable.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field related to Internet hot word mining, and in particular to a hot word mining method and system based on big data. Background Art

[0002] In the era of Internet information explosion, it is very important to promote important data from massive data to user groups. Therefore, hot word mining undertakes the important task of obtaining effective promotional content through massive data.

[0003] Hot word mining in the prior art is mostly obtained by statistically ranking comments on the entries within a certain period of time. However, in actual use, there is a situation where the flow of corresponding content-related topics is limited, resulting in the inability to effectively conduct hot word mining and evaluation. Therefore, some important content cannot be effectively promoted and obtained by more users. Summary of the invention

[0004] The purpose of the present invention is to provide a hot word mining method and system based on big data to solve the problems raised in the above background technology.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] A hot word mining system based on big data, including:

[0007] A search direction division module is used to establish a category vector based on the title entries of the online index content, and the category vector is used to represent the digital search features of the corresponding subject category of the online index content;

[0008] A term vector generation module, used to obtain the main content of the online index content and the word and sentence content of the associated terms corresponding to multiple category vectors within a preset deviation value, perform low-dimensional space mapping based on the words of the word and sentence content, establish content vectors and perform vector statistics, and obtain statistical data within the group;

[0009] The pointing interval collaboration module is used to cross-judge the statistical data in multiple groups, obtain the occurrence frequency of a certain content vector in other groups, and generate the cross heat of the corresponding content vector, wherein the cross heat represents the occurrence ratio of the word content in multiple groups divided based on the category vector;

[0010] The hot word mining and evaluation module is used to perform a comprehensive heat evaluation on the content vector based on statistical data and cross heat, and to obtain a plurality of hot word objects based on the comprehensive heat evaluation ranking of multiple content vectors.

[0011] As a further solution of the present invention: the term vector generation module includes:

[0012] A vector deviation division unit, used to obtain a plurality of category vectors, spatially represent the plurality of category vectors, and perform convergence and divergence analysis on the plurality of category vectors by a preset deviation value to divide the plurality of category vectors into groups, wherein the deviation value is used to represent the fluctuation range of the spatial characteristics of the corresponding category vector when the language meanings of the word and sentence contents are similar or identical;

[0013] A word vector generating unit, used for establishing a co-occurrence matrix of several words and sentences in a group based on the Harris distribution hypothesis, and multiplying the co-occurrence matrix by a preset projection vector so that the co-occurrence matrix is ​​mapped into a content vector in a low-dimensional space;

[0014] The word vector statistics unit is used to perform vector statistics on several content vectors in a group, and correspondingly obtain statistical data within the group including the number of occurrences of several content vectors, wherein the vector statistics are implemented based on a preset deviation value.

[0015] As a further solution of the present invention: the pointing interval cooperation module includes:

[0016] A cross-search unit, used to perform cross-judgment on the content vectors in the group and other groups to obtain the existence of the content vectors in other groups, wherein the cross-judgment is used to compare the content vectors in other groups with the content vectors to be matched based on the deviation value to determine whether they are used to represent the same or similar words and sentences;

[0017] The cross statistics unit is used to count the occurrence frequency of the content vector in other multiple groups and the number of occurrences in different groups to obtain the cross heat of the content vector.

[0018] As a further solution of the present invention: the hot word mining evaluation module includes:

[0019] A cross-value assignment unit, configured to obtain the cross-heat of the plurality of content vectors, and to perform cross-value assignment based on the frequency of occurrence, wherein the cross-value assignment is performed based on a preset value assignment mapping relationship, wherein the value assignment mapping relationship includes a level division of the total number of groups and a cross-value corresponding to the frequency of occurrence at a corresponding level, and when the level division is determined, the cross-value is proportional to the frequency of occurrence;

[0020] The vector evaluation unit is used to perform comprehensive heat calculation based on the statistical data of the content vector and its corresponding cross coefficient and number of occurrences, and sort multiple content vectors based on the calculation results to obtain several hot word objects, where the hot word objects are the words and sentences corresponding to the first preset number of content vectors in the sorting.

[0021] As a further solution of the present invention: it also includes a hot word cooling module;

[0022] The hot word cooling module is used to calculate the cooling coefficient of the content vector by Newton's cooling law. The cooling coefficient acts on the comprehensive heat of the content vector. The cooling coefficient is expressed as ,in Indicates the current comprehensive heat. Indicates the historical comprehensive heat, Indicates the time difference.

[0023] The embodiment of the present invention aims to provide a hot word mining method based on big data, comprising the steps of:

[0024] Establishing a category vector based on the title entries of the online index content, wherein the category vector is used to represent the digital retrieval features of the subject category corresponding to the online index content;

[0025] Obtain the main content of the online index content and the word and sentence content of the associated entries corresponding to multiple category vectors within the preset deviation value, perform low-dimensional space mapping based on the words of the word and sentence content, establish content vectors and perform vector statistics, and obtain statistical data within the group;

[0026] Perform cross-judgment on the statistical data in multiple groups to obtain the occurrence frequency of a certain content vector in other groups to generate the cross heat of the corresponding content vector, wherein the cross heat represents the occurrence ratio of the word or sentence content in multiple groups divided based on the category vector;

[0027] Based on the statistical data and the cross-heat, the content vector is comprehensively evaluated for heat, and based on the comprehensive heat evaluation sorting of multiple content vectors, a plurality of hot word objects are obtained.

[0028] As a further solution of the present invention, the steps of obtaining the main content of the online index content and the sentence content of the associated entries corresponding to the multiple category vectors within the preset deviation value, performing low-dimensional space mapping based on the words of the sentence content, establishing content vectors and performing vector statistics, and obtaining statistical data within the group specifically include:

[0029] Acquire multiple category vectors, spatially represent the multiple category vectors, and perform convergence and divergence analysis on the multiple category vectors by using a preset deviation value to divide the multiple category vectors into groups, wherein the deviation value is used to represent the fluctuation range of the spatial characteristics of the corresponding category vectors when the language meanings of the words and sentences are similar or the same;

[0030] Establishing a co-occurrence matrix of several words and sentences in the group based on the Harris distribution hypothesis, and multiplying the co-occurrence matrix by a preset projection vector, so that the co-occurrence matrix is ​​mapped into a content vector in a low-dimensional space;

[0031] Vector statistics are performed on a plurality of content vectors within a group, and correspondingly, statistical data within the group including the number of occurrences of the plurality of content vectors is obtained, wherein the vector statistics are implemented based on a preset deviation value.

[0032] As a further solution of the present invention, the step of cross-judging the statistical data in multiple groups to obtain the occurrence frequency of a certain content vector in other groups to generate the cross-heat of the corresponding content vector specifically includes:

[0033] Cross-judging the content vectors in the group with other groups to obtain the existence of the content vectors in other groups, wherein the cross-judgment is used to compare the content vectors in other groups with the content vectors to be matched based on the deviation value to determine whether they are used to represent the same or similar words and sentences;

[0034] The occurrence frequencies of the content vector in other multiple groups and the number of occurrences in different groups are counted to obtain the cross-heat of the content vector.

[0035] As a further solution of the present invention: the step of performing comprehensive heat evaluation on the content vector based on statistical data and cross heat, and sorting the comprehensive heat evaluation of multiple content vectors to obtain a plurality of hot word objects specifically includes:

[0036] Obtaining the cross heat of the plurality of content vectors, and assigning a cross coefficient based on the frequency of occurrence, wherein the cross coefficient assignment is performed based on a preset assignment mapping relationship, wherein the assignment mapping relationship includes a level division of the total number of groups and a cross coefficient corresponding to the frequency of occurrence at a corresponding level, and when the level division is determined, the cross coefficient is proportional to the frequency of occurrence;

[0037] A comprehensive heat calculation is performed based on the statistical data of the content vectors and their corresponding cross coefficients and occurrence times, and multiple content vectors are sorted based on the calculation results to obtain several hot word objects, where the hot word objects are the words and sentences corresponding to the first preset number of content vectors in the sorting.

[0038] As a further solution of the present invention: it also includes the steps of:

[0039] The cooling coefficient of the content vector is calculated by Newton's cooling law. The cooling coefficient acts on the comprehensive heat of the content vector. The cooling coefficient is expressed as ,in Indicates the current comprehensive heat. Indicates the historical comprehensive heat, Indicates the time difference.

[0040] Compared with the prior art, the beneficial effect of the present invention is: by classifying the subject content on the Internet, the popularity of entries is evaluated under different categories, and the exposure of the entries between categories is cross-judged, so as to comprehensively consider and select more accurate hot entries. Compared with the hot word mining method in the prior art, the cross-heat judgment can effectively reduce the heat suppression behavior under a certain topic content, and the result is more reliable. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 This is a block diagram of a hot word mining system based on big data.

[0042] Figure 2 This is a block diagram of the composition of the vector generation module in a hot word mining system based on big data.

[0043] Figure 3 This is a block diagram of the composition of the interval-pointing collaboration module in a hot word mining system based on big data.

[0044] Figure 4 This is a block diagram of the composition of a hot word mining evaluation module in a hot word mining system based on big data.

[0045] Figure 5 This is a flowchart of a hot word mining method based on big data. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0047] The specific implementation of the present invention is described in detail below in conjunction with specific embodiments.

[0048] like Figure 1 The hot word mining system based on big data provided by an embodiment of the present invention comprises the following steps:

[0049] The search direction division module 100 is used to establish a category vector based on the title entries of the online index content, and the category vector is used to represent the digital search features of the corresponding subject category of the online index content.

[0050] The term vector generation module 300 is used to obtain the main content of the online index content and the sentence content of the associated terms corresponding to multiple category vectors within a preset deviation value, perform low-dimensional space mapping based on the words of the sentence content, establish content vectors and perform vector statistics, and obtain statistical data within the group.

[0051] The pointing interval collaboration module 500 is used to cross-judge the statistical data within multiple groups, obtain the occurrence frequency of a certain content vector in other groups, and generate the cross-heat of the corresponding content vector. The cross-heat represents the occurrence ratio of the word and sentence content in multiple groups divided based on the category vector.

[0052] The hot word mining and evaluation module 700 is used to perform a comprehensive heat evaluation on the content vector based on statistical data and cross heat, and to obtain a plurality of hot word objects based on the comprehensive heat evaluation ranking of multiple content vectors.

[0053] In this embodiment, a hot word mining system based on big data is provided. By classifying the subject content on the Internet, the popularity of entries is evaluated under different categories, and the exposure of entries between categories is cross-judged, so as to comprehensively consider and select more accurate hot entries. Compared with the hot word mining method in the prior art, the heat suppression behavior under a certain topic content can be effectively reduced through cross-heat judgment, and the result is more reliable. In actual hot word mining, there will be a situation where the heat is suppressed under some topics. Even if the related discussion heat of a related topic is extremely high, the topic still cannot be effectively mined by the algorithm to become a hot word. Therefore, in order to effectively mine some hot topic content, this application adopts a cross-heat judgment method. Specifically, in actual scenarios, when a topic has a discussion degree, especially when a topic has a high discussion degree but is subject to heat restrictions, Relevant online users who have a certain degree of knowledge about the topic may expose the content in other content venues, and the hot word mining method in the prior art is based on the retrieval frequency of the overall online vocabulary. Therefore, this type of relatively scattered and small-scale exposure method cannot effectively increase the popularity of the relevant terms themselves. Therefore, this method cannot make the terms with suppressed popularity become hot words again. The present application first divides the online content based on relevance, and extracts hot words from the divided groups separately. Therefore, even if the relevant content under the topic is suppressed, the corresponding terms in other groups that are continuously and sporadically exposed by online users will be upgraded in judgment priority as the number of sporadic distribution points increases, that is, cross-heat. Therefore, the cross-heat can be improved by distributed hot word exposure, which can effectively affect the conventional hot word mining part to improve the relevant heat of the terms and make them hot words again.

[0054] like Figure 2 As shown, as another preferred embodiment of the present invention, the term vector generation module 300 includes:

[0055] The vector deviation division unit 301 is used to obtain multiple category vectors, spatially represent the multiple category vectors, and perform convergence and divergence analysis on the multiple category vectors through a preset deviation value to divide the multiple category vectors into groups. The deviation value is used to characterize the fluctuation range of the spatial characteristics of the corresponding category vector when the linguistic meaning of the word content is similar or the same.

[0056] The word vector generating unit 302 is used to establish a co-occurrence matrix of several words and sentences in a group based on the Harris distribution hypothesis, and multiply the co-occurrence matrix by a preset projection vector so that the co-occurrence matrix is ​​mapped into a content vector in a low-dimensional space.

[0057] The word vector statistics unit 303 is used to perform vector statistics on a plurality of content vectors in a group, and correspondingly obtain statistical data within the group including the number of occurrences of the plurality of content vectors, wherein the vector statistics are implemented based on a preset deviation value.

[0058] In this embodiment, the entry vector generation module 300 is explained, wherein the step of generating content vectors for entry words and sentences adopts the Harris distribution hypothesis (wherein the step of generating category vectors by the retrieval pointing division module 100 is the same). It can be simply understood as judging the similarity of words based on the similarity matching of the context of a certain word. Compared with the traditional method of judging the linguistic meaning of words and sentences, this method can effectively avoid the wrong judgment when the words and sentences have different interpretations in different scenarios. Specifically, based on the context of the words, a co-occurrence matrix can be established, wherein the co-occurrence matrix can be expressed as P*C, wherein P is the dictionary, C is the adjacent words in the lexical context, etc., and the projection vector is a determined vector used to project the co-occurrence matrix in a plane to obtain a plane vector that is easier to judge. After the processing is completed and multiple content vectors are generated, the heat of different content vectors is evaluated through statistics to judge the heat.

[0059] like Figure 3 As shown, as another preferred embodiment of the present invention, the pointing interval coordination module 500 includes:

[0060] The cross-retrieval unit 501 is used to cross-judge the content vectors in the group with other groups to obtain the existence of the content vectors in other groups. The cross-judgment is used to compare the content vectors in other groups with the matching content vectors based on the deviation value to determine whether they are used to represent the same or similar word and sentence content.

[0061] The cross statistics unit 502 is used to count the occurrence frequency of the content vector in other multiple groups and the number of occurrences in different groups to obtain the cross heat of the content vector.

[0062] In this embodiment, the pointing interval collaboration module 500 is used for cross-heat evaluation between groups, which mainly includes two steps. The first is cross-judgment between groups, and the existence of vectors between multiple groups is retrieved and obtained; the second is to perform statistical calculations on the vectors between groups, including the existence under different topics, that is, the probability of occurrence, and the number of occurrences in different groups, among which the frequency of occurrence is the most important parameter. For example, excluding the original groups online, there are a total of ten topic groups, among which this topic exists in 5 groups, and 0.5 is the frequency of occurrence.

[0063] like Figure 4 As shown, as another preferred embodiment of the present invention, the hot word mining evaluation module 700 includes:

[0064] The cross-assignment unit 701 is used to obtain the cross-heat of multiple content vectors and assign cross-coefficients based on the frequency of occurrence. The cross-coefficient assignment is performed based on a preset assignment mapping relationship, and the assignment mapping relationship includes the level division of the total number of groups and the cross-coefficient corresponding to the frequency of occurrence at the corresponding level. When the level division is determined, the cross-coefficient is proportional to the frequency of occurrence.

[0065] The vector evaluation unit 702 is used to perform comprehensive heat calculation based on the statistical data of the content vector and its corresponding cross coefficient and number of occurrences, and sort multiple content vectors based on the calculation results to obtain several hot word objects, where the hot word objects are the words and sentences corresponding to the first preset number of content vectors in the sorting.

[0066] In this embodiment, the function of the hot word mining evaluation module 700 is to make a collaborative judgment on the cross-heat and the basic heat within the group to obtain the comprehensive heat. In the process of generating the comprehensive heat, it is necessary to allocate the calculation accumulation coefficient between the heat within the group and the cross-heat. In the actual scenario, when a topic is mentioned in more groups other than the original group, it means that the topic may be more important. Therefore, it is necessary to allocate a higher cross-coefficient to it to calculate the comprehensive heat, so as to further improve its overall heat and make its corresponding words and sentences become hot words.

[0067] As another preferred embodiment of the present invention, it also includes a hot word cooling module;

[0068] The hot word cooling module is used to calculate the cooling coefficient of the content vector by Newton's cooling law. The cooling coefficient acts on the comprehensive heat of the content vector. The cooling coefficient is expressed as ,in Indicates the current comprehensive heat. Indicates the historical comprehensive heat, Indicates the time difference.

[0069] In this embodiment, the popularity of a certain hot word gradually decreases with the change of time, so there is a speed of heat cooling, that is, the cooling coefficient. When the cooling efficiency is greater (the faster the heat decays within a period of time), it means that the actual importance or attention of the topic is not high, and people will quickly abandon it after understanding it. Therefore, even if the current heat base is still very high, it is necessary to reduce its heat through the cooling coefficient to provide a window for other content that needs exposure more.

[0070] like Figure 5 As shown, the present invention also provides a hot word mining method based on big data, which comprises the steps of:

[0071] S200, establishing a category vector based on the title entries of the online index content, wherein the category vector is used to represent digital retrieval features of the subject category corresponding to the online index content.

[0072] S400, obtaining the main content of the online index content and the sentence content of the associated entries corresponding to multiple category vectors within a preset deviation value, performing low-dimensional space mapping based on the words of the sentence content, establishing a content vector and performing vector statistics, and obtaining statistical data within the group.

[0073] S600, cross-judging the statistical data in multiple groups, obtaining the occurrence frequency of a certain content vector in other groups, so as to generate the cross heat of the corresponding content vector, wherein the cross heat represents the occurrence ratio of the word or sentence content in multiple groups divided based on the category vector.

[0074] S800, performing comprehensive heat evaluation on the content vector based on statistical data and cross heat, and sorting the comprehensive heat evaluations of multiple content vectors to obtain a plurality of hot word objects.

[0075] As another preferred embodiment of the present invention, the step of obtaining the main content of the online index content and the sentence content of the associated terms corresponding to multiple category vectors within the preset deviation value, performing low-dimensional space mapping based on the words of the sentence content, establishing content vectors and performing vector statistics, and obtaining statistical data within the group specifically includes:

[0076] A plurality of category vectors are obtained, the plurality of category vectors are spatially represented, and a convergence and divergence analysis is performed on the plurality of category vectors through a preset deviation value so as to divide the plurality of category vectors into groups, wherein the deviation value is used to characterize the fluctuation range of the spatial characteristics of the corresponding category vector when the linguistic meaning of the content of the sentence is similar or the same.

[0077] A co-occurrence matrix of several words and sentences in a group is established based on the Harris distribution hypothesis, and the co-occurrence matrix is ​​multiplied by a preset projection vector, so that the co-occurrence matrix is ​​mapped into a content vector in a low-dimensional space.

[0078] Vector statistics are performed on a plurality of content vectors within a group, and correspondingly, statistical data within the group including the number of occurrences of the plurality of content vectors is obtained, wherein the vector statistics are implemented based on a preset deviation value.

[0079] As another preferred embodiment of the present invention, the step of cross-judging the statistical data in multiple groups to obtain the occurrence frequency of a certain content vector in other groups to generate the cross-heat of the corresponding content vector specifically includes:

[0080] The content vectors in the group are cross-judged with other groups to obtain the existence of the content vectors in other groups. The cross-judgment is used to compare the content vectors in other groups with the matching content vectors based on the deviation value to determine whether they are used to represent the same or similar word and sentence content.

[0081] The occurrence frequencies of the content vector in other multiple groups and the number of occurrences in different groups are counted to obtain the cross-heat of the content vector.

[0082] As another preferred embodiment of the present invention, the step of performing comprehensive heat evaluation on the content vector based on statistical data and cross heat, and sorting the comprehensive heat evaluation of multiple content vectors to obtain several hot word objects specifically includes:

[0083] The cross-heat of the plurality of content vectors is obtained, and a cross-coefficient is assigned based on the frequency of occurrence. The cross-coefficient assignment is performed based on a preset assignment mapping relationship, and the assignment mapping relationship includes a level division of the total number of groups and a cross-coefficient corresponding to the frequency of occurrence at the corresponding level. When the level division is determined, the cross-coefficient is proportional to the frequency of occurrence.

[0084] A comprehensive heat calculation is performed based on the statistical data of the content vectors and their corresponding cross coefficients and occurrence times, and multiple content vectors are sorted based on the calculation results to obtain several hot word objects, where the hot word objects are the words and sentences corresponding to the first preset number of content vectors in the sorting.

[0085] As another preferred embodiment of the present invention, the steps are also included:

[0086] The cooling coefficient of the content vector is calculated by Newton's cooling law. The cooling coefficient acts on the comprehensive heat of the content vector. The cooling coefficient is expressed as ,in Indicates the current comprehensive heat. Indicates the historical comprehensive heat, Indicates the time difference.

[0087] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0088] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the disclosure in the specification and examples. This application is intended to cover any variations, uses or adaptations of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims.

[0089] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A hot word mining system based on big data, characterized in that: Include: A search direction division module is used to establish a category vector based on the title entries of the online index content, and the category vector is used to represent the digital search features of the corresponding subject category of the online index content; A term vector generation module, used to obtain the main content of the online index content and the word and sentence content of the associated terms corresponding to multiple category vectors within a preset deviation value, perform low-dimensional space mapping based on the words of the word and sentence content, establish content vectors and perform vector statistics, and obtain statistical data within the group; The pointing interval collaboration module is used to cross-judge the statistical data in multiple groups, obtain the occurrence frequency of a certain content vector in other groups, and generate the cross heat of the corresponding content vector, wherein the cross heat represents the occurrence ratio of the word content in multiple groups divided based on the category vector; A hot word mining and evaluation module is used to perform a comprehensive heat evaluation on the content vector based on statistical data and cross heat, and to obtain a plurality of hot word objects based on the comprehensive heat evaluation ranking of multiple content vectors; The term vector generation module includes: A vector deviation division unit, used to obtain a plurality of category vectors, spatially represent the plurality of category vectors, and perform convergence and divergence analysis on the plurality of category vectors by a preset deviation value to divide the plurality of category vectors into groups, wherein the deviation value is used to represent the fluctuation range of the spatial characteristics of the corresponding category vector when the language meanings of the word and sentence contents are similar or identical; A word vector generating unit, used for establishing a co-occurrence matrix of several words and sentences in a group based on the Harris distribution hypothesis, and multiplying the co-occurrence matrix by a preset projection vector so that the co-occurrence matrix is ​​mapped into a content vector in a low-dimensional space; A word vector statistics unit, used to perform vector statistics on a plurality of content vectors in a group, and correspondingly obtain statistical data within the group including the number of occurrences of the plurality of content vectors, wherein the vector statistics are implemented based on a preset deviation value; The pointing interval coordination module includes: A cross-search unit, used to perform cross-judgment on the content vectors in the group and other groups to obtain the existence of the content vectors in other groups, wherein the cross-judgment is used to compare the content vectors in other groups with the content vectors to be matched based on the deviation value to determine whether they are used to represent the same or similar words and sentences; A cross-statistics unit, used to count the total number of times the content vector appears in other multiple groups and the number of times it appears in different groups, so as to obtain the cross-heat of the content vector, that is, the probability of the content vector appearing in other topics; The hot word mining evaluation module includes: A cross-value assignment unit, configured to obtain the cross-heat of the plurality of content vectors, and to perform cross-value assignment based on the frequency of occurrence, wherein the cross-value assignment is performed based on a preset value assignment mapping relationship, wherein the value assignment mapping relationship includes a level division of the total number of groups and a cross-value corresponding to the frequency of occurrence at a corresponding level, and when the level division is determined, the cross-value is proportional to the frequency of occurrence; The vector evaluation unit is used to perform comprehensive heat calculation based on the statistical data of the content vector and its corresponding cross coefficient and number of occurrences, and sort multiple content vectors based on the calculation results to obtain several hot word objects, where the hot word objects are the words and sentences corresponding to the first preset number of content vectors in the sorting.

2. A hot word mining system based on big data according to claim 1, characterized in that: Also includes hot word cooling module; The hot word cooling module is used to calculate the cooling coefficient of the content vector by Newton's cooling law. The cooling coefficient acts on the comprehensive heat of the content vector. The cooling coefficient is expressed as in Indicates the current comprehensive heat. represents the historical comprehensive heat, and Δt represents the time difference.

3. A hot word mining method based on big data, characterized in that: Contains steps: Establishing a category vector based on the title entries of the online index content, wherein the category vector is used to represent the digital retrieval features of the subject category corresponding to the online index content; Obtain the main content of the online index content and the word and sentence content of the associated entries corresponding to multiple category vectors within the preset deviation value, perform low-dimensional space mapping based on the words of the word and sentence content, establish content vectors and perform vector statistics, and obtain statistical data within the group; Perform cross-judgment on the statistical data in multiple groups to obtain the occurrence frequency of a certain content vector in other groups to generate the cross heat of the corresponding content vector, wherein the cross heat represents the occurrence ratio of the word or sentence content in multiple groups divided based on the category vector; Performing a comprehensive heat evaluation on the content vector based on statistical data and cross heat, and sorting the comprehensive heat evaluations of multiple content vectors to obtain a plurality of hot word objects; The steps of obtaining the main content of the online index content and the word and sentence content of the associated entries corresponding to the multiple category vectors within the preset deviation value, performing low-dimensional space mapping based on the words of the word and sentence content, establishing content vectors and performing vector statistics, and obtaining statistical data within the group specifically include: Acquire multiple category vectors, spatially represent the multiple category vectors, and perform convergence and divergence analysis on the multiple category vectors by using a preset deviation value to divide the multiple category vectors into groups, wherein the deviation value is used to represent the fluctuation range of the spatial characteristics of the corresponding category vectors when the language meanings of the words and sentences are similar or the same; Establishing a co-occurrence matrix of several words and sentences in the group based on the Harris distribution hypothesis, and multiplying the co-occurrence matrix by a preset projection vector, so that the co-occurrence matrix is ​​mapped into a content vector in a low-dimensional space; Performing vector statistics on a plurality of content vectors within a group, and correspondingly obtaining statistical data within the group including the number of occurrences of the plurality of content vectors, wherein the vector statistics is implemented based on a preset deviation value; The step of cross-judging the statistical data in multiple groups to obtain the occurrence frequency of a certain content vector in other groups to generate the cross-heat of the corresponding content vector specifically includes: Cross-judging the content vectors in the group with other groups to obtain the existence of the content vectors in other groups, wherein the cross-judgment is used to compare the content vectors in other groups with the content vectors to be matched based on the deviation value to determine whether they are used to represent the same or similar words and sentences; Counting the total number of times the content vector appears in other multiple groups and the number of times the content vector appears in different groups to obtain the cross-heat of the content vector, that is, the probability of the content vector appearing in other topics; The step of performing comprehensive heat evaluation on the content vector based on statistical data and cross heat, and obtaining a plurality of hot word objects based on comprehensive heat evaluation sorting of multiple content vectors specifically includes: Obtaining the cross heat of the plurality of content vectors, and assigning a cross coefficient based on the frequency of occurrence, wherein the cross coefficient assignment is performed based on a preset assignment mapping relationship, wherein the assignment mapping relationship includes a level division of the total number of groups and a cross coefficient corresponding to the frequency of occurrence at a corresponding level, and when the level division is determined, the cross coefficient is proportional to the frequency of occurrence; A comprehensive heat calculation is performed based on the statistical data of the content vectors and their corresponding cross coefficients and occurrence times, and multiple content vectors are sorted based on the calculation results to obtain several hot word objects, where the hot word objects are the words and sentences corresponding to the first preset number of content vectors in the sorting.

4. The method for mining hot words based on big data according to claim 3, characterized in that: Also includes the steps: The cooling coefficient of the content vector is calculated by Newton's cooling law. The cooling coefficient acts on the comprehensive heat of the content vector. The cooling coefficient is expressed as in Indicates the current comprehensive heat. represents the historical comprehensive heat, and Δt represents the time difference.

Citation Information

Patent Citations

  • Hot word discovery method and system

    CN112667814A

  • Hot topic mining method and device, storage medium and equipment

    CN116738068A