Method, device and equipment for acquiring detail content of event title and medium

Abstract

The invention provides an event title detail content acquisition method and device, equipment and a medium, and relates to the field of large language models. The method comprises the steps of inputting a first prompt text into a large language model, and judging whether an event title belongs to a user attention range or not through the large language model; performing integrity evaluation on a first event title which is output by the large language model and belongs to a user attention range to obtain a target event title which passes the integrity evaluation; calling a search engine through the large language model, and retrieving detail contents of the target event title; and inputting the second prompt text into a large language model, sorting the detail content of the target event title according to a specified format through the large language model, and obtaining the sorted detail content of the target event title, so as to introduce the large language model in each stage of event collection and content recording. The large language model assists human in event judgment, analysis and arrangement, the accuracy of data collection is improved, and the problem that the difference of event records is large is solved.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of large language models, and in particular to a method, device, equipment and medium for obtaining detailed content of an event title. Background Art

[0002] Currently, event monitoring refers to: integrating Internet information collection technology and information intelligent processing technology to automatically capture, classify and cluster massive amounts of Internet information, detect topics, and focus on topics, so as to meet users' information needs such as network event monitoring and news topic tracking, and form analysis results such as briefings, reports, and charts.

[0003] Event monitoring systems generally need to solve the following problems: 1. Event capture: based on the user's areas of concern, realize real-time discovery of events such as text, pictures, and videos on the entire network; 2. Event details: inform the user of the event details; 3. Event analysis: identify event levels, sentiment classifications, event labels, and event classifications; 4. Event reporting: send reports regularly or in real time.

[0004] To achieve the above functions, traditional event monitoring systems need to implement the following technologies: 1. Event collection through crawlers or APIs combined with keywords and traditional search technologies; 2. Crawling and parsing event details through crawler technology; 3. Through traditional big data technologies, such as PaddlePaddle and TensorFlow, the corpus is collected in advance through manual sorting, classification, labeling, and grading, and various event analysis models are established through machine learning to achieve intelligent analysis and sorting of events.

[0005] However, in the event capture and event detail recording stage, the following problems exist: 1. When crawling events in a specified field, it is necessary to manually specify keywords in advance and collect events according to keywords. The quality of collected data in this way directly depends on the quality of keyword definitions with manual participation. When collecting, traditional matching technologies such as word segmentation, keyword matching, and synonym matching may miss some events that users are concerned about. On the other hand, various new keywords will appear in the development of various events, and it is difficult to maintain and track them in a timely manner through manual specification. 2. At present, the specific description of the event on the website where the event is located is generally crawled through traditional crawler technology. The result is that the description details, format, style, and focus of the event details collected at each event collection point are inconsistent, making it difficult for users to have a consistent reading and understanding experience of the event details. For example, the content corresponding to a certain Weibo hot search is a specific Weibo, and the news articles obtained through the Black Cat Complaint Platform are news articles, while the hot content of WeChat public accounts is long articles of public accounts, which are quite different. Summary of the invention

[0006] Embodiments of the present invention provide a method, apparatus, device and medium for obtaining detailed content of an event title to overcome the above problems or at least partially solve the above problems.

[0007] A first aspect of an embodiment of the present invention provides a method for obtaining detailed content of an event title, which is applied to a temporary networking configuration server, and the method includes:

[0008] Inputting a first prompt text into a large language model, and determining whether the event title belongs to a user's attention range through the large language model; the first prompt text includes: at least one user's attention range, a determination step, a first example, and a first output format;

[0009] Performing a completeness evaluation on the first event title that is output by the large language model and belongs to the user's attention scope, to obtain a target event title that passes the completeness evaluation;

[0010] Invoking a search engine through the large language model to retrieve detailed content of the target event title;

[0011] The second prompt text is input into the large language model, and the details of the target event title are sorted according to the specified format by the large language model to obtain the sorted details of the target event title; the second prompt text includes: original content, sorting steps, a second example and a second output format.

[0012] Optionally, performing completeness evaluation on the first event title output by the large language model and belonging to the user's attention scope to obtain a target event title that passes the completeness evaluation includes:

[0013] Performing a completeness evaluation on the first event title from multiple dimensions of data completeness, data timeliness, data accuracy, data consistency, data uniqueness, and data validity to obtain a target event title that passes the completeness evaluation;

[0014] Among them, the data integrity represents the ratio of the first event title with complete content to all the first event titles; the data timeliness represents the ratio of the first event title processed within the specified time to all the first event titles; the data accuracy represents the ratio of the first event title with accurate collection to all the first event titles; the data consistency represents the ratio of the first event title with consistent data structure to all the first event titles; the data uniqueness represents the ratio of the first event title with unique data source to all the first event titles; the data validity represents the ratio of the first event title that meets the constraint range to all the first event titles.

[0015] Optionally, calling a search engine through the large language model to retrieve detailed content of the target event title includes:

[0016] Calling multiple search engines through the large language model to retrieve detailed content of the target event title;

[0017] The large language model is used to call a summary module to summarize the details of the multiple target event titles retrieved by the multiple search engines until the preset content arrangement requirements are met.

[0018] Optionally, the aggregating the detailed contents of the multiple target event titles retrieved by the multiple search engines includes:

[0019] Based on the user interaction amounts corresponding to the detailed contents of the multiple target event titles, the detailed contents of the multiple target event titles are sorted, and the detailed contents of the target event titles corresponding to the top K user interaction amounts are summarized;

[0020] The user interaction volume includes at least one of the following: user click volume, user search volume; K is a natural number greater than or equal to 1.

[0021] Optionally, the performing completeness evaluation on the first event title output by the large language model and belonging to the user's attention scope includes:

[0022] Encrypting the first event title;

[0023] Sending the encrypted first event title to the management terminal, and decrypting the encrypted first event title by the management terminal;

[0024] The management terminal performs integrity assessment on the decrypted first event title.

[0025] Optionally, after obtaining the target event title that passes the completeness evaluation, the method further includes:

[0026] Perform keyword matching on the target event titles, screen the target event titles, and obtain target event titles with successful keyword matching;

[0027] The target event title that successfully matches the keyword is used as the target event title for retrieval.

[0028] A second aspect of an embodiment of the present invention provides a device for obtaining detailed content of an event title, the device comprising:

[0029] A judgment module, used to input a first prompt text into a large language model, and judge whether the event title belongs to the user's attention range through the large language model; the first prompt text includes: at least one range of user attention, a judgment step, a first example, and a first output format;

[0030] An evaluation module, configured to perform a completeness evaluation on the first event title output by the large language model and belonging to the user's attention scope, to obtain a target event title that passes the completeness evaluation;

[0031] A retrieval module, used to call a search engine through the large language model to retrieve detailed content of the target event title;

[0032] The sorting module is used to input the second prompt text into the large language model, sort the details of the target event title according to the specified format through the large language model, and obtain the sorted details of the target event title; the second prompt text includes: original content, sorting steps, a second example and a second output format.

[0033] Optionally, the evaluation module comprises:

[0034] A completeness evaluation module, used to perform a completeness evaluation on the first event title from multiple dimensions of data completeness, data timeliness, data accuracy, data consistency, data uniqueness and data validity, to obtain a target event title that passes the completeness evaluation;

[0035] Among them, the data integrity represents the ratio of the first event title with complete content to all the first event titles; the data timeliness represents the ratio of the first event title processed within the specified time to all the first event titles; the data accuracy represents the ratio of the first event title with accurate collection to all the first event titles; the data consistency represents the ratio of the first event title with consistent data structure to all the first event titles; the data uniqueness represents the ratio of the first event title with unique data source to all the first event titles; the data validity represents the ratio of the first event title that meets the constraint range to all the first event titles.

[0036] Optionally, the retrieval module includes:

[0037] A search module, configured to call multiple search engines through the large language model to retrieve detailed content of the target event title;

[0038] The summarizing module is used to call the summarizing module through the large language model to summarize the details of the multiple target event titles retrieved by the multiple search engines until the preset content arrangement requirements are met.

[0039] Optionally, the aggregation module includes:

[0040] A content sorting module, configured to sort the detailed contents of the multiple target event titles based on the user interaction amounts corresponding to the detailed contents of the multiple target event titles, and summarize the detailed contents of the target event titles corresponding to the top K user interaction amounts;

[0041] The user interaction volume includes at least one of the following: user click volume, user search volume; K is a natural number greater than or equal to 1.

[0042] Optionally, the evaluation module comprises:

[0043] An encryption module, used for encrypting the first event title;

[0044] A decryption module, used to send the encrypted first event title to the management terminal, and decrypt the encrypted first event title through the management terminal;

[0045] The decryption evaluation module is used to evaluate the integrity of the decrypted first event title through the management terminal.

[0046] Optionally, the device further comprises:

[0047] A matching module, configured to, after obtaining a target event title that passes the completeness evaluation, perform keyword matching on the target event title, screen the target event title, and obtain a target event title with a successful keyword matching;

[0048] The title determination module is used to use the target event title that successfully matches the keyword as the target event title for retrieval.

[0049] A third aspect of an embodiment of the present invention provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed by the processor, the method for obtaining the details of an event title according to the first aspect of the embodiment of the present invention is implemented.

[0050] A fourth aspect of an embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method for obtaining the detailed content of an event title according to the first aspect of the embodiment of the present invention is implemented.

[0051] In the method for obtaining the details of an event title provided by an embodiment of the present invention, firstly, a first prompt text is input into a large language model, and the large language model is used to determine whether the event title belongs to the user's attention scope; then, a completeness evaluation is performed on the first event title that belongs to the user's attention scope output by the large language model, and a target event title that passes the completeness evaluation is obtained; then, a search engine is called through the large language model to retrieve the details of the target event title; finally, a second prompt text is input into the large language model, and the large language model is used to organize the details of the target event title in a specified format to obtain the organized details of the target event title.

[0052] In this method, a large language model is integrated in the event collection stage, and the large language model determines whether the event title belongs to the user's field of concern. This process only requires the specified event collection scope, descriptive words and some samples, without the need to manually set keywords, segmentation words, synonyms and maintain them at any time, saving a lot of manpower and time, and improving the accuracy of data collection. In the event detail recording stage, the network content of the target event title is retrieved through a search engine, and the large language model is used to form accurate, detailed and unified format details of the event, thereby solving the problem of inconsistent event recording standards and large differences among various event sources. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0054] Figure 1 is a flow chart of a method for obtaining detailed content of an event title according to an embodiment of the present invention;

[0055] Figure 2 This is a data encryption communication flow chart provided by an embodiment of the present invention;

[0056] Figure 3 It is an overall flow chart of a method for improving the quality of event monitoring content based on a large language model according to an embodiment of the present invention;

[0057] Figure 4 It is a structural block diagram of a device for obtaining detailed content of an event title provided by an embodiment of the present invention;

[0058] Figure 5 It is a schematic diagram of an electronic device shown in one embodiment of the present invention. DETAILED DESCRIPTION

[0059] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0060] Hereinafter, specific examples of the present solution will be described in more detail with reference to the accompanying drawings.

[0061] refer to Figure 1 , Figure 1 FIG. 1 is a flow chart of a method for obtaining detailed content of an event title according to an embodiment of the present invention. Figure 1 As shown, the method for obtaining the detailed content of the event title in this embodiment may include the following steps:

[0062] Step S11: input the first prompt text into the large language model, and use the large language model to determine whether the event title belongs to the user's attention range; the first prompt text includes: at least one range of user attention, a determination step, a first example, and a first output format.

[0063] In this embodiment, for event collection, a large language model can be integrated in the event collection stage. The large language model (LLM) specifically refers to the generative AI (Generative Artificial Intelligence) large model in the document. It is a type of artificial intelligence technology that can generate content, solutions or new concepts. It is usually based on advanced algorithms such as deep learning and neural networks, and through training with large amounts of data, it enables computers to think and create like humans. LLM is different from traditional AI models. Generally, no training is required. The AI ​​capabilities in application projects can be realized through development methods such as prompt word engineering, knowledge base, and agent.

[0064] For event collection, the first prompt text can be input into the large language model, and the large language model can be used to determine whether the event title belongs to the user's attention range, and the first event title belonging to the user's attention range can be obtained. Among them, the first prompt text is used to guide the large language model to output the event title that the user is concerned about. The first prompt text at least includes: at least one range of user attention (which can be understood as a specified event collection range), a judgment step, a first example, and a first output format. The first example is the sample required by the large language model in the event collection stage, and the first output format is the output format of the large language model specified in the event collection stage.

[0065] In an optional example, multiple event collection sources and event collection schemes corresponding to each event collection source may be determined, and then collection may be started: 1. For event collection sources that can collect events in batches, the event collection sources may be sent in batches to the large language model, and the first prompt text may be input into the large language model; such as "For the above events, please determine all events related to the focus range A". In order to improve the accuracy of the judgment of the large language model, the first prompt sample also includes some event examples (i.e., the first example), and the first example may be filtered according to the historical event collection situation. 2. For event collection sources that collect a single event, the event collection source is sent to the large language model, and the first prompt text is input into the large language model; such as "For the above events, please determine whether it is an event related to the focus range A".

[0066] For example, in one embodiment, the first prompt text of the large language model is as follows:

[0067] “““

[0068] Your role is an event collector. Please determine whether the following event titles are events that require attention.

[0069] User attention range: 1, 2, 3

[0070] Judgment steps: 1, 2, 3

[0071] First example: 1, 2, 3

[0072] First output format: xxx

[0073] ”””

[0074] Step S12: Performing a completeness evaluation on the first event title that is output by the large language model and belongs to the user's attention scope, to obtain a target event title that passes the completeness evaluation.

[0075] In this embodiment, after obtaining the first event title that belongs to the user's attention range through the large language model, the first event title can be evaluated for completeness to obtain the target event title that passes the completeness evaluation. The first event title is the event title that the user needs to pay attention to, and the target event title is the event title that passes the completeness evaluation. The completeness evaluation of this embodiment is to evaluate the data completeness of the event title from multiple dimensions.

[0076] Step S13: calling a search engine through the large language model to retrieve detailed content of the target event title.

[0077] In this embodiment, after obtaining the target event title, the search engine can be called through the large language model to search the target event title on the Internet to obtain the details corresponding to the target event title. Among them, the search engine of this embodiment can be langchain, which is an AI application development framework that can give AI applications the ability to load local documents, perform semantic matching on local documents, connect to local databases, connect to local API interfaces, and chain calls to large language models and local capabilities (chain) while connecting to LLM capabilities, and achieve the ability to achieve the specified operation target of the application through multiple rounds of calls.

[0078] Step S14: input the second prompt text into the large language model, and organize the details of the target event title according to the specified format through the large language model to obtain the organized details of the target event title; the second prompt text includes: original content, organization steps, a second example and a second output format.

[0079] In this embodiment, after obtaining the details of the target event title, in the event details recording stage, the second prompt text can be input into the large language model, and the details of the target event title are sorted according to the specified format by the large language model to obtain the sorted details of the target event title. Among them, the second prompt text is used to guide the large language model to output the sorted details of the target event title, and the second prompt text at least includes: original content, sorting steps, second examples, and second output format. Among them, the original content is the details of the target event title input into the large language model for sorting, the second example is the example required by the large language model in the event details recording stage, and the second output format is the unified output format of the large language model specified in the event details recording stage.

[0080] In an optional example, the target event title and the details of the target event title may be input into the large language model, and the second prompt text may be input into the large language model, and the large language model may sort out the descriptive words according to the preset content, and further process the details of the target event title in terms of language, such as "Please sort out a detailed event report of about 1,000 words based on the target event title and the following details retrieved through the search engine."

[0081] For example, in one embodiment, the second prompt text of the large language model is as follows:

[0082] “““

[0083] Your role is an event content organizer. The original content provided is the relevant articles found from Internet search engines for the title of this target event. Please organize this event into the required format.

[0084] Original content: 1, 2, 3

[0085] Second output format: 1, 2, 3

[0086] Arrangement steps: 1, 2, 3

[0087] Second example: 1, 2, 3

[0088] ”””

[0089] In this embodiment, a large language model is introduced in each stage of the event monitoring system's collection and content recording. The large language model with human language understanding ability assists humans in judging, analyzing, and sorting events, improves the accuracy of event collection and recording, reduces the workload of manual participation, and greatly improves work quality and work efficiency. Among them, in the event collection stage, a large language model is integrated, and the large language model is used to determine whether the event title belongs to the user's field of concern. This process only requires the implementation of specifying the event collection scope, descriptive words, and some samples, without manually setting keywords, segmentation words, synonyms, and maintaining them at any time, saving a lot of manpower and time, and improving the accuracy of data collection. In the event detail recording stage, the network content of the target event title is retrieved through a search engine, and the large language model is used to form accurate, detailed, and unified format details of the event, thereby solving the problem of inconsistent event recording standards and large differences among various event sources.

[0090] In addition, in a preferred embodiment, all collected events and related fields (such as event titles) are temporarily stored in the redis database as initialized content. After obtaining the sorted details of the target event title, the sorted details and related fields of the target event title are stored in the mysql database.

[0091] In combination with the above embodiments, in one implementation, the present invention also provides a method for obtaining the details of an event title. Specifically, in this method, the above step S12 may include step S21:

[0092] Step S21: Perform a completeness evaluation on the first event title from multiple dimensions of data integrity, data timeliness, data accuracy, data consistency, data uniqueness and data validity to obtain a target event title that passes the completeness evaluation.

[0093] In this embodiment, the data integrity of the collected event titles is evaluated from the dimensions of content integrity, timeliness of event release time, accuracy of whether the event information is required to be collected, consistency of event information data format, uniqueness of event information data source, and validity of whether the event information is garbled. Specifically, after obtaining the first event title, the integrity of the first event title is evaluated from multiple dimensions of data integrity, data timeliness, data accuracy, data consistency, data uniqueness, and data validity, and the target event title that passes the integrity evaluation is screened from the first event title.

[0094] Among them, data integrity represents the ratio of first event titles with complete content to all first event titles; data timeliness represents the ratio of first event titles processed within the specified time to all first event titles; data accuracy represents the ratio of first event titles with accurate collection to all first event titles; data consistency represents the ratio of first event titles with consistent data structure to all first event titles; data uniqueness represents the ratio of first event titles with unique data source to all first event titles; data validity represents the ratio of first event titles that meet the constraint range to all first event titles.

[0095] In an optional manner, the ratio method can be used to evaluate the completeness of the first event title from multiple dimensions of data integrity, data timeliness, data accuracy, data consistency, data uniqueness and data validity. Among them, the ratio method has a simple mathematical form X=A / B. Where A is the number of data that meets data quality expectations (such as non-empty data), and the denominator B is the total number of data. The closer X is to 1, the better the data quality, and vice versa, it means that the data quality is poor. The ratio method is used to separately calculate dimensions such as data integrity, data timeliness, data accuracy, data consistency, data uniqueness and data validity. For example, a data set has 100% integrity, 80% timeliness, 85% accuracy, 70% consistency, 100% uniqueness, and 98% validity, thereby determining the completeness corresponding to the first event title based on data integrity, data timeliness, data accuracy, data consistency, data uniqueness and data validity.

[0096] For example, 1. For data integrity, data integrity is equal to the ratio of all first event titles with complete content to all first event titles. Let f(x) be the assignment of the evaluation object E in the (0,1) mapping. If the E data is complete, the value is 1, otherwise it is 0, as shown in formula (3-1).

[0097]

[0098] In this way, the integrity calculation method of the data set P is obtained, as shown in (Formula 2):

[0099] 2. Regarding data timeliness, data timeliness is equal to the ratio of all first event titles processed within the specified time to all first event titles. Let the interval (T1, T2) represent the specified time range for processing data, and let g(x) be the mapping of the assignment of the evaluation object E to (0, 1). If the value of E is within the interval, the value is 1, otherwise it is 0, as shown in formula (3-3).

[0100]

[0101] In this way, the timeliness calculation method of the data set P is obtained, as shown in formula (Formula 4):

[0102]

[0103] 3. Regarding data accuracy, data accuracy is equal to the ratio of all first event titles that are accurately collected to all first event titles. Let h(x) be the mapping of the value result of the evaluation object E to (0,1). If En is equal to the true value S, then the value is 1, otherwise it is 0, as shown in formula (Formula 5):

[0104]

[0105] In this way, the accuracy calculation method of the data set P is obtained, as shown in formula (Formula 6).

[0106]

[0107] 4. Regarding data consistency, data consistency is equal to the ratio of all first event titles that meet the consistency value (i.e., first event titles with consistent data structures) to all first event titles. Let the set Q i = {Q i1 , Q i2 , …, Q ij} represents the possible value range of E. Let d(x) be the mapping of the value of the evaluation object E to (0,1). If the value of E is equal to any value in Q, then the value is 1, otherwise it is 0, as shown in formula (7):

[0108]

[0109] In this way, the consistency calculation method of the data set P is obtained, as shown in formula (Formula 8):

[0110]

[0111] 5. Regarding data uniqueness, data uniqueness is equal to the ratio of the first event title that is unique to all data sources to all first event titles. Let w(x) be the mapping of the value of the evaluation object E in (0,1), let the interval [1,∞) be the number of times the data appears, if the value of E is 1, then the value is 1, otherwise it is 0, as shown in formula (9):

[0112]

[0113] In this way, the uniqueness calculation method of the data set P is obtained, as shown in formula (Formula 10):

[0114]

[0115] 6. Regarding data validity, data validity is equal to the ratio of all first event titles that meet the constraint range to all first event titles. Let q(x) be the mapping of the value of the evaluation object E in (0,1). If the value of E is any value in the set S, then the value is 1, otherwise it is 0, as shown in formula (Formula 11):

[0116]

[0117] In this way, the validity calculation method of the data set P is obtained, as shown in formula (Formula 12):

[0118] In an optional example, each evaluation value (data integrity, data timeliness, data accuracy, data consistency, data uniqueness and data validity) takes a value between 0 and 1, and the six indicators are weighted averaged to obtain the evaluation index of this completeness assessment, that is, the data processing effect.

[0119] In combination with the above embodiments, in another implementation, the present invention also provides a method for obtaining the details of an event title. Specifically, in the method, the above step S13 may specifically include step S31 and step S32:

[0120] Step S31: calling multiple search engines through the large language model to retrieve detailed content of the target event title.

[0121] In this embodiment, since the content formats of various event sources are not uniform, it is necessary to retrieve the standardized and easy-to-read detailed content of this event by searching the Internet content. First, through langchain, load the search engine chain (such as baiduchain or bingchain), and then search the event content on the Internet through the target event title. For example, the large language model and langchain can be used to search the network content of other content sources on the Internet for the target event title. Specifically, multiple search engines can be called through the large language model to retrieve the detailed content of the target event title.

[0122] Step S32: calling a summary module through the large language model to summarize the details of the multiple target event titles retrieved by the multiple search engines until the preset content arrangement requirements are met.

[0123] In this embodiment, after obtaining the detailed contents of multiple target event titles retrieved by multiple search engines, the summary module can be called through the large language model to summarize the detailed contents of the multiple target event titles until the preset content organization requirements are met.

[0124] For example, an agent may be set up to implement chain execution of multiple chains, and multiple search engine chains may be used at the same time. The number of contents to be searched by each search engine chain may be set, and these contents may be executed in sequence. Then, a summarization chain (i.e., a summarization module) may be added to summarize the details of multiple target event titles of multiple search engines until the preset content organization requirements are met.

[0125] In combination with the above embodiments, in one implementation, an embodiment of the present invention further provides a method for obtaining the details of an event title. Specifically, in the method, the "summarizing the details of the multiple target event titles retrieved by the multiple search engines" in the above step S32 may specifically include step S41:

[0126] Step S41: sorting the detailed contents of the multiple target event titles based on the user interaction amounts corresponding to the detailed contents of the multiple target event titles, and summarizing the detailed contents of the target event titles corresponding to the top K user interaction amounts.

[0127] In this embodiment, in order to obtain better quality details content, the user interaction volume corresponding to the details content of multiple target event titles retrieved by multiple search engines can be obtained, and then based on the user interaction volume corresponding to the details content of the multiple target event titles, for example, in order from large to small, the details content of the multiple target event titles are sorted, and the details content of the target event titles corresponding to the first K (K is a natural number greater than or equal to 1) user interaction volumes are summarized as the details content that requires further content organization.

[0128] The user interaction volume in this embodiment includes at least one of the following: user click volume and user search volume.

[0129] In combination with the above embodiments, in one implementation, an embodiment of the present invention further provides a method for obtaining detailed content of an event title. Specifically, in the method, the "performing a completeness evaluation on the first event title output by the large language model and belonging to the user's attention range" in the above step S12 may specifically include steps S51 to S53:

[0130] Step S51: Encrypt the first event title.

[0131] In this embodiment, the agent collection encryption security is set to combine the sub-threads in the event data extraction process with the agent communication encryption to ensure data cleaning, encryption and desensitization. Specifically, after obtaining the first event title output by the large language model, the first event title can be encrypted to obtain the encrypted first event title.

[0132] Step S52: Send the encrypted first event title to the management terminal, and decrypt the encrypted first event title through the management terminal.

[0133] In this embodiment, after obtaining the encrypted first event title, the encrypted first event title may be sent to a management end (ie, an agent management end), and the management end may decrypt the encrypted first event title.

[0134] Step S53: the management terminal performs integrity evaluation on the decrypted first event title.

[0135] In this embodiment, after the encrypted first event title is decrypted by the management end to obtain the decrypted first event title, the management end performs integrity assessment on the decrypted first event title. For example, the management end may perform integrity assessment on the decrypted first event title from multiple dimensions of data integrity, data timeliness, data accuracy, data consistency, data uniqueness, and data validity.

[0136] In an optional example, the Agent management end may generate a certificate for the Agent based on its own CA; then the Agent sends a heartbeat encrypted with the certificate, and after receiving the heartbeat, the Agent management end determines whether the Agent is a managed Agent based on the reporting information, and returns a Token in the reporting information. Among them, a Token is a word unit, which refers to a symbol used to represent a word or phrase in the natural language processing process. A token can be a single character or a sequence of multiple characters.

[0137] The Agent gets the Token for recording, and then sends a heartbeat with the Token. The management end authenticates the Token, and returns the corresponding data if the authentication is successful. The Agent sends the first event title when it learns that the authentication is successful. If the heartbeat fails, the collected information (i.e., the first event title) is recorded locally. The management end sends a command to the specified Agent with the Token. The Agent compares the encrypted and stored Token to determine the authenticity of the management end and prevent illegal access and attacks. Finally, the Agent and the management end perform two-way authentication through certificate signature and use the certificate to encrypt communication data to ensure the security of communication.

[0138] In one embodiment, if Figure 2 As shown, Figure 2 This is a data encryption communication flow chart provided by an embodiment of the present invention. Figure 2 In the process, the Agent client initiates an https request to the Agent management end, and the Agent management end gives the Agent client a CA certificate, which includes the management end's B_ public key. After the Agent client obtains the CA certificate, it uses the CA's public key (which comes with the operating system and browser) to decrypt the CA certificate to verify the digital signature successfully, and obtains the B_ public key from the CA certificate. In addition, the Agent client sends its own client certificate to the server (i.e., the Agent management end), and the client certificate contains the client's C_ public key. The Agent management end decrypts the client integer and obtains the client's C_ public key.

[0139] Then, the Agent client sends the symmetric encryption scheme it can support to the server, allowing the server to choose. The server encrypts the selected encryption scheme with C_public key and sends it to the Agent client. After that, the Agent client decrypts with C_private key to obtain the encryption scheme, generates a random number (key F), uses the server's B_public key to encrypt key F, and sends it to the Agent management end. At this time, the Agent management end uses B_private key to decrypt to obtain key F. In this way, the two parties can communicate with key F, such as encrypting and decrypting the first event title with key F. Among them, the public key is the non-secret half of the key pair used with the private key algorithm. The public key is usually used to encrypt session keys, verify digital signatures, or encrypt data that can be decrypted with the corresponding private key. Token is a unique serial number ID card for the public key to avoid multiple public key conflicts.

[0140] In combination with the above embodiments, in one implementation, an embodiment of the present invention further provides a layer 2 networking method based on 5G customized network networking. Specifically, in the method, after the above step S12, steps S61 and S62 may also be included:

[0141] Step S61: keyword matching is performed on the target event title, and the target event title is screened to obtain target event titles with successful keyword matching.

[0142] In this embodiment, in order to improve the efficiency of event collection, when collecting events, keyword matching can be performed on the target event title first, such as setting preset keywords with a higher coverage range in advance to avoid omissions. Keyword matching is performed on the target event title based on the preset keywords to screen the target event title and obtain the target event title with successful keyword matching.

[0143] Step S62: The target event title that successfully matches the keyword is used as the target event title for retrieval.

[0144] In this embodiment, the target event title with successful keyword matching obtained after keyword matching is the target event title for retrieval. At this time, the search engine can be called through the large language model to retrieve the details corresponding to the target event title with successful keyword matching.

[0145] In one embodiment, if Figure 3 As shown, Figure 3 This is an overall flow chart of a method for improving the quality of event monitoring content based on a large language model, which is shown in an embodiment of the present invention. This embodiment aims to improve the accuracy of event monitoring collection, organization, analysis, etc. through the application of LLM technology. Figure 3In the event collection stage, first, determine the event collection source and collection source collection plan, and collect event data. At the same time, you can use the open source AI big model to specify the focus area and samples, and use the AI ​​big language model's ability to understand semantic content. The big language model will judge whether the event belongs to the user's focus area. There is no need to prepare keywords in advance or maintain keywords at any time, and the event data has been screened.

[0146] Then, in the event detail recording stage, through AI big models and web crawlers, accurate and detailed content of the event is obtained in real time through the Internet, solving the problems of inconsistent recording standards for various events, incomplete details of some event sources, and untimely updates. Specifically, it can be through the langchain AI application framework, specifying the search source, using langchain to search for event details, and specifying the word count and format requirements of the details through the open source AI big model, and using AI to organize the event details, thereby completing the analysis of event data and generating event reports.

[0147] Among them, in an optional example, the event collection technology stack is: development language: python; crawler and web content parsing framework: Scrapy, Playwright; LLM: chatglm2, chatglm3, llama2; specific event source: open API, such as Weibo open platform API; specified prompt words: specified areas of concern and examples; crawler task management: kafka, beanstalkd, celery; database: mysql; memory database: redis.

[0148] The event content acquisition technology stack is: development language: python; AI application development framework: langchain; LLM: chatglm2, chatglm3, llama2; specified prompt words: specified description words for event content organization requirements; database: mysql.

[0149] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0150] Based on the same inventive concept, an embodiment of the present invention provides a device 400 for obtaining detailed content of an event title. Figure 4 , Figure 4FIG. 1 is a structural block diagram of a device for obtaining detailed content of an event title provided by an embodiment of the present invention. Figure 4 As shown, the device 400 includes:

[0151] The judgment module 401 is used to input the first prompt text into the large language model, and judge whether the event title belongs to the user's attention range through the large language model; the first prompt text includes: at least one range of user attention, judgment steps, a first example and a first output format;

[0152] An evaluation module 402 is used to perform a completeness evaluation on the first event title that is output by the large language model and belongs to the user's attention range, to obtain a target event title that passes the completeness evaluation;

[0153] A retrieval module 403 is used to call a search engine through the large language model to retrieve the details of the target event title;

[0154] The sorting module 404 is used to input the second prompt text into the large language model, and sort the details of the target event title according to the specified format through the large language model to obtain the sorted details of the target event title; the second prompt text includes: original content, sorting steps, a second example and a second output format.

[0155] Optionally, the evaluation module 402 includes:

[0156] A completeness evaluation module, used to perform a completeness evaluation on the first event title from multiple dimensions of data completeness, data timeliness, data accuracy, data consistency, data uniqueness and data validity, to obtain a target event title that passes the completeness evaluation;

[0157] Among them, the data integrity represents the ratio of the first event title with complete content to all the first event titles; the data timeliness represents the ratio of the first event title processed within the specified time to all the first event titles; the data accuracy represents the ratio of the first event title with accurate collection to all the first event titles; the data consistency represents the ratio of the first event title with consistent data structure to all the first event titles; the data uniqueness represents the ratio of the first event title with unique data source to all the first event titles; the data validity represents the ratio of the first event title that meets the constraint range to all the first event titles.

[0158] Optionally, the retrieval module 403 includes:

[0159] A search module, configured to call multiple search engines through the large language model to retrieve detailed content of the target event title;

[0160] The summarizing module is used to call the summarizing module through the large language model to summarize the details of the multiple target event titles retrieved by the multiple search engines until the preset content arrangement requirements are met.

[0161] Optionally, the aggregation module includes:

[0162] A content sorting module, configured to sort the detailed contents of the multiple target event titles based on the user interaction amounts corresponding to the detailed contents of the multiple target event titles, and summarize the detailed contents of the target event titles corresponding to the top K user interaction amounts;

[0163] The user interaction volume includes at least one of the following: user click volume, user search volume; K is a natural number greater than or equal to 1.

[0164] Optionally, the evaluation module 402 includes:

[0165] An encryption module, used for encrypting the first event title;

[0166] A decryption module, used to send the encrypted first event title to the management terminal, and decrypt the encrypted first event title through the management terminal;

[0167] The decryption evaluation module is used to evaluate the integrity of the decrypted first event title through the management terminal.

[0168] Optionally, the device 400 further includes:

[0169] A matching module, configured to, after obtaining a target event title that passes the completeness evaluation, perform keyword matching on the target event title, screen the target event title, and obtain a target event title with a successful keyword matching;

[0170] The title determination module is used to use the target event title that successfully matches the keyword as the target event title for retrieval.

[0171] Based on the same inventive concept, another embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps in the method for obtaining the details of the event title as described in any of the above embodiments of the present invention are implemented.

[0172] Based on the same inventive concept, another embodiment of the present invention provides an electronic device 500, such as Figure 5 shown. Figure 51 is a schematic diagram of an electronic device according to an embodiment of the present invention. The electronic device includes a memory 502, a processor 501, and a computer program stored in the memory and executable on the processor, and when executed by the processor, the steps in the method for obtaining the details of the event title according to any of the above embodiments of the present invention are implemented.

[0173] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0174] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0175] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, embodiments of the present invention may take the form of complete hardware embodiments, complete software embodiments, or embodiments combining software and hardware. Furthermore, embodiments of the present invention may take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0176] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0177] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0178] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0179] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0180] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0181] The above is a detailed introduction to the method, device, equipment and medium for obtaining the detailed content of an event title provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A method for obtaining detailed content of an event title, characterized in that: The method comprises: Inputting a first prompt text into a large language model, and determining whether the event title belongs to a user's attention range through the large language model; the first prompt text includes: at least one user's attention range, a determination step, a first example, and a first output format; Performing a completeness evaluation on the first event title that is output by the large language model and belongs to the user's attention scope, to obtain a target event title that passes the completeness evaluation; Invoking a search engine through the large language model to retrieve detailed content of the target event title; The second prompt text is input into the large language model, and the details of the target event title are sorted according to the specified format by the large language model to obtain the sorted details of the target event title; the second prompt text includes: original content, sorting steps, a second example and a second output format.

2. The method according to claim 1, characterized in that The step of performing completeness evaluation on the first event title output by the large language model and belonging to the user's attention scope to obtain a target event title that passes the completeness evaluation includes: Performing a completeness evaluation on the first event title from multiple dimensions of data completeness, data timeliness, data accuracy, data consistency, data uniqueness, and data validity to obtain a target event title that passes the completeness evaluation; Among them, the data integrity represents the ratio of the first event title with complete content to all the first event titles; the data timeliness represents the ratio of the first event title processed within the specified time to all the first event titles; the data accuracy represents the ratio of the first event title with accurate collection to all the first event titles; the data consistency represents the ratio of the first event title with consistent data structure to all the first event titles; the data uniqueness represents the ratio of the first event title with unique data source to all the first event titles; the data validity represents the ratio of the first event title that meets the constraint range to all the first event titles.

3. The method according to claim 1, characterized in that The calling of a search engine through the large language model to retrieve detailed content of the target event title includes: Calling multiple search engines through the large language model to retrieve detailed content of the target event title; The large language model is used to call a summary module to summarize the details of the multiple target event titles retrieved by the multiple search engines until the preset content arrangement requirements are met.

4. The method according to claim 3, characterized in that The step of aggregating the details of the multiple target event titles retrieved by the multiple search engines includes: Based on the user interaction amounts corresponding to the detailed contents of the multiple target event titles, the detailed contents of the multiple target event titles are sorted, and the detailed contents of the target event titles corresponding to the top K user interaction amounts are summarized; The user interaction volume includes at least one of the following: user click volume, user search volume; K is a natural number greater than or equal to 1.

5. The method according to claim 1, characterized in that The performing completeness evaluation on the first event title output by the large language model and belonging to the user's attention scope includes: Encrypting the first event title; Sending the encrypted first event title to the management terminal, and decrypting the encrypted first event title by the management terminal; The management terminal performs integrity assessment on the decrypted first event title.

6. The method according to any one of claims 1 to 5, characterized in that: After obtaining the target event title that passes the completeness evaluation, the method further includes: Perform keyword matching on the target event titles, screen the target event titles, and obtain target event titles with successful keyword matching; The target event title that successfully matches the keyword is used as the target event title for retrieval.

7. A device for obtaining detailed content of an event title, characterized in that: The device comprises: A judgment module, used to input a first prompt text into a large language model, and judge whether the event title belongs to the user's attention range through the large language model; the first prompt text includes: at least one range of user attention, a judgment step, a first example, and a first output format; An evaluation module, configured to perform a completeness evaluation on the first event title output by the large language model and belonging to the user's attention scope, to obtain a target event title that passes the completeness evaluation; A retrieval module, used to call a search engine through the large language model to retrieve detailed content of the target event title; The sorting module is used to input the second prompt text into the large language model, sort the details of the target event title according to the specified format through the large language model, and obtain the sorted details of the target event title; the second prompt text includes: original content, sorting steps, a second example and a second output format.

8. The device according to claim 7, characterized in that The retrieval module comprises: A search module, configured to call multiple search engines through the large language model to retrieve detailed content of the target event title; The summarizing module is used to call the summarizing module through the large language model to summarize the details of the multiple target event titles retrieved by the multiple search engines until the preset content arrangement requirements are met.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the computer program is executed by the processor, the method for obtaining the detailed content of the event title as described in any one of claims 1 to 6 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for obtaining detailed content of an event title as described in any one of claims 1 to 6 is implemented.

Images