Research report generation and distribution method, system and equipment and storage medium

Abstract

The invention provides a research report generation and distribution method, system and device and a storage medium, and the method comprises the steps: employing a high-concurrency crawler mode to obtain information; based on the processing parameters, carrying out adaptive online arrangement on the information message to obtain a first text, and carrying out data preprocessing on the first text to obtain a second text; reasoning contents in the second text by using a deep learning model to generate a research report; all users in a channel corresponding to the research report theme and corresponding users in other channels are screened as distribution users, and an intelligent dynamic routing method is adopted to distribute the research report to the distribution users; and the distribution result is visually monitored, and the processing parameters are adjusted according to the distribution result. According to the method, the content in the text is reasoned by using the depth model, so that key information is fully mined to generate the research report; and the efficiency in the distribution process is ensured by adopting intelligent dynamic routing.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method, system, device and storage medium for generating and distributing a research report. Background Art

[0002] In various industries, such as the financial market, users need accurate research reports to keep abreast of situations and assist in decision-making. There is an increasing demand for integrating multi-source information and distributing it to users.

[0003] In related technologies, traditional research report generation usually relies on manual analysis. The above technical means are not only time-consuming and labor-intensive, but also easily affected by subjective judgment, resulting in inaccurate analysis results. There are currently technical means of integrating resources using natural language processing or other machine learning, but there is a lack of report interactivity and feedback mechanism. In addition, the report content only intercepts information from the source document and is not integrated and optimized, making it difficult for users to obtain key information from it. Moreover, intelligent routing distribution is not performed based on the current network status, resulting in users not receiving reports in a timely manner.

[0004] Based on the above analysis of the development status of this technical field, the existing technology lacks a solution for generating research reports by organizing and reasoning from multi-source documents and adopting intelligent routing algorithms in the distribution process. Summary of the Invention

[0005] The purpose of the present invention is to provide a research report generation and distribution method, system, device and storage medium, aiming to solve the above-mentioned problems in the prior art.

[0006] According to a first aspect of an embodiment of the present invention, a method for generating and distributing a research report is provided, comprising:

[0007] Use high-concurrency crawlers to obtain information;

[0008] Adaptively organize the news information online based on the processing parameters to obtain a first text, and perform data preprocessing on the first text to obtain a second text;

[0009] Use deep learning models to infer the content of the second text and generate a research report;

[0010] Screen all users in the channel corresponding to the research report topic, and corresponding users in other channels as distribution users, and use intelligent dynamic routing method to distribute the research report to the distribution users;

[0011] Monitor the distribution results and adjust processing parameters or deep learning model parameters based on the distribution results.

[0012] According to a second aspect of an embodiment of the present invention, a research report generation and distribution system is provided, comprising:

[0013] Crawler module, used to obtain news information using a high-concurrency crawler method;

[0014] A sorting and preprocessing module is used to adaptively sort the news information online based on the processing parameters to obtain a first text, and perform data preprocessing on the first text to obtain a second text;

[0015] An inference module is used to use a deep learning model to infer the content of the second text and generate a research report;

[0016] The routing distribution module is used to screen all users in the channel corresponding to the research report topic, and the corresponding users in other channels as distribution users, and use intelligent dynamic routing method to distribute the research report to the distribution users;

[0017] The monitoring result module is used to monitor the distribution results and adjust the processing parameters or deep learning model parameters based on the distribution results.

[0018] According to a third aspect of an embodiment of the present invention, an electronic device is provided, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed by the processor, the steps of the research report generation and distribution method provided in the first aspect of the present disclosure are implemented.

[0019] According to a fourth aspect of an embodiment of the present invention, a computer-readable storage medium is provided, on which an implementation program for information transmission is stored. When the program is executed by a processor, the steps of the research report generation and distribution method provided in the first aspect of the present disclosure are implemented.

[0020] The technical solution provided by the embodiment of the present invention includes the following beneficial effects: organizing and preprocessing the initial information obtained by the crawler, effectively controlling the amount of crawled information and improving the content quality, and then using the deep model to infer the content in the text to fully mine key information to generate a research report; in the process of report distribution, not only relevant users in relevant channels are obtained, but also users who are interested in the topic are obtained from other channels based on their behavioral habits, and intelligent dynamic routing is used to ensure efficiency in the distribution process; monitoring the distribution results to provide a feedback mechanism to optimize the user experience.

[0021] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate one or more embodiments of this specification or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments recorded in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0023] Figure 1 is a flow chart of a method for generating and distributing a research report according to an embodiment of the present invention;

[0024] Figure 2 is a schematic diagram of a research report generation and distribution system according to an embodiment of the present invention;

[0025] Figure 3 is a schematic diagram of a distribution architecture according to an embodiment of the present invention;

[0026] Figure 4 is a schematic diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0027] In order to enable those skilled in the art to better understand the technical solutions in one or more embodiments of this specification, the technical solutions in one or more embodiments of this specification will be clearly and completely described below in conjunction with the drawings in one or more embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on one or more embodiments of this specification, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this document.

[0028] Method Example

[0029] According to an embodiment of the present invention, a method for generating and distributing a research report is provided. Figure 1 This is a flow chart of a method for generating and distributing a research report according to an embodiment of the present invention. The embodiment of the present invention is for generating and distributing a research report in the financial industry. Figure 1 As shown, the research report generation and distribution method according to an embodiment of the present invention specifically includes:

[0030] In step S110, a high-concurrency crawler method is used to obtain news information, specifically including:

[0031] An asynchronous I / O model is used to retrieve topic-related news within a time threshold from multiple source texts. Asynchronous I / O operations do not block the main thread and can execute other tasks while waiting for the I / O operation to complete, providing high concurrency. In this embodiment of the present invention, the time threshold is 12 hours, which means that popular news within the past 12 hours from the current time is captured.

[0032] The crawler uses a connection pool and a re-request mechanism during the acquisition process. The connection pool is a technology for managing network connections. It reduces the time and cost of establishing a new connection for each request by reusing existing connections. The re-request mechanism is a strategy that automatically retries when a network request fails, which can improve the success rate of data crawling.

[0033] The above technical means have realized a high-performance HTTP request library and reactive programming paradigm to ensure the real-time and reliability of data.

[0034] In step S120, the news information is adaptively sorted online based on the processing parameters to obtain a first text, and the first text is preprocessed to obtain a second text, which specifically includes:

[0035] Using processing parameters to control the number of multi-source texts in the information to obtain the first text, it is necessary to integrate consulting information from different websites and sources. If there are too many multi-source texts, it may be difficult to obtain key useful information from them. After adaptive online sorting, resource information documents that are related to the topic and meet the required number of articles are integrated together. If the current number of source texts exceeds the range allowed by the processing parameters, the source texts with higher user click volume are filtered. Preferably, the processing parameters also include resource integration templates, etc.

[0036] Cleaning the first text, removing special characters, performing word segmentation, and filtering stop words on the first text in sequence to obtain a second text;

[0037] Deleting special characters means removing punctuation marks, numbers, symbols, etc. from the first text; word segmentation is the process of dividing long sentences into short sentences or words to better understand the semantics of the text; stop word filtering is to filter out words that do not carry actual semantic information, otherwise they may introduce noise and affect the accuracy of subsequent processing.

[0038] In step S130, the deep learning model is used to infer the content of the second text to generate a research report, which specifically includes:

[0039] A recursive neural network model is used to extract entities, relationships, and keywords from the second text to generate a structured text document. The recursive neural network model RNN is a special neural network. In the embodiment of the present invention, LSTM or GRU processing is used. It is suitable for processing sequence data such as text and can extract entities, relationships, and keywords in natural language processing. Entities refer to specific content such as names of people, places, and organizations. Relationships refer to relationships between entities. Keywords are relatively important words. Users summarize text content to obtain a structured text document.

[0040] The graph attention mechanism model is used to infer the potential relationships in the second text. The images generated by the graph attention mechanism (GAT) can discover patterns and potential relationships in the text. The graph attention mechanism model of the embodiment of the present invention infers the second text instead of the structured text extracted by the recurrent neural network. This is because directly extracting information from the complete document can effectively retain more contextual information, which is very important for understanding complex contexts. This enables the visualization of financial market data and information.

[0041] Generate an image and scale and crop the image. In the embodiment of the present invention, OpenCV is used to scale and crop the image; merge the text document and the scaled and cropped image into a research report, providing users with a more intuitive and easier to understand research report format;

[0042] Deep learning model reasoning uses deep learning and graph neural network technologies to intelligently integrate and deeply mine multi-source heterogeneous data, achieving high-level abstraction and intelligent reasoning of data, and providing richer and deeper understanding solutions.

[0043] Preferably, sentiment analysis is performed on the generated research report, and a sentiment score is output using a TextBlob model. The sentiment score range is [-1, 1], where -1 represents negative, 1 represents positive, and 0 represents neutral. The sentiment score is used as a label for the research report; using the label to more accurately identify key customers among the distribution users;

[0044] In step S140, all users in the channel corresponding to the research report topic and corresponding users in other channels are selected as distribution users, and the research report is distributed to the distribution users using an intelligent dynamic routing method, specifically including:

[0045] A correlation analysis method is used to obtain software or websites corresponding to the research report topic as the corresponding channel, and all users in the channel are obtained as the first user set. In the embodiment of the present invention, the financial industry is used, and the channel includes financial news software. Users who download this type of software are generally interested in financial topics. Therefore, all users in the corresponding channel are regarded as distribution users.

[0046] Filter users matching the topic in other channels based on historical click data as the second user set. For example, in some popular software or websites, although these software are not related to the financial industry, users often click on posts related to the topic. Therefore, such users can also be added to the distribution user set.

[0047] The first user set and the second user set are used as distribution users;

[0048] Preferably, a support vector machine is used to identify the emotional tendencies of distribution users, which are also divided into negative, positive and neutral. The labels of the research report are matched with the user's emotional tendencies, and the users who match the labels of the research report are used as distribution users, thereby achieving accurate dissemination based on user needs.

[0049] Extract link utilization, node load, and path delay in the propagation network in real time as feature vectors;

[0050] Link utilization represents the ratio of data transmission rate to maximum transmission rate; node load represents the processing task volume or traffic volume on the node, that is, the ratio of processing task volume to maximum processing capacity, or the ratio of the sum of input and output traffic to maximum output capacity; path delay represents the transmission time of data packets from source node to destination node;

[0051] The link utilization, node load and path delay are monitored using SNMP tools. The feature vector is a sequential concatenation of vectors.

[0052] Based on the feature vectors, an unsupervised clustering method is used to identify normal and abnormal paths in the propagation network. The unsupervised clustering method can group data according to the similarity of feature vectors without labels. Abnormal paths are eliminated in the propagation network to obtain an optimized network.

[0053] The Dijkstra algorithm is used in the optimization network to obtain the optimal path, and the optimal path is used to distribute the research report to the distribution users.

[0054] In step S150, the distribution results are monitored and processing parameters or deep learning model parameters are adjusted according to the distribution results, specifically including:

[0055] Monitor distribution speed, click-through rate, download volume and user feedback as distribution results, and conduct multi-dimensional real-time monitoring and analysis of the dissemination effect of the research report; in the process of adjusting output parameters or deep learning model parameters, use multi-generation evolution and multi-objective optimization algorithms to optimize the distribution results, realizing intelligent adjustment and optimization of the research report generation process.

[0056] To sum up, in response to the existing problems, this invention proposes a method for generating and distributing research reports, which adopts an asynchronous I / O model crawler to obtain information to improve the efficiency and response speed of the crawler. In the process of crawler acquisition, a connection pool and a re-request mechanism are used to ensure the reliability of data transmission; the initial information obtained by the crawler is sorted and preprocessed to effectively control the amount of crawled information and improve the content quality, avoiding noise information from interfering with the learning of the model; the deep model is used to infer the content in the text to fully mine the key information to generate a research report, and structured text documents and image resources are inferred to enrich the form of the research report; in the process of report distribution, not only relevant users in relevant channels are obtained, but also users who are interested in the topic are obtained from other channels based on their behavioral habits, and intelligent dynamic routing is used to ensure the efficiency of the distribution process, and abnormal paths with poor conditions in the communication network are eliminated when making decisions; the distribution results are monitored to provide a feedback mechanism, and relevant parameters are adaptively adjusted to optimize the user experience.

[0057] System Example

[0058] According to an embodiment of the present invention, a research report generation and distribution system is provided. Figure 2 Schematic diagram of a research report generation and distribution system according to an embodiment of the present invention. Figure 2 As shown, the research report generation and distribution system according to an embodiment of the present invention specifically includes:

[0059] The crawler module 20 is used to obtain news information using a high-concurrency crawler method, specifically for:

[0060] Adopting an asynchronous I / O model to obtain topic-related information within a time threshold from multiple source texts;

[0061] The connection pool and re-request mechanism are used in the crawler acquisition process.

[0062] The sorting and preprocessing module 22 is used to adaptively sort the news information online based on the processing parameters to obtain a first text, and perform data preprocessing on the first text to obtain a second text, specifically for:

[0063] Using processing parameters to control the number of texts from multiple sources in the information to obtain the first text;

[0064] The first text is cleaned, and special characters are deleted, words are segmented, and stop words are filtered out in sequence to obtain a second text.

[0065] The reasoning module 24 is configured to use a deep learning model to infer the content of the second text and generate a research report, specifically for:

[0066] Use a recurrent neural network model to extract entities, relationships, and keywords from the second text to generate a structured text document. Use a graph attention mechanism model to infer the potential relationships in the second text, generate images, and scale and crop them.

[0067] Combine text documents and scaled and cropped images into research reports.

[0068] The routing distribution module 26 is used to select all users in the channel corresponding to the research report topic and corresponding users in other channels as distribution users, and distribute the research report to the distribution users using an intelligent dynamic routing method. Specifically, it is used to:

[0069] Use the correlation analysis method to obtain the software or website corresponding to the research report topic as the corresponding channel, and obtain all users in the channel as the first user set;

[0070] Filter users matching the topic in other channels based on historical click data to form the second user set;

[0071] The first user set and the second user set are regarded as distribution users.

[0072] Extract link utilization, node load, and path delay in the propagation network in real time as feature vectors;

[0073] Based on the feature vector, an unsupervised clustering method is used to identify normal paths and abnormal paths in the propagation network, and abnormal paths are eliminated in the propagation network to obtain an optimized network;

[0074] The Dijkstra algorithm is used in the optimization network to obtain the optimal path, and the optimal path is used to distribute the research report to the distribution users.

[0075] The monitoring result module 28 is used to monitor the distribution results and adjust the processing parameters or deep learning model parameters according to the distribution results. Specifically, it is used to:

[0076] Monitor distribution speed, click-through rate, download volume and user feedback as distribution results.

[0077] The above technical solutions of the embodiments of the present invention are illustrated with reference to the following drawings.

[0078] Figure 3 Schematic diagram of the distribution architecture of an embodiment of the present invention. Figure 3 As shown, the complete system architecture design for research report generation and distribution is demonstrated.

[0079] To sum up, in response to the current problems, the research report generation and distribution system invented in this paper adopts an asynchronous I / O model crawler to obtain information to improve the efficiency and response speed of the crawler. In the process of crawler acquisition, a connection pool and a re-request mechanism are used to ensure the reliability of data transmission; the initial information obtained by the crawler is sorted and preprocessed to effectively control the amount of crawled information and improve the content quality, avoiding noise information from interfering with the learning of the model; then the deep model is used to infer the content in the text to fully mine the key information to generate a research report, and structured text documents and image resources are inferred to enrich the form of the research report; in the process of report distribution, not only relevant users in relevant channels are obtained, but also users who are interested in the topic are obtained from other channels based on their behavioral habits, and intelligent dynamic routing is used to ensure the efficiency of the distribution process, and abnormal paths with poor conditions in the communication network are eliminated when making decisions; the distribution results are monitored to provide a feedback mechanism, and relevant parameters are adaptively adjusted to optimize the user experience.

[0080] Electronic device embodiment

[0081] Figure 4 is a schematic diagram of an electronic device according to an embodiment of the present invention. Electronic device 400 may include at least one processor 410 and memory 420. Processor 410 can execute instructions stored in memory 420. Processor 410 is communicatively coupled to memory 420 via a data bus. In addition to memory 420, processor 410 may also be communicatively coupled to input device 430, output device 440, and communication device 450 via the data bus.

[0082] The processor 410 may be any conventional processor, such as a commercially available CPU. The processor may also include a graphics processing unit (GPU), a field programmable gate array (FPGA), a system on chip (SOC), an application specific integrated circuit (ASIC), or a combination thereof.

[0083] The memory 420 may be implemented by any type of volatile or non-volatile memory device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0084] In the embodiment of the present disclosure, executable instructions are stored in the memory 420, and the processor 410 can read the executable instructions from the memory 420 and execute the instructions to implement all or part of the steps of any of the research report generation and distribution methods in the above exemplary embodiments.

[0085] Computer readable storage medium embodiments

[0086] In addition to the above-mentioned methods and systems, exemplary embodiments of the present disclosure may also be a computer program product or a computer-readable storage medium storing the computer program product, wherein the computer product includes computer program instructions that can be executed by a processor to implement all or part of the steps described in any of the research report generation and distribution methods in the above-mentioned exemplary embodiments.

[0087] The computer program product may be written in any combination of one or more programming languages to write program code for performing the operations of the embodiments of the present application, including object-oriented programming languages such as Java, C++, etc., as well as conventional procedural programming languages such as "C" or similar programming languages and scripting languages (e.g., Python). The program code may be executed entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0088] Computer-readable storage media can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or components, or any combination thereof. More specific examples of readable storage media include: static random access memory (SRAM) electrically connected with one or more wires, electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk, or any suitable combination thereof.

[0089] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for generating and distributing a research report, characterized in that: include: Use high-concurrency crawlers to obtain information; Adaptively organizing the news information online based on processing parameters to obtain a first text, and performing data preprocessing on the first text to obtain a second text; Using a deep learning model to infer the content of the second text, generating a research report; Screen all users in the channel corresponding to the research report topic, and corresponding users in other channels as distribution users, and use intelligent dynamic routing method to distribute the research report to the distribution users; Monitor the distribution results and adjust processing parameters or deep learning model parameters based on the distribution results.

2. The method according to claim 1, characterized in that The use of high-concurrency crawlers to obtain information specifically includes: Adopting an asynchronous I / O model to obtain topic-related information within a time threshold from multiple source texts; The connection pool and re-request mechanism are used in the crawler acquisition process.

3. The method according to claim 1, characterized in that The adaptive online sorting of the news information based on the processing parameters to obtain the first text, and the data preprocessing of the first text to obtain the second text specifically include: Using the processing parameters to control the amount of texts from multiple sources in the news information to obtain a first text; The first text is cleaned, and special characters are deleted, words are segmented, and stop words are filtered out of the first text in sequence to obtain a second text.

4. The method according to claim 1, wherein The generating of a research report by inferring the content of the second text using a deep learning model specifically includes: Using a recursive neural network model to extract entities, relationships, and keywords from the second text to generate a structured text document; using a graph attention mechanism model to infer potential relationships in the second text, generating an image and scaling and cropping the image; The text document and the scaled and cropped images are combined into a research report.

5. The method according to claim 1, wherein All users in the channels corresponding to the screening and research report topics, as well as corresponding users in other channels as distribution users specifically include: Using a correlation analysis method, obtain software or websites corresponding to the research report topic as corresponding channels, and obtain all users in the channels as a first user set; Filter users matching the topic in other channels based on historical click data to form the second user set; The first user set and the second user set are used as distribution users.

6. The method according to claim 1, characterized in that The use of the intelligent dynamic routing method to distribute the research report to the distribution user specifically includes: Extract link utilization, node load, and path delay in the propagation network in real time as feature vectors; Based on the feature vectors, an unsupervised clustering method is used to identify normal paths and abnormal paths in the propagation network, and the abnormal paths are eliminated from the propagation network to obtain an optimized network; An optimal path is obtained by using the Dijkstra algorithm in the optimization network, and the research report is distributed to the distribution user by using the optimal path.

7. The method according to claim 1, characterized in that The monitoring of distribution results specifically includes: monitoring distribution speed, click rate, download volume and user feedback as distribution results.

8. A research report generation and distribution system, characterized in that: include: Crawler module, used to obtain news information using a high-concurrency crawler method; a sorting and preprocessing module, configured to adaptively sort the news information online based on processing parameters to obtain a first text, and perform data preprocessing on the first text to obtain a second text; an inference module, configured to use a deep learning model to infer the content of the second text and generate a research report; A routing distribution module is used to screen all users in the channel corresponding to the research report topic, and corresponding users in other channels as distribution users, and distribute the research report to the distribution users using an intelligent dynamic routing method; The monitoring result module is used to monitor the distribution results and adjust the processing parameters or deep learning model parameters according to the distribution results.

9. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed by the processor, the steps of the research report generation and distribution method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores an implementation program for information transmission, and when the program is executed by the processor, the steps of the research report generation and distribution method according to any one of claims 1 to 7 are implemented.

Images