Subject intelligence data acquisition and analysis system based on network

Abstract

The invention designs a network-based subject intelligence data acquisition and analysis system, which comprises a data acquisition module, a data storage module, a data preprocessing module, a data processing module, a data analysis module, a data display module, a cloud service host and a display terminal, the method comprises the following steps: regularly and automatically acquiring subject information related data from a preset information source website through a data acquisition module, cleaning and sorting the data through a data preprocessing module, and extracting, integrating, calibrating and supplementing the obtained subject information data through the data processing module. The data statistical analysis module is used for carrying out statistical analysis on the processed subject information data, the data display module is used for displaying an analysis result on a display terminal in a visual form, and the data storage module is used for storing the collected subject information original data, the processed information data and the analysis result in a database. The system has the advantages that comprehensive, accurate and timely collection, analysis and display of subject construction current conditions, in-industry top talents and industry key information of hospitals and scientific research institutions can be realized through the Internet, the hospitals and the scientific research institutions are helped to accurately insight into subject development trends, and a basis is provided for formulating scientific subject construction plans; potential talents and frontier information are found, and hospitals are assisted to introduce top talents to preempt industry commanding height points; resource allocation is optimized, subject competitiveness is improved, and high-quality development of hospital subject construction is promoted.

Description

Technical Field

[0001] The present invention relates to a network-based subject intelligence data acquisition and analysis system and method thereof, specifically, a system and method thereof capable of acquiring, processing, analyzing and displaying subject intelligence data from multiple information source websites, and is suitable for subject construction and development planning of hospitals and scientific research institutes. Background Art

[0002] With the progress of society and the development of science and technology, the discipline construction and development in the field of medicine are facing more and more challenges and opportunities. How to improve the discipline level and competitiveness of hospitals has become an important issue of concern to hospital managers and scholars. In order to effectively carry out discipline construction and development, it is necessary to have a comprehensive, accurate and timely understanding and analysis of the status quo, trends, advantages and disadvantages of the discipline. At the same time, it is necessary to analyze and compare the excellent discipline portraits in the industry in order to formulate reasonable discipline planning and development strategies. However, the current discipline intelligence data is scattered on multiple different information source websites, such as hospital comprehensive ranking and specialty ranking websites, professional committee websites, academic journal paper databases, etc. The content update frequency, format, structure, quality, etc. of these websites are different, which brings great difficulties and challenges to the acquisition, processing, analysis and display of data. Therefore, there is an urgent need for a system and method that can acquire, process, analyze and display discipline intelligence data from multiple information source websites to meet the needs of hospital discipline construction and development.

[0003] Comparative document CN106776755A discloses an object-oriented intelligence management system, which includes a cloud data platform, an intelligence filtering module, an intelligence de-redundancy module, an intelligence classification module, a database, an intelligence analysis module, an intelligence editing module, an intelligence library, an intelligence push module, a monitoring host and a handheld mobile terminal; the intelligence management system is aimed at ordinary construction workers. Before the workers go out to work, professional and technical personnel complete the intelligence search and intelligence editing process, obtain the optimal operation method and process and transmit it to the handheld mobile terminal of the operator, so that the operator does not need to have any professional knowledge of library intelligence, and also allows the operator to carry the handheld mobile terminal to areas with underdeveloped networks for use. In addition, the intelligence push module regularly pushes the tendency of intelligence according to the user's preferences, so that the technicians can keep abreast of the latest intelligence information related to the industry. This document is aimed at ordinary construction workers with low cultural level. Professional and technical personnel complete intelligence search and editing before going out to work, and transmit the optimal operation method to the handheld mobile terminal of the operator, so that the operator does not need to have professional knowledge of library intelligence. The patent does not describe the specific content of the collection in detail, but emphasizes that professional and technical personnel complete intelligence search and editing before going out to work. The patent emphasizes that professional technicians complete the search and editing of intelligence before going out for work, and transmit the optimal operation method to the operator's handheld mobile terminal. The patent does not describe the specific means of analysis in detail, but mentions that professional technicians complete the search and editing of intelligence.

[0004] Comparative document CN115994693A A visual analysis system based on a subject evaluation report includes a data visualization module, which is connected to a data preprocessing module through a network transmission node, and the data visualization module is connected to a client module, and the client module includes a background management client and a subject evaluation client. The present invention uses a powerful visual data analysis platform to assist manual operation to perform correlation analysis on data, analyze network data, text data, multidimensional data and time series data, and make a complete analysis chart, which contains relevant information of all events, and also fully displays the process of data analysis and the direction of data chain for relevant personnel to read. The purpose of this comparative document is to solve the shortcomings of the prior art that the data structure may not be unified, usually based on manual analysis, and the non-structural and uncertain analysis process, so it is not easy to form a fixed analysis process or mode, and it is difficult to transfer data into the application system for analysis and mining, and a visual analysis system based on subject evaluation reports is proposed, and the meaning of subject evaluation reports is not specified in the document. And the system described in the document CN115994693A is not targeted at data, but more generalized; it is impossible to prepare and process data. Summary of the invention

[0005] The purpose of the present invention is to provide a network-based subject intelligence data collection and analysis system and method thereof, which can acquire, process, analyze and display subject intelligence data from multiple network information source websites, including the current status of subject construction, top talents in the industry and key industry information, etc. The method can realize the effective acquisition, processing, analysis and display of subject intelligence data, and provide quantitative decision-making support for the hospital's subject construction and development.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] A network-based subject intelligence data collection and analysis system, including the following contents:

[0008] It includes a data acquisition module, a data storage module, a data preprocessing module, a data processing module, a data analysis module, a data display module, a cloud service host and a display terminal, wherein the modules are closely connected to ensure efficient and seamless information flow, the data storage module stores data at different stages, and includes a data lake module, an intelligence database module and a statistical analysis result library module, the data acquisition module is responsible for collecting original subject intelligence data from various network resources or other external data sources, the data acquisition module is closely connected to the data lake module, the data acquisition module stores the collected original intelligence data in the data lake module in its entirety according to the data source format, the data lake module is closely connected to the data preprocessing module, the preprocessing The module includes three submodules, namely, data extraction submodule, data cleaning submodule and data conversion submodule. The data extraction submodule automatically extracts intelligence data from the data lake module. The data output by the extraction submodule flows into the data cleaning submodule. The data cleaning submodule cleans the extracted data to ensure the quality of the data. The cleaning process includes data format checking, data denoising and data standardization. The cleaned data is input into the data conversion submodule. The data conversion submodule converts the cleaned data. Data conversion includes but is not limited to data feature extraction and data format conversion. The data converted by the data conversion submodule flows into the intelligence data storage module to become the initial version of the intelligence database. The intelligence data storage module is closely connected with the intelligence data processing module, and the intelligence data processing module includes an intelligence data summarization and fusion algorithm submodule, a data supplementation submodule and a data verification submodule. The intelligence data summarization and fusion algorithm submodule extracts the data required for analysis from the initial version of the intelligence database for multi-dimensional fusion processing. The data output by the intelligence data summarization and fusion algorithm submodule can be supplemented by the data supplementation submodule to obtain the complete data required for analysis. The data supplementation submodule can select and add from the initial version of the intelligence data or import from the outside. The data output by the data integration submodule and the data supplementation submodule needs to be confirmed by the data verification submodule for quality. The output result of the data verification submodule flows into the intelligence data storage module and becomes the final version of the intelligence data. Both the initial version and the final version of the intelligence database need to be associated with the metadata submodule. The metadata submodule defines and manages metadata. Metadata can describe the attributes, content, subject, source, format, time and space of intelligence information objects. Both the initial version and the final version of the intelligence database need to be connected to the backup submodule for data backup. The final version of the intelligence database is input into the data statistical analysis module. The data statistical analysis module performs in-depth analysis and statistics on the final version of the intelligence database to provide comprehensive insights into the subject area and extract key information.The data statistics and analysis module is closely connected with the data display module. The data display module displays the statistical analysis results of the data statistics and analysis module. The data display module presents the data analysis results in a graphical or other easy-to-understand form. All modules are connected to the cloud service host. The cloud service host serves as the basic architecture of the system, connecting all modules, providing data storage and computing services, overall system coordination and task scheduling, and processing user requests. The display terminal provides an interface for users to interact with the system, allowing users to query, configure parameters and view data analysis results. The display terminal is connected to the service host, and triggers the various modules of the system to perform corresponding tasks through user input.

[0009] The modules include:

[0010] (1) Data collection module, used to collect subject intelligence data from preset information source websites, including hospital comprehensive ranking and specialist ranking data, expert information data and industry key information data, and various public authoritative websites or authorized access websites accessible through the Internet. The authoritative websites provide authoritative policy guidelines, industry consulting information, hospital authoritative rankings, industry talent information, scientific research results, operational performance and other data. The collection method can be full collection or incremental collection, and the collection frequency can be set according to actual needs, such as real-time collection, scheduled collection, etc.

[0011] (2) The data lake submodule is used to effectively store and manage the original subject intelligence data obtained from the data collection module, and supports various data formats, including text, images, videos, etc. It is closely related to the metadata submodule and manages the metadata of subject intelligence data, including information such as data source, collection time, and format.

[0012] (3) Data preprocessing module is the module that performs the first data processing after data collection. It will correctly extract and store relevant data according to the requirements of the subject intelligence system, including:

[0013] (3.1) Data extraction: Identify content related to subject intelligence from crawled data through custom rules. Parse crawled data through natural language processing and OCR (Optical Character Recognition), and use text parsing technology, regular expressions or rule-based methods to extract required data fields, such as hospital name, ranking, score, expert name, affiliated hospital, professional title, position, subject direction, information title, release date, abstract, etc. Use keywords, specific patterns or structures to filter and extract data such as comprehensive hospital rankings, specialty rankings, expert information and industry information.

[0014] (3.2) Data cleaning: Clean and process data obtained from different sources to ensure data quality and consistency. Handle missing values, outliers, and duplicates to ensure data integrity and accuracy. Standardize data formats and naming conventions for subsequent analysis and processing.

[0015] (3.3) Data conversion: Convert and integrate the cleaned data to adapt it to the unified data model of the system. Adjust, merge and aggregate the data structure to facilitate analysis and query. This may involve operations such as data type conversion, encoding conversion or date and time format standardization.

[0016] (4) The data storage module consists of five sub-modules, namely, the data lake sub-module, the intelligence database sub-module, the statistical analysis result database sub-module, the metadata sub-module, and the backup sub-module.

[0017] (4.1) Metadata management module:

[0018] The metadata management module is an important part of the data storage module, responsible for defining and managing metadata related to subject intelligence data. The metadata is used to describe the attributes and characteristics of intelligence data. It provides detailed information about intelligence data, including key information such as the attributes, content, subject, source, format, time and space of intelligence information objects. The metadata management module is closely associated with the initial version and the final version of the intelligence database to ensure the integrity and traceability of the data.

[0019] (4.2) Intelligence database module

[0020] The initial version of the intelligence database is included. The initial version of the intelligence database module receives the cleaned and converted subject intelligence data output from the data conversion submodule. This module stores the converted data and becomes the initial version of the intelligence database. The initial version of the intelligence database is associated with the metadata management module to ensure that each type of information has corresponding metadata for subsequent retrieval and management.

[0021] Contains the final version of the intelligence database, which receives the subject intelligence data that has passed quality verification output from the data verification submodule. This is the final version of the system after data induction, fusion, supplementation and verification. Associated with the metadata management module, the data of the final version of the intelligence database needs to be fully verified and reviewed to ensure the high quality and credibility of the data.

[0022] (4.3) Backup submodule:

[0023] The backup submodule is connected to the initial version and the final version of the intelligence database, and is responsible for regular data backup to deal with potential data loss risks. Through the backup submodule, the system can quickly restore data in unexpected situations to ensure the security and reliability of subject intelligence data. This module ensures that a usable and complete version of the intelligence database can be accessed at any time.

[0024] (5) Data processing module, used to summarize, integrate, supplement and verify the acquired subject intelligence data, including:

[0025] (5.1) Intelligence data summarization and fusion algorithm module, including data mapping processing, summarization and fusion processing and data extraction, is used to integrate the extracted subject intelligence data, compare, remove duplicates, merge and other operations on data from different information source websites to form a unified data format and structure;

[0026] Following are the main features of the data processing module:

[0027] Discipline intelligence data collection, induction and fusion algorithm: The data statistical analysis module integrates advanced analysis algorithms and methods, including data mining, machine learning, statistical analysis, etc., to deeply explore the potential information and trends in the final version of the intelligence database.

[0028] The details are as follows:

[0029] The intelligence data induction and fusion algorithm includes data mapping stage, induction and fusion stage and extraction stage;

[0030] The mapping stage specifically includes:

[0031] (1) Data source registration: Create a data source list, including the URL (Uniform Resource Locator) of each type of data, update frequency, and metadata of the data format;

[0032] (2) Map task allocation: A Map task is allocated to each data source. The task is responsible for regularly accessing the data source and calling web crawler technology or API interface to pull data;

[0033] (3) The Map task performs preliminary mapping of the raw data, including: data identification: assigning a unique identifier to each piece of raw data, metadata annotation: recording metadata of the data source and collection time, and format conversion: converting the data into a unified intermediate format;

[0034] Local storage: Each Map task temporarily stores the mapped data locally, preparing to enter the next stage;

[0035] The induction and fusion stage specifically includes:

[0036] (1) Data sharding: The data generated in the Map phase is logically sharded in preparation for the Merge phase.

[0037] (2) Task scheduling: Start a Merge (data fusion) task for each shard, which is responsible for processing all the data in the shard;

[0038] (3) Data deduplication and merging: Within each shard, data is deduplicated to ensure uniqueness and identical records are merged.

[0039] (4) Data cleaning: Perform data cleaning to ensure data quality;

[0040] (5) Data conversion: Perform necessary conversions on the cleaned data to prepare for subsequent analysis;

[0041] The extraction stage specifically includes:

[0042] (1) Secondary Map Task: After data fusion, a secondary map task can be deployed to perform deep mapping of the data, focusing on information extraction;

[0043] (2) Information extraction rules: define a set of information extraction rules to identify and extract key information;

[0044] (3) Feature mapping: Each Map task maps data to specific analysis features according to the rules;

[0045] (5.2) Supplementary submodule, which is used to supplement the extracted and integrated subject intelligence data by accessing other data sources, such as the hospital's official website, academic journals and paper databases, etc., to obtain more data, such as detailed information of experts, hospital research results, academic papers, etc., and to associate the supplementary data with the original data;

[0046] (5.3) Verification submodule, which is used to verify the content of the extracted, integrated and supplemented subject intelligence data, and to ensure the quality and reliability of the data by checking and comparing the consistency, accuracy, completeness and timeliness of the data;

[0047] (6) The data statistical analysis module is responsible for in-depth analysis and statistics of the final version of the intelligence database to provide comprehensive insights into the subject area and extract key information.

[0048] Key indicator extraction: This module can extract key performance indicators in subject areas, such as hospital ranking trends, expert activity, key industry information, etc., to provide comprehensive subject intelligence insights.

[0049] Customized analysis reports: The data statistical analysis module generates customized analysis reports, including charts, trend analysis, correlation analysis, etc., to meet users' diverse needs for subject intelligence.

[0050] Real-time analysis and monitoring: This module supports real-time data analysis and monitoring, enabling users to quickly obtain the latest information and adjust decisions and strategies in a timely manner.

[0051] The results of the statistical analysis are stored in the statistical analysis result database in the data storage module.

[0052] (7) Data display module, used to display the analysis results in a visual form, including:

[0053] (7.1) Chart submodule, used to display the analysis results in the form of charts, including but not limited to bar charts, line charts, pie charts, radar charts, scatter charts, etc. According to different analysis purposes and data characteristics, select appropriate chart types to present the distribution, changes, relationships, proportions, etc. of data in an intuitive and beautiful way;

[0054] (7.2) Report submodule, which is used to present the analysis results in the form of reports, including but not limited to text, pictures, tables, formulas, etc. According to different analysis purposes and data characteristics, the appropriate report format is selected to present the overview, details, conclusions, suggestions, etc. of the data in a clear and standardized manner;

[0055] Furthermore, the application scenarios of the present invention include the following:

[0056] (1) Hospital management: Assist hospital managers in discipline construction planning, resource allocation, talent introduction, performance appraisal, etc.

[0057] (2) Scientific research decision-making: Provide scientific researchers with references such as discipline development trends, information on experts and scholars, and the latest industry dynamics to assist in scientific research decision-making.

[0058] (3) Talent development: Provide medical students with information on the direction of academic development, research fields of experts and scholars, and help employees clarify their learning goals and plan their career development.

[0059] Furthermore, the data sources collected by the data collection module include policy guidelines, industry consultations, authoritative hospital rankings, hospital official websites, industry talent information, and hospital scientific research, clinical and performance data, which can provide complete data required for subject intelligence analysis.

[0060] Furthermore, the data cleaning submodule includes data format checking, data denoising and data standardization.

[0061] Furthermore, the data conversion submodule includes data feature extraction and data format conversion.

[0062] Furthermore, the extraction submodule can extract formatted intelligence information from the acquired original intelligence information in various formats through natural language processing and OCR (Optical Character Recognition), etc., and is characterized in that the system is dynamic and can adapt to the ever-changing data environment and user needs to ensure the accuracy of the extracted intelligence data.

[0063] Furthermore, the collected original intelligence data, after extraction, cleaning and conversion, forms the initial version of the intelligence database, which is further summarized and integrated, supplemented and verified, to form the final version of the intelligence database. It is characterized in that the process from the initial version of the intelligence database to the final version of the intelligence database can be carried out according to dimensions such as subject and time, and the data of each dimension can also be iteratively updated. The subject here can be ranking, talent, reputation, innovation, scientific research, etc.

[0064] In the present invention, data processing is divided into four stages. The first stage is used to collect raw intelligence data and store it in the data lake. The second stage cleans the data in the data lake and stores the output results in the intelligence database. The third stage summarizes and integrates the cleaned results to output accurate and complete intelligence knowledge. The fourth stage performs big data statistical analysis on the intelligence data and stores the results of the statistical analysis in the statistical analysis library. Big data analysis is performed on the established discipline intelligence data to establish a discipline portrait. The discipline analysis and discipline development can be analyzed from multiple dimensions to provide a basis for the hospital to formulate a scientific discipline construction plan. The system provides multi-level and multi-dimensional data display so that users can have a deep understanding of the data from different angles. Charts, visualization tools, and reports can all be used to present complex analysis results.

[0065] The user groups of the present invention include senior managers, researchers, scholars or professional users who need in-depth data analysis in hospitals and scientific research institutes. The system may provide more advanced intelligent analysis and decision support; the collected data is clearer, and the data related to discipline construction is mainly collected. The data includes but is not limited to policy guidelines, industry consultations, authoritative hospital rankings, hospital official websites, industry talent information, and hospital scientific research, clinical and performance data, etc.; the system emphasizes the intelligence of the data processing stage, and can process a large amount of discipline intelligence data more comprehensively and deeply through the discipline intelligence data induction and fusion algorithm; more advanced data analysis methods are used, such as machine learning algorithms, deep learning, etc., to provide deeper insights and intelligent decision support; in terms of data display, more emphasis is placed on multi-level and multi-dimensional presentation, and more intuitive analysis results are provided to users through charts, visualization tools, etc.

[0066] The beneficial effects of the present invention are:

[0067] (1) The present invention can realize the comprehensive, accurate and timely collection, analysis and display of the current status of discipline construction, top talents in the industry and key industry information through data induction and fusion algorithms, providing strong support and guidance for the discipline construction and development of hospitals;

[0068] (2) The present invention can acquire, process, analyze and display subject intelligence data from multiple information source websites, overcoming the limitations and shortcomings of the prior art and improving the coverage and quality of data;

[0069] (3) The present invention can analyze the big data of subject intelligence data through data induction and fusion algorithms, providing multi-dimensional and multi-angle data analysis results, thereby enhancing the value and significance of the data;

[0070] (4) The present invention can display the analysis results in a visual form, which improves the readability and comprehensibility of the data and facilitates users' viewing and use;

[0071] (5) The present invention can store the acquired subject intelligence data and analysis results in a local knowledge base, thus ensuring the security of the data. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] Figure 1 Schematic diagram of the system structure of the present invention.

[0073] Figure 2 Storage structure diagram.

[0074] Figure 3 The main subject intelligence information collected.

[0075] Figure 4 The working principle diagram of the present invention.

[0076] Figure 5 Flowchart of data induction and fusion algorithm. DETAILED DESCRIPTION

[0077] The present invention relates to a network-based subject intelligence data collection and analysis system and method, specifically, a system and method that can acquire, process, analyze and display subject intelligence data from multiple information source websites, and is suitable for subject construction and development planning in hospitals and scientific research institutes.

[0078] In order to fully explain the purpose, core features and significant advantages of the present invention, the following content will be integrated with the drawings to elaborate on the specific implementation scheme of the present invention, aiming to provide a thorough and easy-to-understand description. It should be clear that the content described here represents part of the many possible implementation schemes of the present invention and does not cover all its application scope or variations.

[0079] like Figure 1 As shown, a network-based subject intelligence data collection and analysis system includes a data collection module, a data preprocessing module, a data processing module, a data analysis module, a data display module, a cloud service host and a display terminal, which are respectively connected to the data storage module;

[0080] like Figure 2 As shown, the implementation method of the network-based subject intelligence data collection and analysis system is as follows:

[0081] Data collection module: First, the system automatically or manually collects intelligence information of relevant disciplines from multiple sources through the data collection module. These data may come from public databases, academic publications, network resources, etc.

[0082] Data preprocessing module: The collected data usually needs to be preprocessed to remove irrelevant or erroneous information, unify the format, or do preliminary sorting so that subsequent steps can be processed more efficiently. This step ensures the quality and consistency of the data.

[0083] Data storage module: The pre-processed and final processed data are transmitted and stored in a dedicated data storage module. This module is responsible for managing and maintaining the security, reliability and efficient access of data, providing a stable data foundation for subsequent analysis. The data of statistical analysis results are also stored in the storage module.

[0084] Data processing module: The stored data then enters the data processing module, where data induction and fusion algorithms are used for deeper processing, such as data conversion and feature extraction, in preparation for analysis.

[0085] Data Analysis Module: Next, the system uses advanced analytical algorithms and techniques to deeply explore patterns, trends and associations in the data in the Data Analysis Module to extract key insights into subject intelligence.

[0086] Data display module: The analysis results are presented to users in the form of charts, reports or other visual forms through the data display module. This makes complex data analysis results easy to understand and helps decision makers quickly grasp the dynamics and trends of the discipline.

[0087] Cloud service host: The entire system relies on the cloud service host to run, using the elastic computing power and large-scale storage advantages of cloud computing to ensure that the system can efficiently and flexibly serve a large number of users and support remote access.

[0088] Display terminal: Ultimately, users access the system through various display terminals (such as computers, tablets or smart phones), view customized subject intelligence analysis reports, conduct interactive exploration or download and share, and promote knowledge exchange and decision-making.

[0089] Among them, Figure 4 As shown, the data storage module includes a data lake module, an intelligence database module and a statistical analysis result library module. The data acquisition module is responsible for collecting original subject intelligence data from the data source. The data acquisition module is connected to the data lake module. The data acquisition module stores the collected original intelligence data in the data lake module in full according to the data source format. The intelligence database module includes an initial version of the intelligence database and a final version of the intelligence database.

[0090] The data lake module is connected to the data preprocessing module, which includes a data extraction submodule, a data cleaning submodule and a data conversion submodule. The data extraction submodule automatically extracts intelligence data from the data lake module. The data output by the data extraction submodule flows into the data cleaning submodule. The data cleaning submodule cleans the extracted data. The cleaned data flows into the data conversion submodule. The data conversion submodule converts the cleaned data. The data converted by the data conversion submodule flows into the intelligence data storage module to become the initial version of the intelligence database.

[0091] The data storage module is closely connected with the data processing module. The data processing module includes an intelligence data summarization and fusion algorithm, a data supplementation submodule and a data verification submodule. The intelligence data summarization and fusion algorithm submodule extracts the data required for analysis from the initial version of the intelligence database. The data output after multi-dimensional fusion processing can be supplemented by the data supplementation submodule to obtain the complete data required for analysis. The data output by the data supplementation submodule needs to be quality verified by the data verification submodule. The output result of the data verification submodule flows into the intelligence data storage module to become the final version of the intelligence data.

[0092] The initial version of the intelligence database and the final version of the intelligence database are associated with the metadata submodule and the backup submodule respectively. The metadata submodule defines and manages metadata. The final version of the intelligence database is input into the data statistical analysis module. The data analysis module performs in-depth analysis and statistics on the final version of the intelligence database to provide comprehensive insights into the subject area and extraction of key information. The data statistical analysis module is closely connected with the data display module. The data display module displays the statistical analysis results of the data statistical analysis module. The data display module presents the data analysis results in graphical or other easy-to-understand forms.

[0093] Among them, Figure 3As shown, the present invention suggests that intelligence data should be collected from five aspects, namely, operational performance, scientific research results, innovation conversion, authoritative ranking and core talents, with disciplines as the core. Through systematic data integration, it will be possible to gain a more comprehensive and in-depth insight into the mutual influence between disciplines, and further reveal the inherent logic and potential trends of discipline development.

[0094] Among them, Figure 5 As shown, the intelligence data induction and fusion algorithm is divided into three stages: data mapping, data induction and fusion, and data extraction.

[0095] Mapping phase: This phase includes two steps: data source registration and Map task assignment. Data source registration is to create a list of metadata including URL, update frequency and data format. Map task assignment is to regularly access the data source, pull and preliminarily map the data (assign unique identifiers, record metadata, format conversion), and temporarily store it locally.

[0096] Inductive fusion phase: In this phase, data shards will be mapped, and the Merge task will be started to process the shard data, remove duplicates, merge identical records, ensure data quality, and perform necessary conversions.

[0097] Extraction phase: This phase is divided into two steps: secondary mapping task and information extraction rules and feature mapping. The secondary mapping task is to deeply map the data and focus on information extraction. Information extraction rules and feature mapping are to define rules to extract key information and map them into analysis features.

[0098] In summary, the present invention is a highly comprehensive network-based subject intelligence data collection and analysis system and method thereof, which realizes in-depth data support for hospitals and scientific research institutes in subject construction and development planning through carefully designed data collection, preprocessing, processing, analysis and display processes. With the help of intelligent data processing algorithms and the powerful power of cloud services, the present invention not only greatly improves the efficiency and quality of subject intelligence data management, but also enables decision makers to quickly grasp key information through intuitive data visualization, guide strategy formulation and scientific research direction adjustment. More importantly, the system integrates data from different sources and in various formats into standard and standardized data and stores them through data induction and fusion algorithms, and finally uses statistical analysis methods to analyze subject intelligence in multiple dimensions and in depth, providing a powerful tool for insight into academic dynamics and prediction of development trends.

[0099] Implementation Cases:

[0100] 1. Data Collection

[0101] 1.1 Determine the data source

[0102] According to the analysis requirements, determine the sources of subject intelligence data that need to be collected, including:

[0103] Comprehensive hospital rankings and specialty ranking websites: for example, the "China Hospital Ranking" published by the Hospital Management Institute of Fudan University and the "China Hospital Science and Technology Influence Ranking" published by the Chinese Institute of Scientific and Technological Information.

[0104] Expert information websites: for example, the National Science and Technology Expert Database, China National Knowledge Infrastructure Database, LinkedIn, etc.

[0105] Industry information websites: for example, Medical World, Dingxiangyuan, Yimaitong, etc.

[0106] 1.2 Data Collection

[0107] Use web crawler technology or other data collection tools to obtain subject intelligence data from preset information source websites, including: Hospital ranking: hospital name, ranking, specialty ranking, subject leader, hospital address, contact information, etc.

[0108] Expert information: expert’s name, affiliation, title, research direction, academic achievements, contact information, etc.

[0109] Industry information: industry news, policies and regulations, academic research, conference activities, market analysis, etc.

[0110] 1.3 Data Preprocessing

[0111] Clean and process the collected raw data, including:

[0112] · Remove duplicate data: Use hashing algorithms and other methods to remove duplicate data.

[0113] ·Null value processing: interpolate or delete null values.

[0114] Formatting: Format data uniformly, such as formatting dates into a uniform format.

[0115] 2. Data processing

[0116] 2.1 Data Integration

[0117] Integrate subject intelligence data from different data sources into a unified data format and structure. For example, integrate hospital ranking data, expert information data, and industry information data into one database.

[0118] 2.2 Data Supplement

[0119] By accessing other data sources, such as the hospital's official website, academic journals and paper databases, more data can be obtained, such as detailed information on experts, hospital research results, academic papers, etc., and the supplementary data can be associated with the original data. For example, information such as the hospital's discipline construction plan and talent introduction policy can be obtained from the hospital's official website.

[0120] 2.3 Data Verification

[0121] Verify the integrated and supplemented subject intelligence data to ensure the accuracy, completeness, consistency and timeliness of the data. For example, verification can be carried out through manual verification, data comparison and other methods.

[0122] 3. Data Analysis

[0123] 3.1 Data Mining

[0124] Use data mining techniques to discover potential knowledge and patterns from subject intelligence data, such as:

[0125] Hospital ranking change trends: Analyze the changing trends of hospital rankings and find out the factors that affect hospital rankings.

[0126] Discipline portrait: Analyze the characteristics of the hospital discipline portrait and find out the key influencing factors of the discipline portrait.

[0127] Expert research direction: Analyze the papers and patents published by experts and scholars to find out their main research directions.

[0128] Industry development rules: Analyze industry news, policies and regulations to find out the trends and rules of industry development.

[0129] 3.2 Machine Learning

[0130] Use machine learning technology to build subject intelligence analysis models for:

[0131] Predict hospital ranking: Predict the hospital's future ranking based on factors such as the hospital's discipline construction, talent team, and scientific research results.

[0132] ·Evaluate experts and scholars: Evaluate the academic level of experts and scholars based on their academic achievements, research influence and other factors.

[0133] Analyze industry trends: Analyze industry development trends and patterns based on industry news, policies and regulations, and other data.

[0134] 3.3 Statistical analysis

[0135] Use statistical analysis methods to conduct statistical analysis on subject intelligence data, such as:

[0136] Calculate the mean, standard deviation, trend, etc. of hospital rankings to analyze the overall level of hospital discipline construction.

[0137] Analyze the activity and research results of experts and scholars, and evaluate their academic contributions.

[0138] ·Evaluate industry development status, analyze industry competition pattern, market size, etc.

[0139] 4. Data display

[0140] 4.1 Visualization

[0141] Use visualization methods such as charts, tables, and maps to intuitively present analysis results to users, making it easier for them to understand and analyze. For example, use a line chart to show the changing trend of hospital rankings, use a bar chart to show the research results of experts and scholars, and use a map to show the development distribution of an industry.

[0142] 4.2 Interactive Analysis

[0143] Provide interactive analysis functions, allowing users to perform data query, filtering, drilling and other operations according to their needs to obtain more in-depth analysis results. For example, users can query hospital rankings based on department, region and other conditions, query experts and scholars based on research direction, publication time and other conditions, and query industry information based on industry type, geographical distribution and other conditions.

[0144] 5. Application

[0145] 5.1. Hospital management

[0146] · Assist hospital managers in discipline construction planning, resource allocation, talent introduction, performance appraisal, etc. For example:

[0147] · According to the changing trend of hospital rankings, formulate discipline construction plans, focus on developing advantageous disciplines and improve weak disciplines.

[0148] ·Introduce talents according to the research directions of experts and scholars and strengthen the training of discipline leaders.

[0149] Adjust the discipline layout and optimize resource allocation according to the development rules of the industry.

[0150] ·Based on the results of performance appraisal, we will reward the advanced and encourage the laggards to improve the level of discipline construction.

[0151] 2. Scientific research decision-making

[0152] Provide scientific researchers with references such as discipline development trends, information on experts and scholars, and the latest industry news to assist in scientific research decision-making.

[0153] For example:

[0154] Determine research directions and select appropriate scientific research projects based on discipline development trends.

[0155] ·Based on the information of experts and scholars, find partners and form a scientific research team.

[0156] ·According to the latest industry trends, understand industry development needs and conduct technological innovation.

[0157] 3. Talent cultivation

[0158] Provide scientific researchers and medical students with information on discipline development directions, research fields of experts and scholars, etc., to help them clarify their learning goals and plan their career development.

[0159] The present invention is not limited to the specific technical solutions described in the above implementation cases. In addition to the above embodiments, the present invention may also have other implementation methods. For those skilled in the art, any technical solutions formed by any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A network-based subject intelligence data collection and analysis system, characterized in that: It includes a data acquisition module, a data storage module, a data preprocessing module, a data processing module, a data analysis module and a data display module; wherein the data storage module includes a data lake module, an intelligence database module and a statistical analysis result database module, The data acquisition module collects original subject intelligence data from the data source; the data acquisition module is connected to the data lake module, and the data acquisition module stores the collected original intelligence data in the data lake module in full according to the data source format; The data lake module is connected to the data preprocessing module, which includes a data extraction submodule, a data cleaning submodule and a data conversion submodule. The data extraction submodule automatically extracts intelligence data from the data lake module. The data output by the data extraction submodule flows into the data cleaning submodule. The data cleaning submodule cleans the extracted data. The cleaned data flows into the data conversion submodule. The data conversion submodule converts the cleaned data. The data converted by the data conversion submodule flows into the intelligence data storage module to become the initial version of the intelligence database. The data storage module is closely connected with the data processing module. The data processing module includes an intelligence data summarization and fusion algorithm submodule, a data supplementation submodule and a data verification submodule. The intelligence data summarization and fusion algorithm submodule extracts the data required for analysis from the initial version of the intelligence database and performs multi-dimensional fusion processing. The data output by the multi-dimensional fusion processing can be supplemented by the data supplementation submodule to obtain the complete data required for analysis and statistics. The data output by the data supplementation submodule needs to be confirmed by the data verification submodule for data quality. The output result of the data verification submodule flows into the intelligence data storage module to become the final version of the intelligence data. The initial version of the intelligence database and the final version of the intelligence database are respectively associated with a metadata submodule and a backup submodule, and the metadata submodule defines and manages metadata; The final version of the intelligence database is connected to the data statistical analysis module, and the data analysis module performs in-depth analysis and statistics on the final version of the intelligence database to provide comprehensive insights into the subject area and extraction of key information. The data statistical analysis module is closely connected with the data display module, and the data display module displays the statistical analysis results of the data statistical analysis module. The data display module presents the data analysis results in a graphical or other easy-to-understand form.

2. A network-based subject intelligence data collection and analysis system according to claim 1, characterized in that: The data sources collected by the data collection module include policy guidelines, industry information, authoritative hospital rankings, hospital official websites, industry talent information, and hospital scientific research, clinical and performance data, which can provide complete data required for subject intelligence analysis.

3. A network-based subject intelligence data collection and analysis system according to claim 1, characterized in that: The data cleaning submodule includes data format checking, data denoising and data standardization.

4. The network-based subject intelligence data collection and analysis system according to claim 1, characterized in that: The data conversion submodule includes data feature extraction and data format conversion.

5. A network-based subject intelligence data collection and analysis system according to claim 4, characterized in that: The data extraction submodule can extract formatted intelligence information from the acquired original intelligence information in various formats through natural language processing and optical character recognition.

6. A network-based subject intelligence data collection and analysis system according to claim 5, characterized in that: From the first version of the intelligence database to the final version of the intelligence database, it is carried out according to each subject and time dimension, and the data in each dimension can be updated iteratively.

7. A network-based subject intelligence data collection and analysis system according to claim 1, characterized in that: The data display module provides multi-level and multi-dimensional data display so that users can gain in-depth understanding of the data from different angles. Charts, visualization tools and comprehensive reports can be used to present complex analysis results.

8. The network-based subject intelligence data collection and analysis system according to claim 1, characterized in that: The intelligence data induction and fusion algorithm in the data processing module includes a data mapping stage, an induction and fusion stage, and an extraction stage; The mapping stage specifically includes: (1) Data source registration: Create a data source list containing the URL, update frequency, and metadata of each type of data; (2) Map task allocation: Assign a Map task to each data source. The task is responsible for regularly accessing the data source and calling web crawler technology or API interface to pull data; (3) The Map task performs preliminary mapping of the raw data, including: data identification: assigning a unique identifier to each piece of raw data, metadata annotation: recording metadata of the data source and collection time, and format conversion: converting the data into a unified intermediate format; Local storage: Each Map task temporarily stores the mapped data locally, preparing to enter the next stage; The induction and fusion stage specifically includes: (1) Data sharding: The data generated in the Map phase is logically sharded and prepared to enter the Merge phase; (2) Task scheduling: Start a Merge (data fusion) task for each shard, which is responsible for processing all data in the shard; (3) Data deduplication and merging: Within each shard, data is deduplicated to ensure uniqueness and identical records are merged. (4) Data cleaning: Perform data cleaning to ensure data quality; (5) Data conversion: Perform necessary conversions on the cleaned data to prepare for subsequent analysis; The extraction stage specifically includes: (1) Secondary Map Task: After data fusion, a secondary map task can be deployed to perform deep mapping of the data and focus on information extraction; (2) Information extraction rules: define a set of information extraction rules to identify and extract key information; (3) Feature mapping: Each Map task maps data into specific analysis features according to the rules.

9. A computer system based on the network-based subject intelligence data collection and analysis system according to claims 1 to 8, characterized in that: Based on claims 1 to 8, it also includes a cloud service host and a display terminal. All modules of the system are connected to the cloud service host. The cloud service host serves as the basic architecture of the system, connects all modules, provides data storage and computing services, overall system coordination and task scheduling, and processes user requests. The display terminal provides an interface for users to interact with the system, allowing users to query, configure parameters and view data analysis results. The display terminal is connected to the service host, and triggers each module of the system to perform corresponding tasks through user input.

Images