User-oriented CSTR identification registration analysis log analysis method and system
By generating and analyzing CSTR identifier registration and parsing logs, combined with intelligent analysis algorithms, the problem of users having difficulty obtaining effective information from a large number of logs has been solved, enabling observable analysis of the entire identifier lifecycle and improving resource management efficiency and security.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-11
- Publication Date
- 2026-04-14
AI Technical Summary
In existing technologies, users find it difficult to intuitively obtain useful information from a large number of CSTR identifier registration and parsing logs, which makes it difficult to guarantee resource accessibility and security. In particular, non-technical personnel cannot track the failure of batch identifier registration and parsing in a timely manner.
This paper provides a user-oriented CSTR identifier registration and resolution log analysis method and system. By generating and analyzing identifier registration logs, resolution logs and resolution inspection logs, it generates user warning reports, uses multi-dimensional intelligent analysis algorithms to identify abnormal behavior, and combines spatiotemporal offset analysis and IP geographic information to identify abnormal access patterns, thereby achieving observable analysis of the entire identifier lifecycle.
It has significantly improved the sophistication of technology resource management, lowered the barrier to entry, enabled non-technical users to manage resource identifiers independently, improved resource accessibility and security, enhanced the platform's security and reliability, and promoted the continuous optimization of resource utilization efficiency.
Smart Images
Figure CN121864575A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of information technology and relates to identifier registration and parsing log analysis, specifically to a user-oriented CSTR identifier registration and parsing log analysis method and system. Background Technology
[0002] CSTR comes from my country's national standard GB / T 32843—2016 "Science and Technology Resource Identification" (CSTR for short). This standard specifies the objects and generation methods of science and technology resource identification, the structure and writing rules of the identifier, and the management and application of the identifier. It is applicable to the unified management of various science and technology resources, as well as the cataloging, registration, publication, query, maintenance and management of science and technology resources.
[0003] The CSTR identifier encoding structure is as follows: Figure 1 As shown, it includes four parts: the Chinese Science and Technology Resource Identifier Code, prefix, type code, and suffix. "CSTR:" is the Chinese Science and Technology Resource Identifier Code, "32128" is the prefix (one for each journal and data center), "32128" represents the Journal of the Chinese Academy of Sciences, and "CASbulletin.20240321003" is the suffix, defined by the user. The storage method for the CSTR identifier master node and child node identifier resolution addresses is as follows... Figure 2 As shown. The master node does not store the specific identifier address, but instead stores the addresses of the child nodes corresponding to the prefix. Each child node stores the identifier and its corresponding resource address.
[0004] The identifier registration and resolution log is a collection of operation log files generated during identifier registration and resolution in the identifier system. These include identifier registration success logs, identifier registration failure logs, identifier resolution success logs, resolution error logs, and resolution inspection logs. The logs are mainly used for identifier availability verification, data security auditing, statistical analysis, and system optimization to ensure the reliability, security, and efficiency of the identifier service.
[0005] Currently, users primarily entrust technology companies (e.g., journal users entrust Magtech, Renhe, etc.) to register identifiers. The backend management systems provided by these technology companies can only offer registration error messages and cannot directly confirm whether the identifier has been successfully resolved. Furthermore, if the URL entered during resource registration is invalid, the technology companies lack specific prompts for users. Users are mainly journal editors and data center administrators, not professional technical R&D personnel. They find it difficult to effectively extract information from large amounts of parsing logs, and cannot promptly track the specific circumstances of batch identifier registration failures and batch resource parsing failures, thus affecting resource accessibility. Summary of the Invention
[0006] To address the aforementioned problems, this invention provides a user-oriented CSTR identifier registration parsing log analysis method and system.
[0007] The technical solution adopted in this invention is as follows: A user-oriented CSTR identifier registration parsing log analysis method includes the following steps: Generate identifier registration logs, identifier resolution logs, and identifier resolution inspection logs; Analyze the identifier registration log, identifier resolution log, and identifier resolution inspection log to generate user warning reports.
[0008] Furthermore, the identifier registration log is generated using the following steps: The child node receives the user's identifier registration / modification request; The child nodes check whether there are any problems with the metadata submitted during registration; If there is a problem with the metadata, the user will be notified through the interface and the issue will be recorded in the registration / modification failure log. If there are no issues with the metadata, the registration is completed and feedback is sent to the user via the interface. Record it in the registration / modification success log.
[0009] Furthermore, the identifier resolution log is generated using the following steps: The child node receives the parsing request initiated by the user; The child node determines whether the identifier is present in the parsed data table; If the identifier is not in the parsed data table, the child node will directly return the error message to the user; After providing feedback to the user, the child node will record the corresponding information in the parsing log table; If the identifier is in the parsed data table, the child node will send the relevant information to the master node; The master node retrieves the identifier resolution address from the identifier information table and sends it to the child nodes; The child node will return the identifier resolution address to the user; The child nodes record the relevant information in the parsing log.
[0010] Furthermore, the identifier resolution inspection log is generated using the following steps: The master node obtains the identifier and resolved address from the child node; The master node summarizes the identifiers obtained from the child nodes; The master node accesses the resolved addresses of the identifiers sequentially; The master node records address access information and generates an identifier resolution inspection log.
[0011] Furthermore, the analysis of the identifier registration log, identifier resolution log, and identifier resolution inspection log includes: Clean and structure the data fields of the logs generated by the master node and child nodes to ensure that the log data of the same fields in the master node and child nodes have the same format. Complete the log data, including completing the identification details in the log and completing the parsed source address in the log; Perform spatiotemporal offset detection; Calculate the abnormal registration failure rate and parsing failure rate, and automatically generate user warning reports based on the abnormal registration failure rate and parsing failure rate and send them to the relevant users to improve user resource security and academic influence.
[0012] Furthermore, the spatiotemporal offset detection includes the following steps: Based on the information of the country, province, and city of origin and the resolution time in the previous successful and failed resolution logs of the prefix, a heat map of the prefix's identifier resolution is formed; Define the offset index as: number of exception parsings / total number of parsings; A parsing anomaly flag is determined if any of the following two conditions are met: a) the parsing IP is in the parsing IP blacklist; b) the flag has failed to resolve 5 times consecutively during parsing inspection; c) historical successful resolutions are concentrated on non-academic IPs; d) all flags under a prefix whose offset index exceeds a set threshold. If a parsing anomaly is identified, it will not be calculated or analyzed in subsequent statistics, and the organization and administrator of the flag will be notified.
[0013] Furthermore, the abnormal registration failure rate is calculated using the following formula: in, The number of registration failures is due to a technical issue. Total registrations The ratio; It represents the number of failures for similar resources. It represents the total number of attempts to find similar resources.
[0014] Furthermore, the resolution failure rate is calculated using the following formula: in, It is the effective failure rate of resources. This is the number of valid failures excluding normal network fluctuations. This represents the total number of parsing attempts; The failure time dispersion is represented and calculated as follows: in, For time entropy, For statistical purposes: number of days; time entropy The calculation method is as follows: ,in in, For the first time in the statistical period Total number of parsing failures occurring within a 3-hour time period; A collection of failure logs; It is the hour value extracted from the logs; The parsing failure event occurred in the first... The probability within a time period, when When =0, the term It is defined as 0 in the summation.
[0015] A user-oriented CSTR identifier registration and resolution log analysis system employing the above method includes a main node and sub-nodes; the main node includes an identifier resolution module, an identifier management module, and a data service module, which respectively implement identifier resolution function, identifier management function, and data service function; the sub-nodes include an identifier registration module, an identifier resolution module, and a data service module, which respectively implement identifier registration function, identifier resolution function, and data service function.
[0016] Furthermore, in the master node, the identifier resolution module includes an identifier resolution submodule and a log database resolution submodule; the identifier management module includes a user authentication submodule, a permission management submodule, a metadata management submodule, and a log management submodule; and the data service module includes a statistical analysis submodule, a user management submodule, a user report submodule, and an identifier query submodule. In the sub-nodes, the identifier registration module includes an identifier registration submodule and a registration log submodule; the identifier resolution module includes an identifier resolution submodule and a log database resolution submodule; and the data service module includes a log management submodule and a data synchronization submodule.
[0017] This invention, based on the Technology Resource Identifier (CSTR) system, significantly improves the precision of technology resource management and data service capabilities by deeply managing and intelligently analyzing log data generated during the CSTR identifier registration and resolution process, and establishing a user-oriented visual feedback mechanism. Specific beneficial effects include: (1) A dedicated analysis system for CSTR logs was established, filling a technological gap. For the first time, a dedicated BI analysis framework was proposed, which includes log generation, cleaning and completion, multi-dimensional modeling and automatic report generation. This framework enables observable analysis of the entire lifecycle behavior of CSTR identifiers and solves the pain point of lacking a dedicated method for identifier log analysis in the field of science and technology resources and relying on manual investigation by professionals.
[0018] (2) Significantly lowers the barrier to entry and empowers non-technical users to manage independently. For users who lack professional technical background, such as journal editors and data administrators, it provides intuitive multi-dimensional statistical reports and actionable optimization suggestions (such as "batch update of resolution address"), enabling them to effectively manage resource identifiers without understanding the underlying technical details, and greatly improving management efficiency.
[0019] (3) Several intelligent analysis algorithms are innovatively introduced to achieve accurate attribution and prediction. The time entropy-based analytical failure rate model effectively distinguishes between random failures and malicious attacks (such as web crawlers); the TF-IDF and cosine similarity-based anomaly registration detection model accurately identifies batch erroneous registration behavior and reduces the proportion of invalid resources; the spatiotemporal offset analysis model, combined with IP geographic information, identifies abnormal access patterns to prevent data crawling and unauthorized access. Through these models, intelligent classification and root cause localization of problems are achieved, improving operation and maintenance response speed and management efficiency.
[0020] (4) Build a trustworthy resource ecosystem and enhance platform security and reliability. By proactively monitoring abnormal indicators, failed resources and potential attack behaviors, and combining this with an early warning mechanism to promptly clean up risky nodes, the overall security level and resource trustworthiness of the CSTR system are significantly improved, ensuring legitimate and stable access to scientific and technological resources.
[0021] (5) Establish a closed-loop management mechanism to promote the continuous optimization of resource utilization efficiency. Through the full-process automated management of "monitoring-analysis-early warning-disposal-optimization", users can continuously improve resource accessibility and service quality, thereby enhancing their academic influence and platform stickiness. Attached Figure Description
[0022] Figure 1 This is an example diagram of the CSTR identifier structure.
[0023] Figure 2 This is a node storage architecture diagram.
[0024] Figure 3 This is the CSTR system architecture diagram.
[0025] Figure 4 This identifies the process of generating registration logs.
[0026] Figure 5 This is the process of generating identifier resolution logs.
[0027] Figure 6 This is the process of generating inspection logs by resolving identifiers.
[0028] Figure 7 This is the sample format for the user warning report in Example 1.
[0029] Figure 8 This is the email format shown in Example 2.
[0030] Figure 9 This is the format of the warning email in Example 3. Detailed Implementation
[0031] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to specific embodiments and accompanying drawings.
[0032] 1. CSTR Identification System Architecture CSTR system such as Figure 3 As shown, it includes a master node and child nodes, where the master node's functions include identifier resolution, identifier management, and data service functions.
[0033] 1.1 Master Node Functions The master node includes an identifier resolution module, an identifier management module, and a data service module, which respectively implement identifier resolution functions (identifier resolution and resolution logs), identifier management functions (user authentication, permission management, metadata management, and log management), and data service functions (statistical analysis, user management, user reports, and identifier query).
[0034] 1.1.1 Identifier Resolution Module The identifier resolution module comprises two sub-modules: identifier resolution and resolution log library. The identifier resolution sub-module allows users to perform resolution via the main node page or by specifying a resolution address. The resolution log library sub-module stores log data for successful resolution, failures, and resolution inspections. The log storage data fields and structure are as follows: The identifier resolution success log mainly stores the system-generated log information when a user successfully resolves an identifier. The specific data format is: <Database Primary Key ID: id; uuid: uuid; CSTR Identifier: CSTR; Identifier Prefix: prefix; Resource Name: {Chinese: resource name; English: resource name}; Chinese Keywords: {keyword1, keyword2}; English Keywords: {keyword1, keyword2}; Associated Identifier: {resource type: resource identifier}; Resource Ownership Organization Name: Organization Name: Resolution Address: {Country: Country Name; Province: Province Name; City: City Name}; Resolution Time: Time; Host Address: Host IP; Source Domain: https: / / xxxxx; Resolution IP: IP address; Resolution Node ID: Internal ID encoding; Resolution Resource Type: Resource Type Name>.
[0035] The identifier resolution failure log data field mainly stores the system-generated log information when a user's resolution fails. The specific data format is <Database primary key ID: id; uuid: uuid; CSTR identifier: CSTR; Identifier prefix: prefix; Resolution address: {Country: Country name; Province: Province name; City: City name}; Resolution time: Time; Host address: Host IP; Source domain name: https: / / xxxxx; Resolution IP: IP address; Resolution node id: Internal id encoding; Resolution resource type: Resource type name; Registration status: Status code>.
[0036] The identifier resolution inspection log data mainly stores the log information generated by the system during the resolution inspection. The specific data format is <Database primary key ID: id; uuid: uuid; CSTR identifier: CSTR; Identifier prefix: prefix; Identifier resolution address: {resolution address1, resolution address2}; Inspection time: time; Resolution status: status code (success / failure); Inspection problem found: problem code>.
[0037] 1.1.2 Identification Management Module The identifier management module includes four sub-modules: user authentication, permission management, metadata management, and log management.
[0038] The user authentication submodule is mainly for authenticating users who obtain statistical analysis data, ensuring that user statistical data is not obtained by other personnel.
[0039] The access control submodule primarily manages node and user permissions. Participants include super administrators, node administrators, organization administrators, and general users. Super administrators are system administrators who primarily interact with master and child nodes, supporting node and database maintenance and management, including adding / removing nodes and managing IP whitelists and blacklists. Node administrators maintain and manage the database within their nodes. Organization administrators can query all real-time statistical data under their organization's prefix; these are primarily data center or journal administrators. General users are mainly researchers and only support node-based identifier resolution.
[0040] The metadata management submodule mainly supports the management of metadata submitted during identifier registration and the management of dictionary values filled in during registration.
[0041] The log management submodule primarily supports unified management of logs for both master and child nodes.
[0042] 1.1.3 Data Service Module The data service module comprises four sub-modules: statistical analysis, user management, user reports, and identifier query.
[0043] The statistical analysis submodule mainly supports the summarization, processing, statistics and analysis of log data from each sub-node; The user management submodule mainly supports the management of basic information and contact details of users in the main node and child nodes; The user report submodule mainly supports summarizing the data processed by the statistical analysis submodule to form regular user reports and emergency warning reports, which are then sent to a designated email address. The identifier query submodule mainly supports users in querying identifier details.
[0044] 1.2 Child Node Functions The sub-nodes include an identifier registration module, an identifier resolution module, and a data service module, which respectively implement identifier registration functions (identifier registration and registration logs), identifier resolution functions (identifier resolution and resolution logs), and data service functions (log management and data synchronization).
[0045] 1.2.1 Identifier Registration Module The identifier registration module comprises two sub-modules: identifier registration and registration log. The identifier registration sub-module allows users to register identifiers after authentication, with registration methods including but not limited to online registration, batch registration, and API registration. The registration log sub-module stores log data generated during successful and failed registrations. The log storage data fields and structure are as follows: The registration success log mainly stores logs generated when a user successfully registers. Specific fields include: <Resource Name; {Chinese: Resource Name; English: Resource Name}; English Summary; Resource Type: Resource Type; Chinese Keywords: {Keyword 1, Keyword 2}; English Keywords: {Keyword 1, Keyword 2}; Author List: {(Author Name 1, Language, Author Number, Author Email, Author Affiliation List: {Affiliation 1, Affiliation 2}); (Author Name 2, Author Name Language, Author Number, Author Email, Author Affiliation List: {Affiliation 1, Affiliation 2}); Identifier Resolution Address: {Resolution Address 1, Resolution Address 2}; Registration Time: Registration Time}>.
[0046] The registration failure log mainly stores logs generated when user registration fails. Specific fields are in the following format: <Resource Name; {Chinese: Resource Name; English: Resource Name}; English Summary; Resource Type: Resource Type; Chinese Keywords: {Keyword 1, Keyword 2}; English Keywords: {Keyword 1, Keyword 2}; Author List: {(Author Name 1, Language, Author Number, Author Email, Author Affiliation List: {Affiliation 1, Affiliation 2}); (Author Name 2, Author Name Language, Author Number, Author Email, Author Affiliation List: {Affiliation 1, Affiliation 2}); Identifier Resolution Address: {Resolution Address 1, Resolution Address 2}; Registration Time: Registration Time}; Failure Reason: Failure Reason Code>.
[0047] 1.2.2 Identifier Resolution Module The identifier resolution module consists of two sub-modules: identifier resolution and resolution log. Its main functions are consistent with the master node. The log stores data generated by successful and failed resolution, but does not include identifier resolution inspection logs.
[0048] 1.2.3 Data Service Module The data service module mainly consists of two sub-modules: log management and data synchronization. The log management sub-module primarily supports unified management of logs from child nodes. The data synchronization sub-module primarily supports data parsing and synchronization between the master node and child nodes, ensuring that child nodes can correctly parse all identifiers within the system.
[0049] 2. Identifier Log Generation Method 2.1. Identifier Registration Log Generation Method When a user actively requests to register or modify an identifier, logs are generated indicating successful and failed registration / modification attempts. The identifier registration log generation process is as follows: Figure 4 As shown, it includes the following steps: (1) The user sends an identifier registration / modification request to the child node; (2) The child node checks whether there are any problems with the metadata submitted during registration; (3) If there is a problem with the metadata, the user will be notified through the interface and the issue will be recorded in the registration / modification failure log; (4) If there are no problems with the metadata, complete the registration and provide feedback to the user through the interface; (5) Record it in the registration / modification success log.
[0050] 2.1. Identifier Resolution Log Generation Method User identifier resolution logs primarily originate from user-initiated requests to resolve identifiers. The specific generation process is as follows: Figure 5 As shown, it includes the following steps: (1) After a user initiates a resolution request to a child node, the request will be sent to the child node; (2) Determine whether the child node identifier is in the parsed data table; (3) If the identifier is not in the parsed data table, the child node will directly send the error message to the user; (4) After the user provides feedback, the child node records the corresponding information in the parsing log table; (5) If the identifier is in the parsed data table, the child node will send the relevant information to the master node; (6) The master node retrieves the identifier resolution address from the identifier information table and sends it to the child node; (7) The child node returns the address to the user; (8) The child node records the relevant information into the parsing log.
[0051] 2.2. Identifier Resolution Inspection Log Generation Method 2.2.1 Labeling Sampling Inspection Method To ensure continuous accessibility of the identifiers, the system will periodically inspect them. Each inspection will record the availability of the identifier's DNS resolution address. The inspection scope includes newly registered identifiers, identifiers with failed DNS resolutions, and a random sample of previously successfully resolved data. The system will also tally the number of DNS resolution failures based on the inspection results.
[0052] 2.2.2 Identifier Resolution Inspection Log Generation Method The process of generating inspection logs by resolving identifiers is as follows: Figure 6 As shown, it includes the following steps: (1) The master node obtains the identifier and resolves the address from the child node; (2) The master node summarizes the identifiers obtained from the child nodes; (3) The master node accesses the identifier's resolution address in sequence; if the identifier's resolution address can be accessed normally, the identifier inspection is recorded as successful. If the identifier's resolution address cannot be accessed, the resolution inspection is recorded as failed.
[0053] (4) The master node records the address access status and forms an identifier resolution inspection log.
[0054] 3. Log Analysis Methods Log analysis is performed by the statistical analysis submodule within the master node's data service function, including the following steps: 3.1 Cleaning and structuring generated logs To ensure that the log data can be used and analyzed normally, it is first necessary to clean and structure the data fields of the logs generated by the master node and child nodes, so as to ensure that the log data of the same fields of the master node and child nodes have the same format.
[0055] 3.2 Data completion for generated logs (1) Completion of details in the log To improve log readability, child nodes will simultaneously supplement the successfully parsed logs with metadata information related to the identifier. The specific process is as follows: 1) The user's child node (the requesting child node) sends a request to the master node; 2) The master node is based on the prefix identifier of the child node address, where the prefix refers to the prefix in the CSTR identifier structure, and the child node refers to the child node where the identifier is registered; 3) The master node sends a request to the child node where the identifier details are located to obtain the identifier metadata details; where the metadata details refer to the metadata information submitted during identifier registration, such as resource name, summary, keywords, etc. 4) After obtaining the identifier metadata details, the master node sends the relevant information to the requesting child nodes. This relevant information includes the identifier metadata details required in the successful parsing log, specifically: Resource Name: {Chinese: Resource Name; English: Resource Name}; Chinese keywords: {keyword1, keyword2}; English keywords: {keyword 1, keyword 2}; Association identifier: {Resource type: Resource identifier}; Name of the organization to which the resource belongs: Organization name.
[0056] (2) Completion of source address parsing in logs The resolution log primarily retrieves the source IP address. To facilitate user understanding, the IP recorded during identifier resolution needs to be converted into a human-readable source domain name and its corresponding country, province, and city. The program uses Python code to obtain the resolved domain name and the corresponding country, province, and city of the IP address based on databases such as MaxMind GeoIP, IP2Location, and IPinfo. The resolved domain name refers to the domain name where the researcher is resolving the identifier. For example, if a user resolves the identifier on the official website of the xx Scientific Data Center, the IP recorded during identifier resolution can be converted into the domain name of the xx Scientific Data Center's official website.
[0057] 3.3 Spatiotemporal Migration Detection 1) Based on the information of the country, province, and city of origin and the resolution time in the previous successful and failed resolution logs of the prefix, a heat map of the prefix's identifier resolution is formed.
[0058] 2) Define the offset index: number of anomaly parsings / total number of parsings.
[0059] 3) Determine abnormal parsing. The following conditions must be met for a parsing error to be identified: a) Resolve the IP address in the IP blacklist database; b) During the identifier resolution inspection, identifiers that fail to resolve 5 consecutive times; c) Historical successful analysis focuses on non-academic IPs (such as shopping malls / internet cafes); d) All identifiers under prefixes whose offset index exceeds the set threshold.
[0060] 4) If the flag is determined to be a parsing anomaly, it will not be calculated or analyzed in subsequent statistics, and the organization and administrator of the flag will be notified.
[0061] 3.4 Abnormal Registration Failure Rate (1) Classify the registration failure data within the statistical period of the node where the registration log is located based on the registration failure type (excluding the parsing anomaly identifiers that have already been determined), specifically including registration failures caused by network problems and technical problems; among which: Network problems refer to registration failures caused by network connectivity factors such as interrupted connections between master and child nodes, and API request response timeouts. Specifically, this includes network problems caused by nodes and network problems caused by users. Technical issues refer to registration failures caused by business logic or data specification factors such as metadata format verification errors, mismatched identifier prefix permissions, invalid parsed address format, or non-existent resource type. (2) If the number of registration failures due to network problems of a single node exceeds the threshold, the node administrator will be notified to investigate the node's network status. (3) If the number of registration failures caused by user network problems within 7 days under a single prefix accounts for more than 60% of the total number of registration failures, the person in charge of the organization shall be notified to investigate the network status of the registration interface.
[0062] (3) The calculation method for abnormal registration failure rate is as follows: in, The number of registration failures is due to a technical issue. Total registrations The ratio; It represents the number of failures for similar resources. It represents the total number of attempts to find similar resources. This method limits the proportion of similar resources by calculating the ratio of the number of failed registrations for similar resources to the total number of registration attempts based on the resource name. This reduces the impact of multiple failed registrations for the same data on the registration failure statistics and also makes it easier for users to troubleshoot the cause of the problem.
[0063] The number of similar resources refers to the number of resources whose names are determined to be similar after processing among all registered resources within a statistical period. The "Number of Failed Registrations for Similar Resources" refers to the total number of times that registrations of resources deemed to be similar failed within the statistical period. "Total number of attempts for similar resources" refers to the total number of registration attempts (including successful and failed registrations) for resources identified as "similar resources" within the statistical period.
[0064] The method for determining similar resources includes the following steps: (1) Extract title: Extract the resource name (in Chinese) from the registration failure log; (2)Text preprocessing: Convert the case to ensure that all English characters are in uppercase; remove punctuation marks and stop words (such as "of", "a", etc.); finally, perform stemming processing; (3)Vectorization: Use the TF-IDF model to convert all preprocessed failed resource names into vectors; (4)Similarity calculation and grouping: Calculate the cosine similarity between the vectors of the resource names (in Chinese) of all registered failed resources pairwise. If the similarity exceeds the threshold, the resources are determined to be similar resources, and the number of registration failures of the similar resources is the number of registration failures of the similar resources.
[0065] If the data of a certain institution prefix 12345 within 7 days is as shown in Table 1, the calculation result is: Table 1 index value Total number of registration attempts 50 times Number of registration failures due to technical issues 28 times Number of similar resource failures 25 times Total number of attempts for similar resources 30 times In the actual operation process, network problems and technical problems often correspond to different responsible persons. Network problems are generally responsible for platform operation and maintenance personnel or technology companies. Technical problems are generally responsible for front-line data managers. By classifying and summarizing problems, the technical management efficiency of users for resources and related technical platforms can be improved, the communication cost can be reduced, and the overall use effect and influence of the platform can be improved by improving the accessibility of registered resources.
[0066] 3.5 Parsing failure rate To improve the accessibility of resources, regularly check the resources, set the parsing failure rate, and improve the management effect of managers on resources (the analysis scope does not include the determined parsing exception flags). The calculation method of the parsing failure rate is as follows: Among them, is the effective failure rate of resources, which is the number of parsing failures in the parsing log and the total number of parsing times The ratio of. is the effective failure number excluding normal network fluctuations, is the total number of parsing attempts.
[0067] represents the failure time dispersion. To improve the calculation accuracy, Shannon entropy is introduced, and the specific calculation method is as follows: Among them, is the time entropy, is the number of statistical days.
[0068] Time entropy The calculation method is as follows: First, based on preliminary research and analysis and historical analysis experience, the day is divided into 8 time periods of 3 hours each, and the number of failure logs in each time period is counted. The overall calculation method is as follows: in, Within the statistical period, in the th Total number of parsing failures occurring within a 3-hour time period; This is the time period index number (an integer ranging from 0 to 7). A collection of failure logs; It is the hour value (an integer between 0 and 23) extracted from the log. For example, if the log time is 2025-08-08 14:30:00, then... ; Divide the hour by 3 and round down to 4, meaning 14:30 belongs to the 5th time period, which is 12-15:00.
[0069] get Then, the time entropy is calculated using the following formula. : in in, The parsing failure event occurred in the first... The probability within a time period (i.e., the proportion of failures in that time period out of the total number of failures). Constraint: When When =0, the term It is defined as 0 in the summation.
[0070] Assume the distribution of prefix parsing failures within 7 days is as shown in Table 2. If the parsing originates from different countries, the statistics will be based on local time. For example, if parsing occurs at 5 AM US time, the time will be counted as 5 AM, which is not Beijing time.
[0071] Table 2 The probability is calculated as follows: Similarly, the probability calculation results for the other time periods are shown in Table 3.
[0072] Table 3 Then time entropy and time dispersion The calculation is as follows: This method reveals the concentration of user parsing failures, allowing administrators to better determine whether resources are under attack from external web crawlers.
[0073] 3.6 Generate user alerts or reports Based on the above indicators (abnormal registration failure rate and parsing failure rate), the system automatically generates user alert reports and sends them to the relevant users. This process, through a closed-loop management mechanism of "monitoring-analysis-early warning-handling-optimization," can significantly improve user resource security and academic influence, with the following specific effects: (1) Improve user resource security Proactive risk exposure: In traditional models, resource security issues rely on user complaints or passive discovery by operations and maintenance personnel, resulting in significant delays. This system proactively analyzes the failure rate and its temporal dispersion in real time. It can proactively identify security threats such as suspected web crawler attacks (e.g., a sharp increase in failure rate between 0:00 and 3:00 AM) and resource hijacking (e.g., abnormal parsing regions), and notify users within the golden 72-hour processing period, thus realizing the transformation from "passive response" to "proactive defense".
[0074] Precise Problem Identification: The report not only informs users of "existing risks," but also pinpoints the root cause of problems through multi-dimensional analysis (such as abnormal IPs, related keywords, and failure time distribution). Users can then quickly configure firewall rules or IP blacklists to implement precise blocking, avoid mistakenly blocking legitimate users, and improve security operation and maintenance efficiency.
[0075] Reduce systemic risks: By analyzing abnormal registration failure rates, abnormal registration behavior can be identified, allowing for precise handling of registration issues for users, purifying the system ecosystem, and enhancing the overall security of the platform.
[0076] (2) Enhance users' academic influence Ensuring resource accessibility: The academic influence of resources is built on the foundation of being "discoverable and accessible." Resolution failure rate directly measures resource accessibility. By issuing early warning reports to urge users to promptly repair broken links (such as URL changes not being updated, server downtime), we ensure that researchers worldwide can reliably access these resources 24 / 7, preventing the loss of academic citations due to "broken links." This is a fundamental guarantee for enhancing influence.
[0077] Data-driven resource optimization: The report provides quantitative impact assessment dimensions (such as resolution volume trends, cross-border resolution distribution, and resolution success rate of high-value resources). Users can clearly understand which resources are popular and which regions have the most users. For example, if an institution finds that its "gene dataset" has a high resolution failure rate in international access, it can optimize international bandwidth or deploy CDN to improve the overseas user experience, thereby promoting international academic cooperation and citations.
[0078] Enhancing Academic Reputation: An institution that can promptly address issues and ensure sustainable access to resources enjoys higher reliability and credibility within the academic community. Researchers are more willing to cite and recommend resources that are consistently stable and well-maintained. The regular health reports provided by this system serve as "proof" of a user's rigorous and reliable resource management, indirectly strengthening their academic reputation and influence.
[0079] Here are three application examples: Example 1: Summarize and analyze users' high-resolution resources to generate regular user reports.
[0080] (1) Enter the user prefix and query period (e.g., from January 1, 2025 to March 1, 2025). (2) Obtain the registration log, parsing inspection log, parsing success log and parsing failure log data by prefix; (3) Calculate the abnormal registration failure rate and parsing failure rate based on the identifier metadata, and sort out information such as the inspection situation; (4) Output user warning reports. An example format for a user warning report is as follows: Figure 7 As shown.
[0081] Example 2: Registration Failure Alert. Every 7 days, abnormal registration data within the node is analyzed. If the calculated abnormal registration failure rate exceeds a threshold, an email is sent to the prefix registration administrator. The email format is as follows: Figure 8 As shown.
[0082] Example 3: High-volume identifier resolution failure alert. On the 1st of each month, analyze the identifier resolution log data. If the identifier resolution failure rate exceeds a threshold, send an email alert. The alert email format is as follows: Figure 9 As shown.
[0083] Another embodiment of the present invention provides a computer device (computer, server, etc.) including a memory and a processor, the memory storing a computer program configured to be executed by the processor, the computer program including instructions for performing steps of the method of the present invention.
[0084] Another embodiment of the present invention provides a computer-readable storage medium (such as ROM / RAM, disk, optical disk) that stores a computer program, which, when executed by a computer, implements the steps of the method of the present invention.
[0085] Another embodiment of the present invention provides a computer program product, the computer program product including a computer program, which, when executed by a computer, implements the steps of the method of the present invention.
[0086] The specific embodiments of the present invention disclosed above are intended to help understand the content of the present invention and to implement it accordingly. Those skilled in the art will understand that various substitutions, changes, and modifications are possible without departing from the spirit and scope of the present invention. The present invention should not be limited to the content disclosed in the embodiments of this specification; the scope of protection of the present invention is defined by the claims.
Claims
1. A user-oriented CSTR identifier registration and parsing log analysis method, characterized in that, Includes the following steps: Generate identifier registration logs, identifier resolution logs, and identifier resolution inspection logs; Analyze the identifier registration log, identifier resolution log, and identifier resolution inspection log to generate user warning reports.
2. The method according to claim 1, characterized in that, The identifier registration log is generated using the following steps: The child node receives the user's identifier registration / modification request; The child nodes check whether there are any problems with the metadata submitted during registration; If there is a problem with the metadata, the user will be notified through the interface and the issue will be recorded in the registration / modification failure log. If there are no issues with the metadata, the registration is completed and feedback is sent to the user via the interface. Record it in the registration / modification success log.
3. The method according to claim 1, characterized in that, The identifier resolution log is generated using the following steps: The child node receives the parsing request initiated by the user; The child node determines whether the identifier is present in the parsed data table; If the identifier is not in the parsed data table, the child node will directly return the error message to the user; After providing feedback to the user, the child node will record the corresponding information in the parsing log table; If the identifier is in the parsed data table, the child node will send the relevant information to the master node; The master node retrieves the identifier resolution address from the identifier information table and sends it to the child nodes; The child node will return the identifier resolution address to the user; The child nodes record the relevant information in the parsing log.
4. The method according to claim 1, characterized in that, The identifier resolution inspection log is generated using the following steps: The master node obtains the identifier and resolved address from the child node; The master node summarizes the identifiers obtained from the child nodes; The master node accesses the resolved addresses of the identifiers sequentially; The master node records address access information and generates an identifier resolution inspection log.
5. The method according to claim 1, characterized in that, The analysis of the identifier registration log, identifier resolution log, and identifier resolution inspection log includes: Clean and structure the data fields of the logs generated by the master node and child nodes to ensure that the log data of the same fields in the master node and child nodes have the same format. Complete the log data, including completing the identification details in the log and completing the parsed source address in the log; Perform spatiotemporal offset detection; Calculate the abnormal registration failure rate and parsing failure rate, and automatically generate user warning reports based on the abnormal registration failure rate and parsing failure rate and send them to the relevant users to improve user resource security and academic influence.
6. The method according to claim 5, characterized in that, The spatiotemporal offset detection includes the following steps: Based on the information of the country, province, and city of origin and the resolution time in the previous successful and failed resolution logs of the prefix, a heat map of the prefix's identifier resolution is formed; Define the offset index as: number of exception parsings / total number of parsings; A parsing anomaly flag is determined if any of the following two conditions are met: a) the parsing IP is in the parsing IP blacklist; b) the flag has failed to resolve 5 times consecutively during parsing inspection; c) historical successful resolutions are concentrated on non-academic IPs; d) all flags under a prefix whose offset index exceeds a set threshold. If a parsing anomaly is identified, it will not be calculated or analyzed in subsequent statistics, and the organization and administrator of the flag will be notified.
7. The method according to claim 5, characterized in that, The abnormal registration failure rate is calculated using the following formula: in, The number of registration failures is due to a technical issue. Total registrations The ratio; It represents the number of failures for similar resources. It represents the total number of attempts to find similar resources.
8. The method according to claim 5, characterized in that, The failure rate of the analysis is calculated using the following formula: in, It is the effective failure rate of resources. This is the number of valid failures excluding normal network fluctuations. This represents the total number of parsing attempts; The failure time dispersion is represented and calculated as follows: in, For time entropy, For statistical purposes: number of days; time entropy The calculation method is as follows: ,in in, For the first time in the statistical period Total number of parsing failures occurring within a 3-hour time period; A collection of failure logs; It is the hour value extracted from the logs; The parsing failure event occurred in the first... The probability within a time period, when When =0, the term It is defined as 0 in the summation.
9. A user-oriented CSTR identifier registration parsing log analysis system employing the method described in any one of claims 1 to 8, characterized in that, It includes a master node and child nodes; the master node includes an identifier resolution module, an identifier management module, and a data service module, which respectively implement identifier resolution function, identifier management function, and data service function; the child nodes include an identifier registration module, an identifier resolution module, and a data service module, which respectively implement identifier registration function, identifier resolution function, and data service function.
10. The system according to claim 9, characterized in that, In the master node, the identifier resolution module includes an identifier resolution submodule and a log database resolution submodule; the identifier management module includes a user authentication submodule, a permission management submodule, a metadata management submodule, and a log management submodule; and the data service module includes a statistical analysis submodule, a user management submodule, a user report submodule, and an identifier query submodule. In the sub-nodes, the identifier registration module includes an identifier registration submodule and a registration log submodule; the identifier resolution module includes an identifier resolution submodule and a log database resolution submodule; and the data service module includes a log management submodule and a data synchronization submodule.