A risk information identification method, device and medium
By acquiring target web pages and automatically crawling structured information using risk identification models and vocabulary databases, combined with content confidence scores, the problems of inaccurate and inefficient risk information identification are solved, achieving fast and accurate risk information identification.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- EAST CHINA BRANCH OF STATE GRID CORP
- Filing Date
- 2026-01-15
- Publication Date
- 2026-06-09
AI Technical Summary
In existing technologies, risk information identification is not accurate enough and is inefficient, making it difficult to comprehensively and accurately identify a company's operational risks, policy risks, environmental risks, and financial risks during investment, mergers and acquisitions, and compliance management in the energy industry.
By acquiring target web pages based on key target information, utilizing pre-trained risk identification models and risk vocabulary databases, automatically crawling and identifying structured information, and combining content confidence levels to determine risk advice, rapid and accurate risk information identification is achieved.
It improves the efficiency and accuracy of risk information identification, ensures the reliability and precision of the final identification results, and can quickly identify and screen credible risk advice.
Smart Images
Figure CN122175691A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of computer technology, and in particular to a method, apparatus and medium for identifying risk information. Background Technology
[0002] In the process of investment, mergers and acquisitions, and compliance management in the energy industry, the operational risks, policy risks, environmental risks, and financial risks of target companies can all have a significant impact on investment decisions. Traditional energy auditing and risk identification methods mainly rely on manually collecting publicly available information from companies, such as government regulatory announcements, environmental penalty records, news reports, industry association notices, and company-disclosed information.
[0003] However, publicly available information on the internet is highly fragmented, with complex and varied website structures, and some websites do not provide standardized APIs. At the same time, a large amount of information is presented in multimodal formats such as web pages, PDFs, images, and scanned documents, which makes manual auditing time-consuming, incomplete and inaccurate in identifying risk information, and prone to missing key risk information, resulting in low efficiency in risk information identification.
[0004] Therefore, there is an urgent need for a risk identification method to solve the problems of inaccurate risk information identification and low identification efficiency in existing technologies. Summary of the Invention
[0005] In view of this, the present invention provides a risk information identification method, device and medium, the main purpose of which is to solve the problems of insufficient accuracy and low efficiency in current risk information identification.
[0006] To address the above problems, this application provides a risk information identification method, comprising: Based on the key information of the target, obtain several target web pages that match the key information of the target; Based on the target key information, information is crawled from each of the target web pages to obtain structured information of several target information corresponding to the target key information; Based on a predetermined risk vocabulary database and a pre-trained risk identification model, risk information is identified for each type of structured information to determine whether each target consultation is a risk consultation or a non-risk consultation, thereby obtaining initial identification results. Based on the initial identification results of each target consultation and the content confidence level of each target consultation, each target consultation is determined to be either risky or non-risky, thus obtaining the target identification result.
[0007] Optionally, obtaining a number of target web pages matching the target key information specifically includes: Based on the target user's input of key information, a predetermined search engine is invoked to perform a search process and obtain several target web pages that match the key information.
[0008] Optionally, the step of crawling information from each of the target web pages based on the target key information to obtain structured information of several target information corresponding to the target key information specifically includes: Based on the pre-trained target search model and the target key information, each target webpage is processed for consultation search to obtain an information list page corresponding to the target key information; the consultation list page displays the network addresses of several target consultations. Based on the target search model, the network addresses of each target piece of information in each of the information list pages are extracted to obtain the network addresses of each target piece of information. Information is crawled from each of the aforementioned network addresses to obtain structured information about each of the aforementioned target inquiries.
[0009] Optionally, the step of performing consultation search processing on each target webpage based on the pre-trained target search model and the target key information to obtain an information list page corresponding to the target key information specifically includes: Based on the pre-trained target search model, the search regions in each target webpage are identified to obtain the target search regions; The target key information is filled into the target search area to obtain the information list page corresponding to the target key information.
[0010] Optionally, based on a predetermined risk vocabulary and a pre-trained risk identification model, the risk information identification is performed on each of the structured information items to determine whether each of the target consultations is risky or non-risky, thus obtaining an initial identification result. This specifically includes: Based on a predetermined risk vocabulary database, risk information is identified for each type of structured information to determine whether each target consultation is a risk consultation or a non-risk consultation, thereby obtaining a first identification result; Based on the pre-trained risk identification model, risk information is identified for each of the structured information items to determine whether each of the target consultations is a risk consultation or a non-risk consultation, thereby obtaining a second identification result; Based on the first identification result and the second identification result of each target consultation, the initial identification result of the target consultation is determined.
[0011] Optionally, the step of determining whether each target consultation is a risky consultation or a non-risky consultation based on the initial identification results of each target consultation and the content confidence level of each target consultation, to obtain the target identification result, specifically includes: When the initial identification result of the target consultation is risk consultation and the content confidence level meets the predetermined confidence level condition, the target consultation is determined to be risk consultation; When the initial identification result of the target consultation is non-risk consultation, and / or the content confidence level does not meet the predetermined confidence level condition, the target consultation is determined to be non-risk consultation.
[0012] Optionally, the method further includes: determining the content confidence level of each of the target consultations, specifically including: Obtain the domain name type score of the website corresponding to each of the target information items; Obtain the friendliness rating of the website corresponding to each of the target inquiries; Obtain the risk consultation percentage score for each of the target consultations on the corresponding websites; Obtain the timeliness score of the content of each of the target consultations; Obtain the source consistency score for each of the target consultations; Obtain the structured evidence quality score for each of the aforementioned target consultations; The confidence level of the target consultation content is determined based on any one or more of the following: domain type score, friendliness score, risk consultation ratio score, content timeliness score, source consistency score, and structured evidence quality score.
[0013] Optionally, the method further includes: Determine the webpage confidence level for each of the target webpages; Based on the webpage confidence level of each target webpage, the target crawling cycle for each target webpage is determined; Information is crawled from each target webpage based on each target crawler cycle.
[0014] To address the above problems, this application provides a risk information identification device, comprising: The acquisition module is used to acquire target web pages that match the target key information based on the target key information; The crawling module is used to crawl information from each of the target web pages based on the target key information, and obtain structured information of several target information corresponding to the target key information; The initial identification module is used to identify risk information for each of the structured information based on a predetermined risk vocabulary library and a pre-trained risk identification model, determine whether each of the target consultations is a risk consultation or a non-risk consultation, and obtain the initial identification result. The target identification module is used to determine whether each target consultation is a risk consultation or a non-risk consultation based on the initial identification results of each target consultation and the content confidence of each target consultation, and to obtain the target identification result.
[0015] To address the aforementioned problems, this application provides a storage medium storing a computer program that, when executed by a processor, implements the steps of any of the risk information identification methods described above.
[0016] To address the aforementioned problems, this application provides a computer device, comprising at least a memory and a processor, wherein the memory stores a computer program, and the processor, when executing the computer program in the memory, implements the steps of any of the aforementioned risk information identification methods.
[0017] The risk information identification method, apparatus, and medium in this application obtain corresponding target web pages by utilizing key target information. Subsequently, information can be automatically crawled from these web pages, improving the efficiency of acquiring target information. Based on a risk vocabulary database and risk identification model, it can quickly identify whether each target consultation is risky or non-risky, thus rapidly obtaining initial identification results for each target consultation and ensuring rapid risk information identification. In this application, after obtaining the initial identification results for each target consultation, the credibility / truthfulness of each target consultation can be further determined by combining the content confidence level of each consultation, making the final risk identification / target identification results more accurate and reliable.
[0018] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description
[0019] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings: Figure 1 This is a flowchart illustrating a risk information identification method according to an embodiment of this application; Figure 2 This is a structural block diagram of a risk information identification device according to another embodiment of this application; Figure 3 This is a structural block diagram of a computer device according to an embodiment of this application. Detailed Implementation
[0020] Various embodiments and features of this application are described herein with reference to the accompanying drawings.
[0021] It should be understood that various modifications can be made to the embodiments described herein. Therefore, the above description should not be considered as limiting, but merely as an example of embodiments. Other modifications within the scope and spirit of this application will be apparent to those skilled in the art.
[0022] The accompanying drawings, which are included in and form part of this specification, illustrate embodiments of the present application and, together with the general description of the present application given above and the detailed description of the embodiments given below, serve to explain the principles of the present application.
[0023] These and other features of this application will become apparent from the following description of preferred forms of embodiments given as non-limiting examples, with reference to the accompanying drawings.
[0024] It should also be understood that although this application has been described with reference to some specific examples, those skilled in the art can certainly implement many other equivalent forms of this application.
[0025] The above and other aspects, features and advantages of this application will become more apparent when taken in conjunction with the accompanying drawings and in view of the following detailed description.
[0026] Specific embodiments of this application are described thereafter with reference to the accompanying drawings; however, it should be understood that the claimed embodiments are merely examples of this application, which can be implemented in various ways. Well-known and / or repeated functions and structures are not described in detail to avoid unnecessary or redundant details that could obscure the application. Therefore, the specific structural and functional details claimed herein are not intended to be limiting, but merely serve as the basis and representative basis for the claims to teach those skilled in the art to use this application in a variety of substantially any suitable detailed structures.
[0027] This specification may use the phrases “in one embodiment,” “in another embodiment,” “in yet another embodiment,” or “in other embodiments,” all of which may refer to one or more of the same or different embodiments according to this application.
[0028] This application provides a risk information identification method that can be applied to computer devices such as terminals and servers. Taking its application to a server as an example, the server can be a standalone server or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. Figure 1 As shown, the risk information identification method in this embodiment includes the following steps: Step S101: Based on the target key information, obtain several target web pages that match the target key information; In the specific implementation of this step, the key target information can be the name of the target company, the name of the target organization, or the name of the target institution, etc.
[0029] Specifically, based on the target key information input by the target user, a predetermined search engine can be invoked to perform search processing and obtain several target web pages that match the target key information.
[0030] Step S102: Based on the target key information, crawl information from each of the target web pages to obtain structured information of several target information corresponding to the target key information; In this step, based on a pre-trained target search model and the key target information, information can be crawled from each target webpage to obtain structured information about the target information. Utilizing the target search model improves crawling efficiency and lays the foundation for rapid risk information identification in the future.
[0031] Step S103: Based on the predetermined risk vocabulary database and the pre-trained risk identification model, risk information identification is performed on each of the structured information to determine whether each of the target consultations is risk consultation or non-risk consultation, and an initial identification result is obtained. In the specific implementation process of this step, risk identification can be performed on each of the structured information based on a risk vocabulary database to obtain a first identification result. Then, risk identification can be performed on each of the structured information based on a risk identification model distribution to obtain a second identification result. Finally, the first and second identification results are combined to obtain an initial identification result. That is, for the same target consultation, the initial identification result of the target consultation can only be non-risk consultation if both the first and second identification results are non-risk consultation; conversely, if the first and / or second identification results are risk consultation, the initial identification result of the target consultation can be determined to be risk consultation.
[0032] Step S104: Based on the initial identification results of each target consultation and the content confidence level of each target consultation, determine whether each target consultation is a risk consultation or a non-risk consultation, and obtain the target identification result.
[0033] In this step, if the initial identification result of the target consultation is risky consultation, and the confidence level of the content of the target consultation meets the predetermined confidence level condition, it indicates that the target consultation is true and credible, and thus it can be determined that the target consultation is risky consultation. Conversely, if the initial identification result of the target consultation is non-risky consultation, and / or the confidence level of the content does not meet the predetermined confidence level condition, the target consultation is determined to be non-risky consultation. That is, if the initial identification result of the target consultation is non-risky consultation, regardless of whether the confidence level of the content meets the predetermined condition, the target consultation can be determined to be non-risky consultation; similarly, if the confidence level of the content does not meet the predetermined confidence level condition, it indicates that the target consultation is unreliable, and thus regardless of whether the initial identification result of the target consultation is non-risky consultation or risky consultation, the target consultation can be determined to be non-risky consultation.
[0034] The risk information identification method in this embodiment obtains the corresponding target webpage using key target information. Subsequently, it can automatically crawl information from these webpages, improving the efficiency of acquiring target information. Based on a risk vocabulary database and risk identification model, it can quickly identify whether each target consultation is risky or non-risky, thus rapidly obtaining initial identification results for each consultation and ensuring rapid risk information identification. In this application, after obtaining the initial identification results for each consultation, the credibility / truthfulness of each consultation can be further determined by combining the content confidence level of each consultation, making the final risk identification / target identification results more accurate and reliable.
[0035] Based on the above embodiments, another embodiment of this application provides a risk information identification method, which specifically includes the following steps: Step S201: Based on the target key information input by the target user, call a predetermined search engine to perform search processing and obtain several target web pages that match the target key information; In the specific implementation of this step, the target user can enter the target company name and then call the pre-defined search engine API to obtain all web pages related to the target company name, that is, obtain a number of target web pages.
[0036] Step S202: Based on the pre-trained target search model and the target key information, perform consultation search processing on each target webpage to obtain an information list page corresponding to the target key information; the consultation list page displays the network addresses of several target consultations; In the specific implementation process, this step can be based on the pre-trained target search model to identify the search areas in each target webpage to obtain the target search area; the target key information is then filled into the target search area to obtain the information list page corresponding to the target key information.
[0037] Specifically, a multimodal large-scale model / target search model can be used to crawl target information. That is, each parsed target webpage can be accessed sequentially. Then, the OCR + UI element recognition capabilities of the large-scale model are used to locate the "site search box" on the target webpage. Next, the target company name is automatically entered as a search keyword in the "site search box," thereby obtaining the information list page corresponding to the target company name and entering the information list page within the website.
[0038] Step S203: Based on the target search model, extract the network addresses of each target information in each of the information list pages to obtain the network addresses of each target information. In this step, a large-scale model / targeted search model can be used to parse the information list, export the page content to Markdown format, and extract the URL of each piece of information in the list, thus obtaining the web address of each target information within the information list page. A multimodal model automatically executes the "next page" pagination action, continuously crawling until all search results are parsed, obtaining the web address of each target information.
[0039] Step S204: Crawling information from each of the network addresses to obtain structured information for each of the target inquiries.
[0040] In the specific implementation of this step, an automated web crawler can be launched for the URLs of each target information to obtain structured information. The structured information can specifically include any one or more of the following: information title, information body (text and images are automatically converted to text), attachment content (PDFs and images can be parsed via OCR), publication time, source website type (government, organization, enterprise, media), original HTML, and extracted structured JSON.
[0041] Step S205: Based on a predetermined risk vocabulary database, risk information is identified for each of the structured information items to determine whether each of the target consultations is a risk consultation or a non-risk consultation, and a first identification result is obtained. In this step, the risk vocabulary database includes a vocabulary set corresponding to each risk type. Specifically, the risk types include: administrative penalty risk types, environmental risk types, safety risk types, financial risk types, regulatory compliance types, and public opinion risk types. The vocabulary sets include the following: Administrative penalty vocabulary set, which includes one or more of the following keywords: order to rectify, administrative penalty decision, and illegal acts, etc. Environmental risk vocabulary set, which includes one or more of the following keywords: excessive emissions, environmental pollution, and production restriction rectification, etc. Safety risk vocabulary set, which includes one or more of the following keywords: explosion, accident report, and work safety liability accident, etc. Risk vocabulary set, which includes one or more of the following keywords: widening losses, tight cash flow, and debt default, etc. Regulatory compliance vocabulary set, which includes one or more of the following keywords: non-compliance with regulatory requirements and illegal operations, etc. Public opinion risk vocabulary set, which includes one or more of the following keywords: complaints, media exposure, and negative reports, etc. In the specific implementation process, this step can use high-frequency trigger word matching, risk rule template identification based on regular expressions (such as "punished because of..."), syntactic dependency analysis to extract subject / verb / punishment result, etc., to initially determine whether each target consultation is risk consultation or non-risk consultation, that is, to obtain the first identification result of each target consultation.
[0042] Step S206: Based on the pre-trained risk identification model, risk information identification is performed on each of the structured information to determine whether each of the target consultations is risk consultation or non-risk consultation, and a second identification result is obtained. In the specific implementation of this step, the risk identification model can be based on the Transformer / LLM structure, with multimodal sequence vectors as input and risk labels and evidence paragraphs as outputs.
[0043] Specifically, the risk identification model can be pre-trained. The training process is as follows: First, multi-task objective training is performed, and then the risk identification model is enhanced based on the energy industry knowledge graph. For example, each first sample consultation can be obtained, and then the paragraphs / statements involving risk information in each first sample consultation can be identified, along with the risk type and risk level, thus obtaining the first risk label for each first sample consultation. Then, the initial risk identification model is trained using each first sample consultation and its first risk label to obtain the first risk identification model. Next, second sample consultations containing specific risk information are obtained, where the predetermined risk information could be, for example, "pollution discharge permit," "safety production law," "boiler explosion," "environmental inspection," etc. Then, the paragraphs / statements involving specific risk information in each second sample consultation can be identified, along with the risk type and risk level, thus obtaining the second risk label for each second sample consultation. Finally, the initial risk identification model is enhanced using each second sample consultation and its second risk label to obtain the final risk identification model.
[0044] Step S207: Based on the first identification result and the second identification result of each target consultation, determine the initial identification result of the target consultation; In the specific implementation of this step, for the same target consultation, the initial identification result of the target consultation can only be non-risk consultation if both the first identification result and the second identification result are non-risk consultation; otherwise, if the first identification result and / or the second identification result are risk consultation, the initial identification result of the target consultation can be determined to be risk consultation.
[0045] Step S208: Determine the content confidence level of each of the target consultations; In the specific implementation process of this step, the following can be obtained: the domain name type score of the website corresponding to each of the target information; the friendliness score of the website corresponding to each of the target consultations; the risk consultation ratio score of the website corresponding to each of the target consultations; the content timeliness score of each of the target consultations; the source consistency score of each of the target consultations; and the structured evidence quality score of each of the target consultations. Based on any one or more of the domain name type score, friendliness score, risk consultation ratio score, content timeliness score, source consistency score, and structured evidence quality score corresponding to the same target consultation, the content confidence level of the target consultation can be determined.
[0046] Specifically, for domain name type scoring, different scores D' can be pre-configured for different website domain names based on domain name type D, thereby establishing a correspondence between each domain name type and its score. This correspondence can be shown in Table 1 below.
[0047] Table 1:
[0048] In this embodiment, a friendliness score can be determined based on whether a website allows access / is subject to access permissions. Specifically, a correspondence between each access permission status and its corresponding score can be pre-established. This correspondence can be shown in Table 2 below.
[0049] Table 2:
[0050] In this embodiment, the risk consultation ratio score can be determined by identifying the total number of consultations and the number of risky consultations on the target website. The risk consultation ratio is then determined based on these figures. Finally, the risk consultation ratio score is determined according to the correspondence between the ratio and the score, as shown in Table 3. The formula for calculating the risk consultation ratio P is as follows:
[0051] Table 3
[0052] In this embodiment, the timeliness T of the content can be determined by the time difference Δt between the publication time of the target consultation and the current time. Then, according to the predetermined correspondence between content timeliness and score as shown in Figure 4, the content timeliness score T' is determined. The formula for calculating the content timeliness T is as follows:
[0053] Where e is a predetermined constant. It can be measured in months.
[0054] Table 4:
[0055] In this embodiment, the source consistency C can be determined based on the number of websites containing the target consultation, and then the source consistency score C' can be determined based on the source consistency C. The formula for calculating source consistency C is as follows:
[0056] The formula for determining the source consistency score C' is as follows:
[0057] In other words, when C is greater than or equal to 1, the source consistency score C' is 1 point (maximum 1 point); when the source consistency C is less than 1, the source consistency score C' can be C. That is, reports from at least 3 websites are considered the most credible, and the source consistency score C' can be 1 point.
[0058] In this embodiment, the structured evidence quality score E' can be determined based on the correspondence between structured data features and scores as shown in Table 5.
[0059] In this embodiment, after determining the domain type score D', friendliness score R', risk consultation ratio score P', content timeliness score T', source consistency score C', and structured evidence quality score E', the content confidence level of the target consultation can be determined using the following formula. :
[0060] in, , , , , as well as These are the predetermined weighting coefficients. See Table 6 below for details.
[0061] Table 6:
[0062] Step S209: When the initial identification result of the target consultation is risky consultation and the content confidence level meets the predetermined confidence level condition, the target consultation is determined to be risky consultation; when the initial identification result of the target consultation is non-risky consultation and / or the content confidence level does not meet the predetermined confidence level condition, the target consultation is determined to be non-risky consultation, so as to obtain the target identification result.
[0063] In the specific implementation of this step, for example, if the target consultation is determined to be risky, and the confidence level of the target consultation's content is determined to be greater than or equal to 0.85, then the risky consultation can be determined to be highly credible and can be given priority as audit evidence. If the target consultation is determined to be risky, and the confidence level of the target consultation's content is determined to be greater than 0.6 and less than 0.85, then the risky consultation can be determined to be generally credible and can be used as auxiliary judgment information for auditing. If the target consultation is determined to be risky, and the confidence level of the target consultation's content is determined to be less than 0.60, then the target information is low-credibility and requires further manual confirmation or elimination.
[0064] The risk information identification method in this embodiment obtains the corresponding target webpage using key target information. Subsequently, it can automatically crawl information from these webpages, improving the efficiency of acquiring target information. Based on a risk vocabulary database and risk identification model, it can quickly identify whether each target consultation is risky or non-risky, thus rapidly obtaining initial identification results for each consultation and ensuring rapid risk information identification. In this application, after obtaining the initial identification results for each consultation, the credibility / truthfulness of each consultation can be further determined by combining the content confidence level of each consultation, making the final risk identification / target identification results more accurate and reliable.
[0065] In this embodiment, during specific implementation, the webpage confidence level of each target webpage can be determined separately; based on the webpage confidence level of each target webpage, the target crawling cycle of each target webpage can be determined; and information can be crawled within each target webpage based on the target crawling cycle. In this embodiment, when determining the confidence level of a target webpage, the following can be specifically obtained: the target webpage's website domain type score D''; the target webpage's friendliness score R''; the target webpage's consultation quantity score Q''; the target webpage's content update timeliness score T''; the target webpage's risk consultation ratio score P''; and the target webpage's historical stability score S''. The target webpage's website domain type score, target webpage's friendliness score, and target webpage's risk consultation ratio score are the same as those used in determining content confidence in the above embodiment, and will not be repeated here.
[0066] In this embodiment, the scoring of the number of inquiries for a target webpage can be pre-established by creating a correspondence between different inquiries and scores within a predetermined time period. This allows the score to be determined based on the number of inquiries for the target webpage by searching this correspondence. For example, calculating the number of news items q in the last 90 days: when q > 200, the score is determined to be 1.0; when the number of news items q is within the range of 50–200, the score is determined to be 0.7; and when the number of news items q < 50, the score is determined to be 0.4.
[0067] In this embodiment, the historical stability score of a target webpage can be determined by combining factors such as the recent failed access rate, the frequency of page structure changes, and webpage status such as login requirements or complex anti-crawling measures. Specifically, as shown in Table 7 below, the historical stability score can be determined.
[0068] Table 7:
[0069] In this embodiment, the webpage confidence score of the target webpage can be calculated using the following webpage confidence score calculation formula. .
[0070]
[0071] in, , , , , as well as These are the predetermined weighting coefficients, as shown in Table 8 below.
[0072] Table 8:
[0073] In this embodiment, the crawling period corresponding to different webpage confidence scores is shown in Table 9 below.
[0074] Table 9:
[0075] In this embodiment, by determining the webpage execution degree, the crawling cycle can be reasonably and accurately determined based on the webpage confidence level. This allows for the automatic and rapid periodic acquisition of several target inquiries based on the crawling cycle, ensuring accurate and timely determination of whether each target inquiry is a risky inquiry and laying the foundation for rapid and accurate risk information identification.
[0076] Another embodiment of this application provides a risk information identification device, such as... Figure 2 As shown, it includes: The acquisition module 11 is used to acquire target web pages that match the target key information based on the target key information; The crawling module 12 is used to crawl information from each of the target web pages based on the target key information, and obtain structured information of several target information corresponding to the target key information; The initial identification module 13 is used to identify risk information for each of the structured information based on a predetermined risk vocabulary library and a pre-trained risk identification model, determine whether each of the target consultations is a risk consultation or a non-risk consultation, and obtain the initial identification result. The target identification module 14 is used to determine whether each target consultation is a risk consultation or a non-risk consultation based on the initial identification results of each target consultation and the content confidence of each target consultation, and to obtain the target identification result.
[0077] In this embodiment, the acquisition module is specifically used to: based on the target key information input by the target user, call a predetermined search engine to perform search processing and obtain a number of target web pages that match the target key information.
[0078] In this embodiment, the crawling module is specifically used for: performing consultation search processing on each target webpage based on a pre-trained target search model and the target key information, to obtain an information list page corresponding to the target key information; the consultation list page displays the network addresses of several target consultations; based on the target search model, extracting the network addresses of each target information in each information list page, to obtain the network addresses of each target information; and crawling information from each network address to obtain the structured information of each target consultation.
[0079] In this embodiment, the crawling module is specifically used to: identify the search areas in each target webpage based on a pre-trained target search model to obtain the target search area; fill the target key information into the target search area to obtain the information list page corresponding to the target key information.
[0080] In this embodiment, the initial identification module is specifically used for: identifying risk information in each of the structured information based on a predetermined risk vocabulary library, determining whether each target consultation is a risk consultation or a non-risk consultation, and obtaining a first identification result; identifying risk information in each of the structured information based on a pre-trained risk identification model, determining whether each target consultation is a risk consultation or a non-risk consultation, and obtaining a second identification result; and determining the initial identification result of the target consultation based on the first identification result and the second identification result of each target consultation.
[0081] In this embodiment, the target identification module is specifically used to: determine the target consultation as risk consultation when the initial identification result of the target consultation is risk consultation and the content confidence level meets the predetermined confidence level condition; and determine the target consultation as non-risk consultation when the initial identification result of the target consultation is non-risk consultation and / or the content confidence level does not meet the predetermined confidence level condition.
[0082] In this embodiment, the risk information identification device further includes a determination module, which is used to: determine the content confidence level of each target consultation, specifically by: obtaining the domain name type score of the website corresponding to each target consultation; obtaining the friendliness score of the website corresponding to each target consultation; obtaining the risk consultation ratio score of the website corresponding to each target consultation; obtaining the content timeliness score of each target consultation; obtaining the source consistency score of each target consultation; obtaining the structured evidence quality score of each target consultation; and determining the content confidence level of the target consultation based on any one or more of the domain name type score, friendliness score, risk consultation ratio score, content timeliness score, source consistency score, and structured evidence quality score corresponding to the same target consultation.
[0083] In this embodiment, the risk information identification device further includes a crawler cycle determination module, which is specifically used to: determine the webpage confidence level of each target webpage; determine the target crawler cycle of each target webpage based on the webpage confidence level of each target webpage; the crawling module is also used to: crawl information within each target webpage based on each target crawler cycle.
[0084] The risk information identification device in this embodiment obtains the corresponding target webpage using key target information. Subsequently, it can automatically crawl information from the target webpage, improving the efficiency of acquiring target information. Based on a risk vocabulary database and risk identification model, it can quickly identify whether each target consultation is risky or non-risky, thus rapidly obtaining the initial identification results for each target consultation and ensuring rapid risk information identification. In this application, after obtaining the initial identification results for each target consultation, by further combining the content confidence level of each target consultation, the credibility / authenticity of each target consultation can be further determined, making the final risk identification / target identification results more accurate and reliable.
[0085] Another embodiment of this application provides a storage medium storing a computer program, which, when executed by a processor, implements the following method steps: Step 1: Based on the key target information, obtain several target web pages that match the key target information; Step 2: Based on the target key information, crawl information from each target webpage to obtain structured information of several target information corresponding to the target key information; Step 3: Based on the predetermined risk vocabulary database and the pre-trained risk identification model, risk information identification is performed on each of the structured information to determine whether each of the target consultations is risky or non-risky, and the initial identification results are obtained. Step 4: Based on the initial identification results of each target consultation and the content confidence level of each target consultation, determine whether each target consultation is a risk consultation or a non-risk consultation, and obtain the target identification result.
[0086] The specific implementation process of the above method steps can be found in the embodiments of any of the above risk information identification methods, and will not be repeated here.
[0087] The storage medium of this application, by utilizing key target information to obtain corresponding target web pages, can then automatically crawl information from these web pages, improving the efficiency of acquiring target information. Subsequently, based on a risk vocabulary database and risk identification model, it can quickly identify whether each target consultation is risky or non-risky, thereby rapidly obtaining initial identification results for each target consultation and ensuring rapid risk information identification. In this application, after obtaining the initial identification results for each target consultation, by further combining the content confidence level of each target consultation, the credibility / authenticity of each target consultation can be further determined, making the final risk identification / target identification results more accurate and reliable.
[0088] Another embodiment of this application provides a computer device, such as... Figure 3 As shown, it includes at least a memory 1 and a processor 2. The memory 1 stores a computer program, and the processor 2 performs the following method steps when executing the computer program in the memory: Step 1: Based on the key target information, obtain several target web pages that match the key target information; Step 2: Based on the target key information, crawl information from each target webpage to obtain structured information of several target information corresponding to the target key information; Step 3: Based on the predetermined risk vocabulary database and the pre-trained risk identification model, risk information identification is performed on each of the structured information to determine whether each of the target consultations is risky or non-risky, and the initial identification results are obtained. Step 4: Based on the initial identification results of each target consultation and the content confidence level of each target consultation, determine whether each target consultation is a risk consultation or a non-risk consultation, and obtain the target identification result.
[0089] The specific implementation process of the above method steps can be found in the embodiments of any of the above risk information identification methods, and will not be repeated here.
[0090] The computer equipment described in this application obtains corresponding target web pages using key target information. Subsequently, it can automatically crawl information from these web pages, improving the efficiency of acquiring target information. Based on a risk vocabulary database and risk identification model, it can quickly identify whether each target consultation is risky or non-risky, thus rapidly obtaining initial identification results for each consultation and ensuring rapid risk information identification. In this application, after obtaining the initial identification results for each consultation, the credibility / truthfulness of each consultation can be further determined by combining the content confidence level of each consultation, making the final risk identification / target identification results more accurate and reliable.
[0091] The above embodiments are merely exemplary embodiments of this application and are not intended to limit this application. The scope of protection of this application is defined by the claims. Those skilled in the art can make various modifications or equivalent substitutions to this application within its substance and scope of protection, and such modifications or equivalent substitutions should also be considered to fall within the scope of protection of this application.
Claims
1. A risk information identification method, characterized in that, include: Based on the key information of the target, obtain several target web pages that match the key information of the target; Based on the target key information, information is crawled from each of the target web pages to obtain structured information of several target information corresponding to the target key information; Based on a predetermined risk vocabulary database and a pre-trained risk identification model, risk information is identified for each type of structured information to determine whether each target consultation is a risk consultation or a non-risk consultation, thereby obtaining initial identification results. Based on the initial identification results of each target consultation and the content confidence level of each target consultation, each target consultation is determined to be either risky or non-risky, thus obtaining the target identification result.
2. The method as described in claim 1, characterized in that, The step of obtaining several target web pages that match the target key information specifically includes: Based on the target user's input of key information, a predetermined search engine is invoked to perform a search process and obtain several target web pages that match the key information.
3. The method as described in claim 1, characterized in that, The process of crawling information from each target webpage based on the target key information to obtain structured information of several target news items corresponding to the target key information specifically includes: Based on the pre-trained target search model and the target key information, each target webpage is processed for consultation search to obtain an information list page corresponding to the target key information; the consultation list page displays the network addresses of several target consultations. Based on the target search model, the network addresses of each target piece of information in each of the information list pages are extracted to obtain the network addresses of each target piece of information. Information is crawled from each of the aforementioned network addresses to obtain structured information about each of the aforementioned target inquiries.
4. The method as described in claim 3, characterized in that, The method, based on the pre-trained target search model and the target key information, performs consultation search processing on each of the target web pages to obtain an information list page corresponding to the target key information, specifically including: Based on the pre-trained target search model, the search regions in each target webpage are identified to obtain the target search regions; The target key information is filled into the target search area to obtain the information list page corresponding to the target key information.
5. The method as described in claim 1, characterized in that, The risk identification model, based on a predetermined risk vocabulary and a pre-trained risk identification database, identifies risk information in each type of structured information to determine whether each target consultation is risky or non-risky, thus obtaining initial identification results. Specifically, this includes: Based on a predetermined risk vocabulary database, risk information is identified for each type of structured information to determine whether each target consultation is a risk consultation or a non-risk consultation, thereby obtaining a first identification result; Based on the pre-trained risk identification model, risk information is identified for each of the structured information items to determine whether each of the target consultations is a risk consultation or a non-risk consultation, thereby obtaining a second identification result; Based on the first identification result and the second identification result of each target consultation, the initial identification result of the target consultation is determined.
6. The method as described in claim 1, characterized in that, The process of determining whether each target consultation is a risky or non-risky consultation based on the initial identification results and the content confidence level of each target consultation, thereby obtaining target identification results, specifically includes: When the initial identification result of the target consultation is risk consultation and the content confidence level meets the predetermined confidence level condition, the target consultation is determined to be risk consultation; When the initial identification result of the target consultation is non-risk consultation, and / or the content confidence level does not meet the predetermined confidence level condition, the target consultation is determined to be non-risk consultation.
7. The method according to any one of claims 1-6, characterized in that, The method further includes: determining the content confidence level of each of the target consultations, specifically including: Obtain the domain name type score of the website corresponding to each of the target information items; Obtain the friendliness rating of the website corresponding to each of the target inquiries; Obtain the risk consultation percentage score for each of the target consultations on the corresponding websites; Obtain the timeliness score of the content of each of the target consultations; Obtain the source consistency score for each of the target consultations; Obtain the structured evidence quality score for each of the aforementioned target consultations; The confidence level of the target consultation content is determined based on any one or more of the following: domain type score, friendliness score, risk consultation ratio score, content timeliness score, source consistency score, and structured evidence quality score.
8. The method as described in claim 1, characterized in that, The method further includes: Determine the webpage confidence level for each of the target webpages; Based on the webpage confidence level of each target webpage, the target crawling cycle for each target webpage is determined; Information is crawled from each target webpage based on each target crawler cycle.
9. A risk information identification device, characterized in that, include: The acquisition module is used to acquire target web pages that match the target key information based on the target key information; The crawling module is used to crawl information from each of the target web pages based on the target key information, and obtain structured information of several target information corresponding to the target key information; The initial identification module is used to identify risk information for each of the structured information based on a predetermined risk vocabulary library and a pre-trained risk identification model, determine whether each of the target consultations is a risk consultation or a non-risk consultation, and obtain the initial identification result. The target identification module is used to determine whether each target consultation is a risk consultation or a non-risk consultation based on the initial identification results of each target consultation and the content confidence of each target consultation, and to obtain the target identification result.
10. A storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the steps of the risk information identification method according to any one of claims 1-8.