Method and apparatus for evaluating quality of information service

Abstract

The application provides an information service quality evaluation method and an information service quality evaluation device, and relates to the technical field of information service quality evaluation.The method comprises the following steps: obtaining a measurement value of a target evaluation event; calculating a product of a weight of a target evaluation index and a ratio between an important index of the target evaluation index at a target moment and an actual score corresponding to the target evaluation index at the target moment, to determine a first quality evaluation score of the target evaluation index at the target moment; determining a target quality evaluation score of information service quality based on a maximum amplitude of the first quality evaluation score of the target evaluation index at each moment, an average value and a variance of the first quality evaluation score of the target evaluation index at each moment, and an expectation of the quality evaluation score of the target evaluation index at each moment; and determining an evaluation result of the information service quality based on the target quality evaluation score and a target threshold.The application improves the accuracy of the evaluation result.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of information service quality evaluation, and particularly relates to an information service quality evaluation method and an information service quality evaluation device. BACKGROUND

[0002] In order to help users to select better service providers, the services provided by various service providers are usually evaluated in quality. The traditional information technology service quality evaluation method is based on a time point to collect evaluation materials for evaluation, and the evaluation result obtained by the evaluation method is not accurate. SUMMARY

[0003] The present application provides an information service quality evaluation method and an information service quality evaluation device to solve the defect of low accuracy of information service quality evaluation in the prior art and improve the accuracy of information service quality evaluation.

[0004] The present application provides an information service quality evaluation method, comprising:

[0005] Obtaining a measurement value of a target evaluation event and a target moment corresponding to the target evaluation event, the target evaluation event being an actually occurred evaluation event;

[0006] Calculating the product between the weight of a target evaluation index corresponding to the target evaluation event in a plurality of evaluation indexes and the ratio between the important index of the target evaluation index to the information service at the target moment and the actual score corresponding to the target evaluation index at the target moment, to determine the first quality evaluation score of the target evaluation index at the target moment;

[0007] Determining the target quality evaluation score of the information service quality based on the maximum amplitude of the first quality evaluation score of the target evaluation index at each moment, the average value of the first quality evaluation score of the target evaluation index at each moment, the expectation of the first quality evaluation score of the target evaluation index at each moment and the variance of the first quality evaluation score of the target evaluation index at each moment;

[0008] Determining the evaluation result of the information service quality based on the target quality evaluation score and a target threshold value;

[0009] The actual score corresponding to the target evaluation index at the target moment and the important index of the target evaluation index to the information service at the target moment are both determined based on a target technical standard knowledge graph and the measurement value of the target evaluation event.

[0010] According to the information service quality evaluation method provided by the application, the weight of the target evaluation index in the plurality of evaluation indexes corresponding to the target evaluation event is calculated, and the product between the ratio of the important index of the information service at the target time to the target evaluation index at the target time and the actual score corresponding to the target evaluation index at the target time is determined to determine the first quality evaluation score of the target evaluation index at the target time, comprising:

[0011] By the formula:

[0012]

[0013] Determination, wherein, is the evaluation index I n The first quality evaluation score at time t; is the evaluation index I n The important index of the information service at time t; is the weight of the evaluation index I n ; and is the actual score corresponding to the evaluation index I n at time t; n is used to represent the target evaluation index; n represents the number of evaluation events corresponding to the evaluation index I n , and n is a positive integer.

[0014] According to the information service quality evaluation method provided by the application, the important index of the information service at the target time of the target evaluation index is determined by the following steps:

[0015] Based on the target technical standard knowledge graph, the standard score corresponding to the target evaluation index at the target time is determined;

[0016] Based on the ratio between the difference between the actual score corresponding to the target evaluation index at the target time and the standard score corresponding to the target evaluation index at the target time and the standard score corresponding to the target evaluation index at the target time, the important index of the information service at the target time of the target evaluation index is determined.

[0017] According to the information service quality evaluation method provided by the application, the important index of the information service at the target time of the target evaluation index is determined based on the ratio between the difference between the actual score corresponding to the target evaluation index at the target time and the standard score corresponding to the target evaluation index at the target time and the standard score corresponding to the target evaluation index at the target time, comprising:

[0018] By the formula:

[0019]

[0020] determining, wherein, is an evaluation index I n an important index of an information service at time t; is the evaluation index I n a corresponding actual score; is the evaluation index I n a corresponding standard score.

[0021] According to the information service quality evaluation method provided by the application, the actual score corresponding to the target evaluation index at the target time is determined by the following steps:

[0022] Based on the target technical standard knowledge graph, the standard reference value of the evaluation target evaluation event under the target evaluation index is determined;

[0023] Based on the standard reference value of the evaluation target evaluation event under the target evaluation index, the average of the measurement values of the corresponding evaluation event of the target evaluation index at each time, and the measurement value of the target evaluation event at the target time under the target evaluation index, the actual score corresponding to the target evaluation index at the target time is determined.

[0024] According to the information service quality evaluation method provided by the application, the actual score corresponding to the target evaluation index at the target time is determined based on the standard reference value of the evaluation target evaluation event under the target evaluation index, the average of the measurement values of the corresponding evaluation event of the target evaluation index at each time, and the measurement value of the target evaluation event at the target time under the target evaluation index, comprising:

[0025] by the formula:

[0026]

[0027] determining, wherein, is an evaluation index I n a corresponding actual score; ψ i characterizes the standard reference value of the evaluation index I n evaluation event i; characterizes the average of the measurement values of the corresponding evaluation event at each time under the evaluation index I n ; characterizes the measurement value of the evaluation event i at time t under the evaluation index I n ; n n is a positive integer; i is the i-th evaluation event.

[0028] According to the information service quality evaluation method provided by the application, the target quality evaluation score of the information service quality is determined based on the maximum amplitude of the first quality evaluation score of the target evaluation index at each time, the average value of the first quality evaluation score of the target evaluation index at each time, the expectation of the first quality evaluation score of the target evaluation index at each time, and the variance of the first quality evaluation score of the target evaluation index at each time, and the method comprises the following steps of:

[0029] The target quality evaluation score F is determined by the following formula:

[0030]

[0031] Wherein, F is the target quality evaluation score; x is the evaluation index I n The average value of the first quality evaluation score at each time; b is the evaluation index I n The expectation of the quality evaluation score at each time; c is the evaluation index I n The variance of the first quality evaluation score at each time; The maximum amplitude of the first quality evaluation score of the evaluation index I n .

[0032] According to the information service quality evaluation method provided by the application, before the measurement value of the target evaluation event and the target time corresponding to the target evaluation event are obtained, the method comprises the following steps of:

[0033] Disassembling the target technical standard document to generate a plurality of technical standard knowledge point documents;

[0034] Performing entity recognition on the technical standard knowledge point document to determine the target entity corresponding to the technical standard knowledge point document;

[0035] Splitting the technical standard knowledge point document to obtain a plurality of sentence documents;

[0036] Based on the association relationship between the sentence documents and the target entity, the target technical standard knowledge graph is obtained.

[0037] According to the information service quality evaluation method provided by the application, the target entity corresponding to the technical standard knowledge point document is determined by performing entity recognition on the technical standard knowledge point document, and the method comprises the following steps of:

[0038] Identifying the evaluation event class of the standard evaluation index of the technical standard knowledge point document to determine the first entity corresponding to the technical standard knowledge point document;

[0039] The first entity is marked with entity related information corresponding to the first entity to obtain the target entity; wherein,

[0040] The entity-related information comprises: a standard parameter of each evaluation event in an evaluation event class, and a threshold value corresponding to the standard parameter.

[0041] The application further provides an information service quality evaluation device, comprising:

[0042] The first processing module is configured to obtain a measurement value of a target evaluation event and a target moment corresponding to the target evaluation event, the target evaluation event being an actually occurred evaluation event.

[0043] The second processing module is configured to calculate a product between a weight of a target evaluation index corresponding to the target evaluation event in a plurality of evaluation indexes and a ratio between an important index of the target evaluation index to the information service at the target moment and an actual score corresponding to the target evaluation index at the target moment, and determine a first quality evaluation score of the target evaluation index at the target moment.

[0044] The third processing module is configured to determine a target quality evaluation score of the information service quality based on a maximum amplitude of the first quality evaluation scores of the target evaluation index at each moment, an average value of the first quality evaluation scores of the target evaluation index at each moment, an expectation of the first quality evaluation scores of the target evaluation index at each moment, and a variance of the first quality evaluation scores of the target evaluation index at each moment.

[0045] The fourth processing module is configured to determine an evaluation result of the information service quality based on the target quality evaluation score and a target threshold value.

[0046] The actual score corresponding to the target evaluation index at the target moment and the important index of the target evaluation index to the information service at the target moment are both determined based on a target technical standard knowledge graph and the measurement value of the target evaluation event.

[0047] The application further provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the information service quality evaluation method of the above-mentioned target when executing the program.

[0048] The application further provides a non-transitory computer readable storage medium having a computer program stored thereon, wherein the computer program is executable by a processor to implement the information service quality evaluation method of the above-mentioned target.

[0049] The application further provides a computer program product comprising a computer program, wherein the computer program is executable by a processor to implement the information service quality evaluation method of the above-mentioned target.

[0050] The information service quality evaluation method and the information service quality evaluation device provided by the application determine the first quality evaluation score of the target evaluation index at the target time according to the measurement value of each evaluation event at each time based on the actual occurrence of each evaluation event, and then determine the final target quality evaluation score of the information service based on the first quality evaluation score of each evaluation index at each time, so that when the quality of the information service is evaluated, the evaluation index and the specific information of the evaluation event of the information technology service at each time can be considered according to the actual situation, and the service quality score at each time is calculated according to the evaluation index and the specific information of the evaluation event, and then the final quality score is obtained, the accuracy of the evaluation result is significantly improved, and the real-time updating of the information service quality score is realized. BRIEF DESCRIPTION OF DRAWINGS

[0051] In order to more clearly illustrate the technical solutions in the application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description are some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor.

[0052] Figure 1 It is a flowchart of the information service quality evaluation method provided by the application;

[0053] Figure 2 It is a structural schematic diagram of the information service quality evaluation device provided by the application;

[0054] Figure 3 It is a structural schematic diagram of the electronic device provided by the application. DETAILED DESCRIPTION

[0055] In order to make the objects, technical solutions and advantages of the application clearer, the technical solutions in the application will be described clearly and completely in combination with the drawings in the application. Obviously, the described embodiments are some embodiments of the application, not all embodiments. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor belong to the protection scope of the application.

[0056] The information service quality evaluation method of the application will be described below in combination with Figure 1 The information service quality evaluation method of the application will be described below in combination with

[0057] The execution subject of the information service quality evaluation method can be an information service quality evaluation device, or can be a server, or can also be a terminal of a user, including but not limited to mobile terminals such as mobile phones or tablet computers and non-mobile terminals such as PC terminals.

[0058] As shown in Figure 1As shown, the information service quality evaluation method comprises steps 110, 120, 130 and 140.

[0059] Step 110, obtaining a measurement value of a target evaluation event and a target moment corresponding to the target evaluation event, the target evaluation event being an actually-occurred evaluation event;

[0060] In this step, the evaluation event is an event for evaluating the information service, such as a network attack event and a network stability event.

[0061] It can be understood that different evaluation events correspond to different categories, and different categories of evaluation events can be used to evaluate different evaluation indexes.

[0062] The evaluation indexes include safety, reliability, responsiveness, tangibility and friendliness, etc.

[0063] In the present application, the evaluation event corresponding to each evaluation index can be determined as an evaluation event category, and a corresponding relationship between the evaluation indexes and the evaluation event categories is established; for example, a network attack event can be used to evaluate a safety evaluation index, and a corresponding relationship between the safety evaluation index and the network attack event is established.

[0064] The target evaluation event is any one of all actually-occurred evaluation events.

[0065] It can be understood that for a certain category of evaluation event, it can not occur, or occur once, or occur multiple times in the target period; each occurrence corresponds to a moment.

[0066] The multiple moments are the moments corresponding to all actually-occurred evaluation events in the target period.

[0067] The target moment can be any moment of the multiple moments.

[0068] The target moment is used to represent the actual occurrence moment of the target evaluation event.

[0069] In the present application, all evaluation events occurring in the target period are obtained, and each evaluation event of all evaluation events is determined as a target evaluation event, and the moment corresponding to each actually-occurred evaluation event is determined as the moment corresponding to the evaluation event, that is, each target evaluation event corresponds to a target moment.

[0070] The measurement value is a parameter related to the target evaluation event, and is used to represent the occurrence of the target evaluation event.

[0071] For example, in the case of evaluating an event as a network attack event, the measurement value of the evaluation event at the target moment can represent the number of network attacks occurring at the target moment.

[0072] In actual implementation, the measurement value of the target evaluation event at the target moment and the moment corresponding to the evaluation event can be stored in a local database or a cloud database, and can be called when needed.

[0073] In the present application, by obtaining actually occurring evaluation events instead of evaluation events corresponding to each evaluation index, the amount of evaluation data can be reduced, thereby reducing the evaluation time.

[0074] Step 120, calculating the product between the weight of the target evaluation index corresponding to the target evaluation event in the plurality of evaluation indexes and the ratio between the important index of the target evaluation index to the information service at the target moment and the actual score corresponding to the target evaluation index at the target moment, to determine the first quality evaluation score of the target evaluation index at the target moment;

[0075] In this step, the actual score corresponding to the target evaluation index at the target moment and the important index of the target evaluation index to the information service at the target moment are both determined based on the target technical standard knowledge graph and the measurement value of the target evaluation event.

[0076] Among them, the target evaluation index is an evaluation index corresponding to the target evaluation event in the plurality of evaluation indexes.

[0077] Different evaluation indexes correspond to different categories of evaluation events, that is, different evaluation event classes.

[0078] Each evaluation index corresponds to a weight, and for the same evaluation index, the corresponding weight in information service evaluation with different focus points can also be different.

[0079] For example, for the security index, the corresponding weight in information service evaluation with a focus on network security can be higher than the corresponding weight in information service evaluation with a focus on information service efficiency.

[0080] The weight is determined based on the degree of focus of the focus point, and is used to represent the case of each security index changing with the focus point.

[0081] The user can set different weights based on the actual situation.

[0082] In actual implementation, the target evaluation index and the evaluation event corresponding to the target evaluation index can be determined from the target technical standard knowledge graph.

[0083] Among them, the target technical standard knowledge graph is a pre-generated technical standard knowledge graph used for evaluating information technology services.

[0084] It should be noted that in practical applications, the evaluation indicators of information technology services and the evaluation events corresponding to each evaluation indicator will change over time. The importance of the same evaluation indicator to the same evaluation event may also vary at different times.

[0085] The target evaluation index is a key indicator of the information service at the target time. It is used to characterize the importance of the evaluation index to the information service at that time. This key indicator may change in real time over time.

[0086] For example, for the same cyberattack incident, as users' requirements for network security increase, security indicators may be more important to information services at the current moment than they were before.

[0087] Importance is used to characterize how each security indicator changes over time.

[0088] The actual score corresponding to the target evaluation index at the target time is used to characterize the score determined by evaluating the measured value of the evaluation event corresponding to a certain evaluation index at a certain time.

[0089] The first quality evaluation score is used to characterize the quality evaluation score of the target evaluation index at the target time.

[0090] It is understandable that the quality evaluation score of the same evaluation indicator may be different at different times, and the quality evaluation score of different evaluation indicators may also be different at the same time.

[0091] In some embodiments, the formula can be used:

[0092]

[0093] Determine the first quality evaluation score of the target evaluation indicator at the target time.

[0094] in, Evaluation index I n The first quality assessment score at time t; Evaluation index I n Key indicators for information services at time t; Evaluation index I n The weights; Evaluation index I at time t n Corresponding actual score; I n Used to characterize the target evaluation index; n represents the evaluation index I. na number of corresponding evaluation events, and n is a positive integer; and the target time is a target time.

[0095] The following describes a determination manner of the actual score corresponding to the target evaluation index at the target time.

[0096] In some embodiments, the actual score corresponding to the target evaluation index at the target time is determined by the following steps:

[0097] determining a standard reference value of the evaluation target evaluation event under the target evaluation index based on the target technical standard knowledge graph;

[0098] determining the actual score corresponding to the target evaluation index at the target time based on the standard reference value of the evaluation target evaluation event under the target evaluation index, a mean value of the measurement values of the evaluation target evaluation event under the target evaluation index at each time, and the measurement value of the evaluation target evaluation event under the target evaluation index at the target time.

[0099] In this embodiment, the actual score corresponding to the target evaluation index at the target time represents an actual value corresponding to the target evaluation index under comprehensive consideration of the influence of time change on the evaluation index.

[0100] The mean value of the measurement values of the evaluation target evaluation event under the target evaluation index at each time is a mean value of the measurement values of the evaluation target evaluation event under the target evaluation index at a plurality of times within a target time period.

[0101] For example, in a case where the target evaluation event is a network attack event, network attack events occur at n times within a target time period, and the number of network attacks occurring at each time is N1, N2, and N3 times, respectively; then the mean value of the measurement values of the evaluation target evaluation event under the target evaluation index at each time is (N1+N2+N3) / n times.

[0102] The standard reference value of the evaluation target evaluation event under the target evaluation index is a value of the evaluation event under the evaluation index determined based on the target technical standard knowledge graph without considering the change of the evaluation index with time.

[0103] It can be understood that the standard reference value of the evaluation target evaluation event under the target evaluation index can be determined based on the target technical standard knowledge graph.

[0104] In some embodiments, the actual score corresponding to the target evaluation index at the target time can be determined by the formula:

[0105]

[0106] determining, wherein,​​ evaluation index I at time t n corresponding actual score; ψ i characterizing evaluation index I n standard reference value of evaluation event i characterizing evaluation index I n average of measurement values of corresponding evaluation events at each time characterizing evaluation index I n measurement value of evaluation event i at time t; n represents evaluation index I n number of corresponding evaluation events, and n is a positive integer; i is the i-th evaluation event.

[0107] In this embodiment, evaluation index I n standard reference value ψ of evaluation event i i can be directly found from the target technical standard knowledge graph.

[0108] evaluation index I n average of measurement values of corresponding evaluation events at each time can be determined by averaging the measurement values of all evaluation events at each time. n

[0109] In some embodiments, the importance of the target evaluation index to the information service at the target time is determined by the following steps:

[0110] based on the target technical standard knowledge graph, determining the standard score corresponding to the target evaluation index at the target time;

[0111] based on the difference between the actual score corresponding to the target evaluation index at the target time and the standard score corresponding to the target evaluation index at the target time, and the ratio between the actual score corresponding to the target evaluation index at the target time and the standard score corresponding to the target evaluation index at the target time, determining the importance of the target evaluation index to the information service at the target time.

[0112] In this embodiment, the standard score corresponding to the target evaluation index at the target time is the score corresponding to the target evaluation index determined based on the target technical standard knowledge graph in consideration of the change of the evaluation index over time.

[0113] The determination method of the actual score corresponding to the target evaluation index at the target time has been described in the above embodiments, and will not be repeated here.

[0114] In some embodiments, the importance of the target evaluation index to the information service at the target time can be determined by the formula:

[0115]

[0116] ​Determination, wherein, For evaluation index I n The important index of the information service at time t; For evaluation index I at time t n The corresponding actual score. For evaluation index I at time t n The corresponding standard score.

[0117] For evaluation index I at time t n The corresponding standard score The target technical standard knowledge graph can be determined based on.

[0118] It should be noted that each target evaluation event involved in the actual application process can be historical data, and the target technical standard knowledge graph can be supplemented and optimized by taking the historical data as standard entity related information.

[0119] In some embodiments, based on the regularity of historical data, an association relationship between a historical evaluation event and a time value corresponding to the historical evaluation event can be established, thereby establishing a corresponding relationship between each target entity and a time value in the target technical standard knowledge graph.

[0120] In subsequent application processes, by searching the target technical standard knowledge graph, the standard score corresponding to the target evaluation index at the target time can be directly obtained.

[0121] Step 130, based on the maximum amplitude of the first quality evaluation score of the target evaluation index at each time, the average value of the first quality evaluation score of the target evaluation index at each time, the expectation of the first quality evaluation score of the target evaluation index at each time, and the variance of the first quality evaluation score of the target evaluation index at each time, determine the target quality evaluation score of the information service quality;

[0122] In this step, the target quality evaluation score is the overall evaluation score of the information service quality obtained by comprehensively evaluating the evaluation events corresponding to multiple evaluation indexes at different times.

[0123] Based on step 120, the first quality evaluation score of all evaluation indexes at each time can be generated, and multiple first quality evaluation scores are obtained; then by comprehensively processing the multiple first quality evaluation scores, the target quality evaluation score can be generated.

[0124] The maximum amplitude of the first quality evaluation score of the target evaluation index at each time is the absolute value of the difference between the maximum value and the minimum value of the multiple first quality evaluation scores corresponding to the target period of the evaluation index.

[0125] The average of the first quality evaluation scores at each time point, i.e., the average of the plurality of first quality evaluation scores corresponding to the target period.

[0126] The variance of the first quality evaluation scores at each time point, i.e., the variance of the plurality of first quality evaluation scores corresponding to the target period.

[0127] In some embodiments, the target quality evaluation score of the information service quality can be determined based on the formula:

[0128]

[0129] The target quality evaluation score of the information service quality is determined, wherein F is the target quality evaluation score of the information service quality; x is the evaluation index I n The average of the first quality evaluation scores at each time point; b is the evaluation index I n The expectation of the quality evaluation scores at each time point; c is the evaluation index I n The variance of the first quality evaluation scores at each time point; The evaluation index I n The maximum amplitude of the first quality evaluation scores.

[0130] Step 140, determining the evaluation result of the information service quality based on the target quality evaluation score and the target threshold.

[0131] In this step, the target threshold can be customized by the user.

[0132] The target threshold is a standard value for evaluating the high and low of the target quality evaluation score.

[0133] The evaluation result is used to represent the high and low of the information service quality.

[0134] For example, the evaluation result can include: the information service quality is poor and the information service quality is good.

[0135] It can be understood that, in the case that the target quality evaluation score is lower than the target threshold, the target quality evaluation score is lower, and then the evaluation result used to represent the information service quality is poor is generated.

[0136] In the case that the target quality evaluation score is not lower than the target threshold, the target quality evaluation score is higher, and then the evaluation result used to represent the information service quality is good is generated.

[0137] Of course, in other embodiments, a plurality of target thresholds can also be set to divide the information service quality into three or more grades, which is not limited by the application.

[0138] The following takes the target evaluation index as the safety index, and the network attack event as an example to illustrate the specific implementation of the method.

[0139] For example, network attack events occur at times t1, t2 and t3 respectively, wherein the network attack event occurring at time t1 is denoted as i1, the network attack event occurring at time t2 is denoted as i2, and the network attack event occurring at time t3 is denoted as i3.

[0140] The measurement value corresponding to i1 is x1, the measurement value corresponding to i2 is x2, and the measurement value corresponding to i3 is x3.

[0141] In actual execution, when calculating the first quality evaluation score of the safety index at time t1, the standard reference value ψ1 of the evaluation event i1 under the safety index is directly obtained based on the target technical standard knowledge graph;

[0142] Then the average value of the measurement values of the network attack events at times t1, t2 and t3 under the safety index is calculated and the measurement value α(t) of the evaluation event i1 at time t1 under the safety index i , α(t) i =x1.

[0143] The above values are brought into the above formula (2), wherein n=3, so as to determine the actual score corresponding to the safety index at time t1

[0144] Based on the same method, the actual scores corresponding to the safety index at times t2 and t3 can be obtained respectively; and the actual scores corresponding to other evaluation indexes at times can be obtained respectively.

[0145] After obtaining the actual scores corresponding to each evaluation index at each time, the importance index of the target evaluation index to the information service at the target time can be calculated.

[0146] Continuing to take the importance index of the safety index to the information service at time t1 as an example, the standard score corresponding to the safety index at time t1 can be directly obtained from the target technical standard knowledge graph Then, according to the actual score corresponding to the safety index at time t1 determined before, the above formula (3) is brought into, so as to determine the importance index of the safety index to the information service at time t1

[0147] Based on the same method, the importance index of the safety index to the information service at times t2 and t3 can be obtained respectively, and the importance index of other evaluation indexes to the information service at times can be obtained respectively.

[0148] obtaining the actual score corresponding to the security index at the time t1 and the importance of the security index to the information service at the time t1 Then, the two are brought into the above formula (1), and the first quality evaluation score of the security index at the time t1 is obtained.

[0149] It can be understood that when there are multiple actual evaluation events corresponding to the security index at the time t1, the actual score corresponding to the security index corresponding to each evaluation event at the time t1 can be obtained respectively, and the sum is calculated, as shown in formula (1), to generate the final first quality evaluation score.

[0150] Based on the same method, the first quality evaluation score of each evaluation index at each time can be obtained, and then the target quality evaluation score can be determined based on the above formula (4), and the evaluation result of the information service quality can be determined based on the target quality evaluation score.

[0151] For example, when the first quality evaluation scores of the security index at the times t1, t2 and t3 are f1(t1), f2(t2) and f3(t3) respectively, and f1(t1) is the maximum and f3(t3) is the minimum, the maximum amplitude of the first quality evaluation score of the security index is [f1(t1)-f3(t3)].

[0152] According to the information service quality evaluation method provided by the embodiment of the application, the first quality evaluation score of the target evaluation index at the target time is determined by obtaining the target evaluation event according to the measurement value of each evaluation event actually occurring at each time; then the final target quality evaluation score of the information service is determined based on the first quality evaluation score of each evaluation index at each time; so that when the quality of the information service is evaluated, the evaluation index and the specific information of the evaluation event of the information technology service at each time can be considered according to the actual situation, and the service quality score at each time is calculated according to the evaluation index and the specific information of the evaluation event, and then the final quality score is obtained, which significantly improves the accuracy of the evaluation result, and realizes the real-time updating of the information service quality score.

[0153] In some embodiments, before step 110, the method can include:

[0154] disassembling the target technical standard document to generate a plurality of technical standard knowledge point documents;

[0155] performing entity recognition on the technical standard knowledge point document to determine the target entity corresponding to the technical standard knowledge point document;

[0156] segmenting the technical standard knowledge point document to obtain a plurality of sentence documents;

[0157] Based on the correlation between the sentence documents and the target entity, a target technical standard knowledge graph is acquired.

[0158] In this embodiment, the target technical standard knowledge graph comprises three elements of entity, attribute and relationship.

[0159] The entity is an objectively existing and distinguishable thing, and in the present application, the entity can be an evaluation event class.

[0160] The attribute is used to represent the attributes of each entity, and in the present application, the attribute can be a standard parameter corresponding to each evaluation event and a threshold value corresponding to a labeled parameter.

[0161] For example, in the case of an evaluation event of a network attack event, the standard parameter corresponding to the evaluation event can be the number of attacks, and the threshold value corresponding to the standard parameter can be 1 or 2. It can be understood that in the case that the actual number of attacks does not exceed the attack threshold, it is considered that no network attack event has occurred.

[0162] The relationship is used to represent the correlation between entities, and in the present application, the relationship can represent the correlation between different evaluation events.

[0163] The target technical standard document can be a standard technical evaluation document in the field of information service quality.

[0164] The technical standard knowledge point document can include but is not limited to a service range knowledge point document of information technology service and a standard evaluation index knowledge point document of information technology service.

[0165] The target entity includes various evaluation event entities and standard parameters corresponding to the evaluation events.

[0166] In some embodiments, the technical standard knowledge point document is subjected to entity recognition to determine the target entity corresponding to the technical standard knowledge point document, which can include:

[0167] The evaluation event class of the standard evaluation index of the technical standard knowledge point document is identified to determine the first entity corresponding to the technical standard knowledge point document.

[0168] The first entity is labeled with entity-related information corresponding to the first entity to acquire the target entity; wherein

[0169] The entity-related information includes the standard parameters of each evaluation event in the evaluation event class and the threshold value corresponding to the standard parameter.

[0170] In this embodiment, the first entity is each evaluation event class, that is, each evaluation index corresponding to the evaluation event.

[0171] The evaluation event class of the standard evaluation index of the technical standard knowledge point document is identified, a plurality of evaluation event classes can be obtained, and different evaluation event classes are determined as different first entities.

[0172] For example, the first entity can include: an entity corresponding to a network attack event and an entity corresponding to a network stability event.

[0173] The entity-related information corresponding to each first entity is obtained, the entity-related information is determined as the attribute corresponding to the first entity, and the first entity is labeled based on the entity-related information to generate a target entity.

[0174] The entity-related information includes: standard parameters of each evaluation event in the evaluation event class, and threshold values corresponding to the standard parameters.

[0175] Therefore, different target entities represent different evaluation events and standard parameters corresponding to the evaluation events.

[0176] In some embodiments, the technical standard knowledge point document is segmented to obtain a plurality of sentence documents, which can include: segmenting the technical standard knowledge point document based on a sentence symbol to obtain a plurality of sentence documents.

[0177] In this embodiment, the sentence document is a sentence of text in the technical standard knowledge point.

[0178] The sentence symbol includes but is not limited to a period, a semicolon, and an ellipsis.

[0179] In actual execution, each technical standard knowledge point document is segmented according to the period, semicolon, and ellipsis to obtain a plurality of sentence documents.

[0180] In this embodiment, by segmenting each technical standard knowledge point document according to the period, semicolon, and other sentence symbols, a plurality of sentence documents are obtained, which can realize fine segmentation of the technical standard knowledge point document, thereby helping to improve the accuracy of the association relationship between the target entities in the target technical standard knowledge graph established subsequently, to provide accurate search and improve the precision and accuracy of the final evaluation result.

[0181] After obtaining the target entity and the sentence document, the association relationship between each sentence document is obtained, and the association relationship between each target entity is established based on the target entity corresponding to each sentence document, thereby generating a target technical standard knowledge graph.

[0182] It can be understood that in the target technical standard knowledge graph, a plurality of evaluation event classes, related information corresponding to each evaluation event class, and association relationships between each evaluation event class are included.

[0183] Based on each evaluation event class, the relevant information corresponding to each evaluation event class, and the correlation between each evaluation event class, the evaluation standard value corresponding to the target evaluation indicator under standard conditions can be generated.

[0184] The information service quality evaluation method provided by the embodiments of the present invention establishes a target technical standard knowledge graph before actual application to describe in detail the relationship between various entities in the target technical standard document, realize the linking of entities, and provide accurate search. Thus, it ensures that when users extract knowledge points (such as evaluation indicators and evaluation event classes) in the document data, they can quickly extract the required evaluation indicators and evaluation event classes without traversing the entire document data, which significantly improves the evaluation efficiency, accuracy, and precision of the evaluation results.

[0185] The information service quality evaluation device provided by the present invention is described below. The information service quality evaluation device described below and the information service quality evaluation method described above can be referred to in correspondence.

[0186] like Figure 2 As shown, the information service quality evaluation device includes: a first processing module 210, a second processing module 220, a third processing module 230, and a fourth processing module 240.

[0187] The first processing module 210 is used to obtain the measurement value of the target evaluation event and the target time corresponding to the target evaluation event, wherein the target evaluation event is the evaluation event that actually occurs.

[0188] The second processing module 220 is used to calculate the weight of the target evaluation indicator corresponding to the target evaluation event among multiple evaluation indicators, and the product of the weight of the target evaluation indicator at the target time and the ratio of the important indicators of the target evaluation indicator for information service at the target time to the actual score of the target evaluation indicator at the target time, to determine the first quality evaluation score of the target evaluation indicator at the target time.

[0189] The third processing module 230 is used to determine the target quality evaluation score of information service quality based on the maximum amplitude of the first quality evaluation score of the target evaluation index at each time, the average value of the first quality evaluation score of the target evaluation index at each time, the expectation of the first quality evaluation score of the target evaluation index at each time, and the variance of the first quality evaluation score of the target evaluation index at each time.

[0190] The fourth processing module 240 is used to determine the evaluation result of information service quality based on the target quality evaluation score and the target threshold.

[0191] Among them, the actual score corresponding to the target evaluation index at the target time and the important indicators of the target evaluation index for information service at the target time are both determined based on the target technical standard knowledge graph and the measurement value of the target evaluation event.

[0192] The information service quality evaluation device provided by the embodiment of the present application can obtain the target evaluation event to determine the first quality evaluation score of the target evaluation index at the target time according to the measurement value of each evaluation event at each time; and then determine the target quality evaluation score of the final information service based on the first quality evaluation score of each evaluation index at each time, so that when the quality of the information service is evaluated, the evaluation index of the information technology service at each time and the specific information of the evaluation event can be considered according to the actual situation, and the service quality score at each time is calculated according to the evaluation index and the specific information of the evaluation event, and then the final quality score is obtained, which significantly improves the accuracy of the evaluation result, and realizes the real-time updating of the information service quality score,

[0193] In some embodiments, the second processing module 220 can also be configured to:

[0194] by the formula:

[0195]

[0196] determine the first quality evaluation score of the target evaluation index at the target time, wherein, is the evaluation index I n the first quality evaluation score at t time; is the evaluation index I n the important index of the information service at t time; is the weight of the evaluation index I n ; and is the actual score corresponding to the evaluation index I n at t time; I n is used to represent the target evaluation index; n represents the number of the corresponding evaluation events of the evaluation index I n , and n is a positive integer.

[0197] In some embodiments, the device can further include a fifth processing module configured to:

[0198] determine the standard score corresponding to the target evaluation index at the target time based on the target technology standard knowledge graph;

[0199] determine the important index of the information service of the target evaluation index at the target time based on the difference between the actual score corresponding to the target evaluation index at the target time and the standard score corresponding to the target evaluation index at the target time, and the ratio between the standard score corresponding to the target evaluation index at the target time.

[0200] In some embodiments, the fifth processing module can also be configured to:

[0201] by the formula:

[0202]

[0203] determining an important indicator of the information service at the target moment by the target evaluation indicator, wherein, is an important indicator of the information service at the target moment by the evaluation indicator I n is an important indicator of the information service at the target moment by the evaluation indicator I is an important indicator of the information service at the target moment by the evaluation indicator I n is the corresponding actual score of the evaluation indicator I is an important indicator of the information service at the target moment by the evaluation indicator I n is the corresponding standard score of the evaluation indicator I

[0204] In some embodiments, the apparatus can further include a sixth processing module configured to:

[0205] determine a standard reference value of the evaluation target evaluation event by the target evaluation indicator based on the target technical standard knowledge graph;

[0206] determine the corresponding actual score of the evaluation indicator I at the target moment based on the standard reference value of the evaluation target evaluation event by the target evaluation indicator, the mean value of the measurement value of the corresponding evaluation event of the evaluation indicator I at each moment, and the measurement value of the evaluation target evaluation event by the target evaluation indicator at the target moment.

[0207] In some embodiments, the third processing module 230 can be further configured to:

[0208] by the formula:

[0209]

[0210] determine the corresponding actual score of the evaluation indicator I at the target moment, wherein, is an important indicator of the information service at the target moment by the evaluation indicator I n is the corresponding actual score of the evaluation indicator I; ψ i denotes the standard reference value of the evaluation event i by the evaluation indicator I n denotes the mean value of the measurement value of the corresponding evaluation event at each moment by the evaluation indicator I n denotes the measurement value of the evaluation event i at the target moment by the evaluation indicator I n n denotes the number of the corresponding evaluation events of the evaluation indicator I n n is a positive integer; i is the i-th evaluation event.

[0211] In some embodiments, the second processing module 210 can be further configured to:

[0212] by the formula:

[0213] ​​

[0214] a target quality evaluation score of the information service quality, wherein F is the target quality evaluation score; x is an evaluation index I n an average value of the first quality evaluation score at each time; b is an evaluation index I n an expectation of the quality evaluation score at each time; c is an evaluation index I n a variance of the first quality evaluation score at each time; an evaluation index I n a maximum amplitude of the first quality evaluation score.

[0215] In some embodiments, the apparatus can further include a seventh processing module configured to:

[0216] Before obtaining the measurement value of the target evaluation event at the target time in the plurality of times, disassemble the target technical standard document to obtain a plurality of technical standard knowledge point documents;

[0217] Perform entity recognition on the technical standard knowledge point documents to determine target entities corresponding to the technical standard knowledge point documents;

[0218] Split the technical standard knowledge point documents to obtain a plurality of sentence documents;

[0219] Obtain a target technical standard knowledge graph based on the association relationship between the sentence documents and the target entities.

[0220] In some embodiments, the seventh processing module can be further configured to:

[0221] Identify an evaluation event class of a standard evaluation index of the technical standard knowledge point document to determine a first entity corresponding to the technical standard knowledge point document;

[0222] Annotate the first entity with entity-related information corresponding to the first entity to generate the target entity; wherein

[0223] The entity-related information includes standard parameters of each evaluation event in the evaluation event class and threshold values corresponding to the standard parameters.

[0224] Figure 3 An example of an entity structure diagram of an electronic device is shown in FIG. 1. Figure 3As shown, the electronic device can include a processor 310, a communications interface 320, a memory 330, and a communications bus 340, wherein the processor 310, the communications interface 320, and the memory 330 complete communications with each other through the communications bus 340. The processor 310 can invoke a logical instruction in the memory 330 to execute an information service quality evaluation method, which includes: obtaining a measurement value of a target evaluation event and a target moment corresponding to the target evaluation event, the target evaluation event being an actually occurring evaluation event; calculating a weight of a target evaluation index corresponding to the target evaluation event in a plurality of evaluation indexes, a product between a ratio of an important index of the target evaluation index to the information service at the target moment and an actual score corresponding to the target evaluation index at the target moment, determining a first quality evaluation score of the target evaluation index at the target moment; determining a target quality evaluation score of the information service quality based on a maximum amplitude of the first quality evaluation score of the target evaluation index at each moment, an average value of the first quality evaluation score of the target evaluation index at each moment, an expectation of the first quality evaluation score of the target evaluation index at each moment, and a variance of the first quality evaluation score of the target evaluation index at each moment; determining an evaluation result of the information service quality based on the target quality evaluation score and a target threshold value; wherein the actual score corresponding to the target evaluation index at the target moment and the important index of the target evaluation index to the information service at the target moment are both determined based on a target technical standard knowledge graph and the measurement value of the target evaluation event.

[0225] In addition, the logical instruction in the memory 330 described above can be implemented in the form of a software function unit and sold or used as an independent product, which can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or parts of the prior art that contribute to the present application or parts of the technical solutions can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a plurality of instructions for making a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in the embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0226] In another aspect, the present application also provides a computer program product, which comprises a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions which, when executed by a computer, enable the computer to perform the information service quality evaluation method provided by any of the above methods, the method comprising: obtaining a measurement value of a target evaluation event and a target time corresponding to the target evaluation event, the target evaluation event being an actually occurred evaluation event; calculating a weight of a target evaluation index corresponding to the target evaluation event in a plurality of evaluation indexes, and a product between the weight and a ratio between an important index of the target evaluation index to the information service at the target time and an actual score corresponding to the target evaluation index at the target time, to determine a first quality evaluation score of the target evaluation index at the target time; determining a target quality evaluation score of the information service quality based on a maximum amplitude of the first quality evaluation scores of the target evaluation index at each time, an average value of the first quality evaluation scores of the target evaluation index at each time, an expectation of the first quality evaluation scores of the target evaluation index at each time, and a variance of the first quality evaluation scores of the target evaluation index at each time; determining an evaluation result of the information service quality based on the target quality evaluation score and a target threshold value; wherein the actual score corresponding to the target evaluation index at the target time and the important index of the target evaluation index to the information service at the target time are both determined based on a target technical standard knowledge graph and the measurement value of the target evaluation event.

[0227] In yet another aspect, the present application also provides a non-transitory computer-readable storage medium having a computer program stored thereon, the computer program being executed by a processor to implement the information service quality evaluation method provided by any of the above methods, the method comprising: obtaining a measurement value of a target evaluation event and a target time corresponding to the target evaluation event, the target evaluation event being an actually occurred evaluation event; calculating a weight of a target evaluation index corresponding to the target evaluation event in a plurality of evaluation indexes, and a product between the weight and a ratio between an important index of the target evaluation index to the information service at the target time and an actual score corresponding to the target evaluation index at the target time, to determine a first quality evaluation score of the target evaluation index at the target time; determining a target quality evaluation score of the information service quality based on a maximum amplitude of the first quality evaluation scores of the target evaluation index at each time, an average value of the first quality evaluation scores of the target evaluation index at each time, an expectation of the first quality evaluation scores of the target evaluation index at each time, and a variance of the first quality evaluation scores of the target evaluation index at each time; determining an evaluation result of the information service quality based on the target quality evaluation score and a target threshold value; wherein the actual score corresponding to the target evaluation index at the target time and the important index of the target evaluation index to the information service at the target time are both determined based on a target technical standard knowledge graph and the measurement value of the target evaluation event.

[0228] The device embodiments described above are merely illustrative, wherein the units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed to multiple network units. Part or all of the modules can be selected to achieve the purposes of the embodiments according to actual needs. Those skilled in the art can understand and implement without creative labor.

[0229] Through the description of the above embodiments, those skilled in the art can clearly understand that the embodiments can be realized by means of software and the necessary general hardware platform, and of course can also be realized by hardware. Based on such understanding, the above technical solutions can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute the methods described in each embodiment or some parts of the embodiments.

[0230] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for evaluating the quality of information services, characterized in that, include: The measurement value of the target evaluation event and the target time corresponding to the target evaluation event are obtained, wherein the target evaluation event is an actual evaluation event that has occurred; and the evaluation event is selected from network attack events and network stability events. Based on the target technical standard knowledge graph, the standard reference value for evaluating the target evaluation event under the target evaluation index is determined, and the standard score corresponding to the target evaluation index at the target time is determined. Based on the standard reference value for evaluating the target evaluation event under the target evaluation index, the mean value of the measurement value of the evaluation event corresponding to the target evaluation index at each time, and the measurement value of the target evaluation event under the target evaluation index at the target time, the actual score corresponding to the target evaluation index at the target time is determined; Based on the ratio between the difference between the actual score and the standard score of the target evaluation indicator at the target time and the standard score of the target evaluation indicator at the target time, the important indicator of the target evaluation indicator for information service at the target time is determined. Calculate the weight of the target evaluation indicator corresponding to the target evaluation event among multiple evaluation indicators, and multiply it by the ratio of the important indicator of the target evaluation indicator for information service at the target time to the actual score corresponding to the target evaluation indicator at the target time, to determine the first quality evaluation score of the target evaluation indicator at the target time; The target quality evaluation score of the information service is determined based on the maximum amplitude of the first quality evaluation score of the target evaluation index at each time, the average value of the first quality evaluation score of the target evaluation index at each time, the expected value of the first quality evaluation score of the target evaluation index at each time, and the variance of the first quality evaluation score of the target evaluation index at each time. Based on the target quality evaluation score and the target threshold, the evaluation result of the information service quality is determined; The target technology standard knowledge graph is constructed through the following steps: Deconstruct the target technical standard document and generate multiple technical standard knowledge point documents; The evaluation event class of the standard evaluation indicators of the technical standard knowledge point document is identified to determine the first entity corresponding to the technical standard knowledge point document; The first entity is labeled with entity-related information corresponding to the first entity to obtain the target entity; wherein, the entity-related information includes: standard parameters of each evaluation event in the evaluation event class, and the threshold corresponding to the standard parameters; The document containing the technical standard knowledge points is segmented to obtain multiple sentence documents; Based on the relationships between the sentence documents and the target entity, obtain the target technical standard knowledge graph.

2. The information service quality evaluation method according to claim 1, characterized in that, The step of calculating the weight of the target evaluation indicator corresponding to the target evaluation event among multiple evaluation indicators, and multiplying it with the ratio of the important indicator of the target evaluation indicator for information service at the target time to the actual score corresponding to the target evaluation indicator at the target time, to determine the first quality evaluation score of the target evaluation indicator at the target time, includes: Through the formula: It is confirmed that, among them, Evaluation indicators The first quality assessment score at time t; The evaluation index Key indicators for information services at time t; The evaluation index The weights; Evaluation index at time t The corresponding actual score; Used to characterize the target evaluation index; n represents the evaluation index. The number of corresponding evaluation events, where n is a positive integer.

3. The information service quality evaluation method according to claim 1, characterized in that, The method of determining the importance of the target evaluation indicator for information services at the target time based on the ratio between the difference between the actual score and the standard score of the target evaluation indicator at the target time and the standard score of the target evaluation indicator at the target time includes: Through the formula: It is confirmed that, among them, Evaluation indicators Key indicators for information services at time t; The evaluation index at time t The corresponding actual score; The evaluation index at time t The corresponding standard score.

4. The information service quality evaluation method according to claim 1, characterized in that, The determination of the actual score corresponding to the target evaluation indicator at the target time based on the standard reference value for evaluating the target evaluation event under the target evaluation indicator, the mean of the measured values ​​of the evaluation event corresponding to the target evaluation indicator at each time, and the measured value of the target evaluation event under the target evaluation indicator at the target time includes: Through the formula: It is confirmed that, among them, Evaluation index at time t The corresponding actual score; Characterizing the evaluation index The standard reference value for evaluating event i; Characterizing the evaluation index The mean of the measured values ​​of the corresponding evaluation event at each time point; Characterizing the evaluation index The evaluation index is the measured value of event i at time t; n represents the evaluation index. The number of corresponding evaluation events, where n is a positive integer; i is the i-th evaluation event.

5. The information service quality evaluation method according to any one of claims 1-4, characterized in that, The determination of the target quality evaluation score for the information service quality based on the maximum amplitude of the first quality evaluation score of the target evaluation indicator at each time point, the average value of the first quality evaluation score of the target evaluation indicator at each time point, the expected value of the first quality evaluation score of the target evaluation indicator at each time point, and the variance of the first quality evaluation score of the target evaluation indicator at each time point includes: Through the formula: It is confirmed that, among them, Let x be the target quality evaluation score; x is the evaluation index. The average of the first quality evaluation scores at each time point; b is the evaluation index. The expected quality evaluation score at each time point; c is the evaluation index. The variance of the first quality assessment score at each time point; The evaluation index The maximum amplitude of the first quality evaluation score.

6. An information service quality evaluation device, characterized in that, include: The first processing module is used to acquire the measurement value of the target evaluation event and the target time corresponding to the target evaluation event, wherein the target evaluation event is an actual evaluation event that has occurred; wherein the evaluation event is selected from network attack events and network stability events; The second processing module is used to determine the standard reference value for evaluating the target evaluation event under the target evaluation index based on the target technical standard knowledge graph, and to determine the standard score corresponding to the target evaluation index at the target time. Based on the standard reference value for evaluating the target evaluation event under the target evaluation index, the mean value of the measurement value of the evaluation event corresponding to the target evaluation index at each time, and the measurement value of the target evaluation event under the target evaluation index at the target time, the actual score corresponding to the target evaluation index at the target time is determined; Based on the ratio between the difference between the actual score and the standard score of the target evaluation indicator at the target time and the standard score of the target evaluation indicator at the target time, the important indicator of the target evaluation indicator for information service at the target time is determined. Calculate the weight of the target evaluation indicator corresponding to the target evaluation event among multiple evaluation indicators, and multiply it by the ratio of the important indicator of the target evaluation indicator for information service at the target time to the actual score corresponding to the target evaluation indicator at the target time, to determine the first quality evaluation score of the target evaluation indicator at the target time; The third processing module is used to determine the target quality evaluation score of the information service quality based on the maximum amplitude of the first quality evaluation score of the target evaluation index at each time, the average value of the first quality evaluation score of the target evaluation index at each time, the expectation of the first quality evaluation score of the target evaluation index at each time, and the variance of the first quality evaluation score of the target evaluation index at each time. The fourth processing module is used to determine the evaluation result of the information service quality based on the target quality evaluation score and the target threshold. The target technology standard knowledge graph is constructed through the following steps: Deconstruct the target technical standard document and generate multiple technical standard knowledge point documents; The evaluation event class of the standard evaluation indicators of the technical standard knowledge point document is identified to determine the first entity corresponding to the technical standard knowledge point document; The first entity is labeled with entity-related information corresponding to the first entity to obtain the target entity; wherein, the entity-related information includes: standard parameters of each evaluation event in the evaluation event class, and the threshold corresponding to the standard parameters; The document containing the technical standard knowledge points is segmented to obtain multiple sentence documents; Based on the relationships between the sentence documents and the target entity, obtain the target technical standard knowledge graph.

Images