Information evaluation method and device, electronic equipment and storage medium

Abstract

The application provides an information evaluation method and device, electronic equipment and storage medium, and relates to the technical field of computers. The information value evaluation method comprises the following steps: obtaining to-be-evaluated text information; processing the to-be-evaluated text information based on a pre-trained first large language model to obtain information feature data; and obtaining an evaluation result of the to-be-evaluated text information based on the information feature data and a preset evaluation index system. Since the large language model can understand the meaning of the text information and can generate natural language text, the to-be-evaluated text information in different information formats can be processed by the first large language model, and information feature data can be obtained. Furthermore, the to-be-evaluated text information can be evaluated according to the obtained information feature data, so that the to-be-evaluated text information in different information formats can be evaluated, and the effect of automatically evaluating human-read information can be achieved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computers, in particular to an intelligence evaluation method and device, electronic equipment and a storage medium. BACKGROUND

[0002] Threat intelligence can generally be divided into machine-readable intelligence and human-readable intelligence. Machine-readable intelligence refers to machine-readable intelligence, and machine-readable intelligence can collect enough machine-readable intelligence to ensure that all major threats in machine-readable intelligence can be identified by machines, automatically filter out useless or duplicate data without consuming human resources, and provide current and historical variable data threats. Human-readable intelligence is highly condensed and mainly composed of security items, network entities and emerging hacker organizations, attacks, etc. It mainly solves the problem of information explosion and provides personalized intelligence for enterprises or users. Human-readable intelligence formats are diverse, including security bulletins, vulnerability warnings, virus / APT (Advanced Persistent Threat) analysis, etc. Due to different intelligence types of different intelligence formats, it is currently difficult to automatically evaluate the value of human-readable intelligence. SUMMARY

[0003] The present application provides an intelligence evaluation method and device, electronic equipment and a storage medium to solve the problem that it is difficult to automatically evaluate the value of human-readable intelligence due to different intelligence types of different intelligence formats in the prior art.

[0004] In a first aspect, the present application provides an intelligence value evaluation method, comprising: obtaining text intelligence to be evaluated; processing the text intelligence to be evaluated based on a pre-trained first large language model to obtain intelligence feature data; and obtaining an evaluation result of the text intelligence to be evaluated based on the intelligence feature data and a preset evaluation index system.

[0005] In the present application, the large language model can understand the meaning of the text intelligence and generate natural language text. Therefore, the first large language model can process text intelligence to be evaluated in different intelligence formats and obtain intelligence feature data. Then, the text intelligence to be evaluated can be evaluated according to the obtained intelligence feature data, so that the text intelligence to be evaluated in different intelligence formats can be evaluated, and the effect of automatically evaluating human-readable intelligence can be achieved.

[0006] In combination with the technical solution of the above first aspect, in some possible implementation manners, the evaluation index system includes at least one of a uniqueness evaluation index, an objectivity evaluation index, a relevance evaluation index, an intelligence type richness evaluation index, an intrusion index type richness evaluation index, and a relationship richness evaluation index.

[0007] In the embodiments of the present application, the multiple evaluation indexes included in the evaluation index system can realize multi-angle evaluation of the text information to be evaluated, so that the evaluation result obtained finally can more accurately reflect the value of the text information to be evaluated.

[0008] In combination with the technical scheme provided in the first aspect, in some possible implementation manners, the evaluation index system includes multiple evaluation indexes, and the obtaining of the evaluation result of the text information to be evaluated based on the information feature data and the preset evaluation index system includes: determining all target evaluation indexes from the preset evaluation index system based on the data types included in the information feature data, wherein the target evaluation index is an evaluation index that can be evaluated based on the data included in the information feature data; for each target evaluation index, obtaining a score of the target evaluation index based on the specified type data in the information feature data corresponding to the target evaluation index and the target evaluation index; and obtaining the evaluation result of the text information to be evaluated based on the scores of each target evaluation index.

[0009] In the embodiments of the present application, different target evaluation indexes are determined from the evaluation index system according to different data types included in the obtained information feature data. Thus, the present scheme can more flexibly evaluate the text information to be evaluated, and improve the application range of the present scheme.

[0010] In combination with the technical scheme provided in the first aspect, in some possible implementation manners, the information feature data includes at least one type of data from among a summary of the text information to be evaluated, threat content related data, and intrusion index information, wherein the threat content related data is other feature data in the text information to be evaluated except for the summary, the intrusion index information.

[0011] In the embodiments of the present application, the summary of the text information to be evaluated, the threat content related data, and the intrusion index information are respectively extracted from different angles, so that the extracted information feature data can reflect the characteristics of the text information to be evaluated from different angles, and improve the accuracy of the subsequent evaluation result.

[0012] In some possible implementation manners, the target evaluation index is the objectivity evaluation index, the objectivity evaluation index comprises a pre-trained second large language model and a corresponding relationship between an output result and an objectivity score, the specified type data is a summary of the text intelligence to be evaluated, and obtaining the score of the target evaluation index based on the specified type data in the intelligence feature data corresponding to the target evaluation index and the target evaluation index comprises: inputting the summary and an instruction for judging whether the summary is objective into the pre-trained second large language model to obtain an output result representing whether the summary is objective; and obtaining an objectivity score of the text intelligence to be evaluated based on the output result and the corresponding relationship between the output result and the objectivity score.

[0013] In the embodiments of the present application, the large language model can understand the meaning of the input text, and the summary of the text intelligence to be evaluated is a condensation of the text intelligence to be evaluated and can represent the content such as the argument and viewpoint of the text intelligence to be evaluated. Therefore, whether the summary of the text intelligence to be evaluated is objective can be judged by the second large language model, that is, whether the text intelligence to be evaluated is objective can be evaluated, so that the objectivity score of the text intelligence to be evaluated can be more accurate.

[0014] In some possible implementation manners, the target evaluation index is the intelligence type richness evaluation index, the intelligence type richness evaluation index comprises a corresponding relationship between a category number and an intelligence type richness score, the specified type data is the threat content related data, and obtaining the score of the target evaluation index based on the specified type data in the intelligence feature data corresponding to the target evaluation index and the target evaluation index comprises: counting the categories of the data included in the threat content related data, and obtaining an intelligence type richness score of the text intelligence to be evaluated based on the counted category number and the corresponding relationship between the category number and the intelligence type richness score.

[0015] In the embodiments of the present application, the more the categories of the data included in the threat content related data, the better the intelligence type richness of the text intelligence to be evaluated, so that the intelligence type richness score of the text intelligence to be evaluated can be obtained based on the categories of the data included in the threat content related data and the corresponding relationship between the category number and the intelligence type richness score.

[0016] In some possible implementation manners, the target evaluation index is the richness evaluation index of the intrusion indicator type, the richness evaluation index of the intrusion indicator type comprises a corresponding relationship between a number of intrusion indicator types and a richness score of the intrusion indicator type, the specified type data is the intrusion indicator information, and the obtaining of the score of the target evaluation index based on the specified type data in the intelligence feature data corresponding to the target evaluation index and the target evaluation index comprises: counting the intrusion indicator types included in the intrusion indicator information, and obtaining the richness score of the intrusion indicator type of the text intelligence to be evaluated based on the number of the intrusion indicator types obtained by counting and the corresponding relationship between the number of the intrusion indicator types and the richness score of the intrusion indicator type.

[0017] In the embodiments of the present application, the higher the richness of the intrusion indicator type in the text intelligence to be evaluated is, the more comprehensive the intelligence of the text to be evaluated on the intrusion indicator is. Therefore, the richness score of the intrusion indicator type of the text intelligence to be evaluated can be obtained more accurately by counting the intrusion indicator types included in the intrusion indicator information.

[0018] In some possible implementation manners, the target evaluation index is the relationship richness evaluation index, the specified type data comprises at least one of the intrusion indicator information and threat content related data, and the obtaining of the score of the target evaluation index based on the specified type data in the intelligence feature data corresponding to the target evaluation index and the target evaluation index comprises: determining whether there is an association relationship between any two data in the intelligence feature data; and obtaining the relationship richness score of the text intelligence to be evaluated based on the number of data pairs having the association relationship in the intelligence feature data.

[0019] In some possible implementation manners, the evaluation index system further comprises an intelligence source credibility evaluation index and an intelligence source intelligence update frequency evaluation index, and the method further comprises: obtaining identity information of a target data source corresponding to the text intelligence to be evaluated and an intelligence update frequency of the target data source; and correspondingly, the obtaining of the evaluation result of the text intelligence to be evaluated based on the intelligence feature data and the preset evaluation index system comprises: obtaining the evaluation result of the text intelligence to be evaluated based on the intelligence feature data, the identity information of the target data source, the intelligence update frequency of the target data source, and the evaluation index system.

[0020] In the embodiments of the present application, the information source credibility evaluation index and the information source information update frequency evaluation index are indexes for evaluating the to-be-evaluated text information, so that the to-be-evaluated text information can be more comprehensively evaluated, and the accuracy of the final evaluation result is further improved.

[0021] In a second aspect, the present application provides a model training method, comprising: obtaining a training data set, the training data set comprising a plurality of text information; training an initial large language model based on the training data set to obtain a trained first large language model.

[0022] In a third aspect, the present application provides an information value evaluation device, comprising: an acquisition module and a processing module, the acquisition module being configured to acquire to-be-evaluated text information; the processing module being configured to process the to-be-evaluated text information based on a pre-trained first large language model to obtain information feature data; and the processing module being further configured to obtain an evaluation result of the to-be-evaluated text information based on the information feature data and a preset evaluation index system.

[0023] In a fourth aspect, the present application provides an electronic device, comprising: a memory and a processor, the memory and the processor being connected; the memory being configured to store a program; and the processor being configured to call the program stored in the memory to execute the method of the first aspect and / or any possible implementation manner in combination with the first aspect, and / or execute the method of the second aspect.

[0024] In a fifth aspect, the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a computer to execute the method of the first aspect and / or any possible implementation manner in combination with the first aspect, and / or execute the method of the second aspect. BRIEF DESCRIPTION OF DRAWINGS

[0025] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed in the embodiments. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can also be obtained without creative labor.

[0026] Figure 1 A flowchart of an information value evaluation method according to an embodiment of the present application is shown.

[0027] Figure 2 A flowchart of a model training method according to an embodiment of the present application is shown.

[0028] Figure 3A flow diagram illustrating a specific implementation of an S300 according to an embodiment of the present application is shown.

[0029] Figure 4 A structural block diagram of an information value evaluation device according to an embodiment of the present application is shown.

[0030] Figure 5 A structural block diagram of a model training device according to an embodiment of the present application is shown.

[0031] Figure 6 A structural block diagram of an electronic device according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0032] The technical solutions in the embodiments of the present application will be described below with reference to the drawings.

[0033] It should be noted that similar reference numerals and letters refer to similar items in the following drawings, and therefore, once an item is defined in one drawing, it need not be further defined and explained in subsequent drawings. Meanwhile, in the description of the present application, relationship terms such as "first", "second", etc. are merely used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between the entities or operations. Furthermore, the term "comprise", "include" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed or inherent to such process, method, article or device. Without more limitations, the elements defined by the phrase "comprise" do not exclude the presence of other identical elements in the process, method, article or device comprising the elements.

[0034] The technical solutions of the present application will be described in detail below with reference to the drawings.

[0035] Please refer to Figure 1 , Figure 1 A flow diagram of an information value evaluation method according to an embodiment of the present application is shown. The steps contained therein will be described below. Figure 1

[0036] S100: Obtain information to be evaluated.

[0037] The information to be evaluated can be obtained in advance and stored in a storage medium, and directly called when needed for use; or the information to be evaluated can also be obtained in real time when needed for use.

[0038] ​The to-be-evaluated text intelligence can be any type of intelligence, including but not limited to machine-readable intelligence and human-readable intelligence. The specific type of to-be-evaluated text intelligence is not limited herein.

[0039] S200: processing the to-be-evaluated text intelligence based on the pre-trained first large language model to obtain intelligence feature data.

[0040] The large language model, also referred to as LLM, is a large language model. The specific results and implementation methods of the large language model are well known to those skilled in the art. For brevity, the details are not described herein.

[0041] In one implementation, the intelligence feature data can include at least one type of data from among a summary of the to-be-evaluated text intelligence, threat content related data, and intrusion indicator information.

[0042] The specific types of data to be obtained can be set according to actual needs.

[0043] The summary of the to-be-evaluated text intelligence is the condensed text content of the to-be-evaluated text intelligence.

[0044] The threat content related data is other feature data of the to-be-evaluated text intelligence other than the summary and the intrusion indicator information.

[0045] For example, the threat content related data can include information such as the target industry, the target country, the vulnerability, and the TTP (Time Triggered Protocol) in the to-be-evaluated text intelligence.

[0046] The intrusion indicator (IOC, Indicator Of Compromise) information is an indicator for detecting and identifying network attacks, which can help users to discover and respond to network security threats in a timely manner. For example, it can include IP (Internet Protocol), Domain, URL (Uniform Resource Locator), and Hash (Hash value) information.

[0047] In one implementation, the specific implementation method of processing the to-be-evaluated text intelligence based on the pre-trained first large language model to obtain intelligence feature data can be: taking the to-be-evaluated text intelligence and an operation instruction as inputs of the first large language model, and the result output by the first large language model is the intelligence feature data.

[0048] The operation instruction is a text content input by a user, and the operation instruction is used to instruct the first large language model to perform different functions.

[0049] For ease of understanding, taking obtaining a summary of the to-be-evaluated text information as an example. At this time, the to-be-evaluated text information and the operation instruction for extracting the summary can be input into the trained first large language model, and the first large language model can output the summary of the to-be-evaluated text information.

[0050] The operation instruction herein can be text data such as "generate a summary", "generate a summary of the input text", and the like. The examples herein are for ease of understanding only and should not be construed as limiting the present application.

[0051] Similarly, if the threat content related data of the to-be-evaluated text information is to be obtained, the operation instruction can be text data such as "extract the feature content of the to-be-evaluated text information", "extract the content related to other entities in the to-be-evaluated text information", and the like. The examples herein are for ease of understanding only and should not be construed as limiting the present application.

[0052] If the intrusion indicator information of the to-be-evaluated text information is to be obtained, the operation instruction can be text data such as "extract the intrusion indicator information of the to-be-evaluated text information", "extract the IOC information of the to-be-evaluated text information", and the like. The examples herein are for ease of understanding only and should not be construed as limiting the present application.

[0053] Optionally, if the obtained text feature data is required to include multiple types of data such as the summary, the threat content related data, and the intrusion indicator information of the to-be-evaluated text information, the operation instruction input into the first large language model can represent the obtaining of multiple types of data.

[0054] For example, if the obtained text feature data is required to include three types of data such as the summary, the threat content related data, and the intrusion indicator information of the to-be-evaluated text information, the operation instruction can be "generate a summary, and extract the feature content and the IOC information of the to-be-evaluated text information". The examples herein are for ease of understanding only and should not be construed as limiting the present application.

[0055] In an implementation, before S200, the information value evaluation method further includes S210-S220, as shown in Figure 2 .

[0056] S210: Obtain a training data set, and the training data set includes a plurality of text information.

[0057] The training data set described above can include multiple types of information, such as security bulletins, vulnerability warnings, virus / APT analysis, and the like. The types of text information in the training data set are not limited herein.

[0058] S220: Train an initial large language model based on the training data set to obtain a trained first large language model.

[0059] The specific manner and principle of training a large language model are well known to those skilled in the art, and will not be described here for brevity.

[0060] S300: Obtain an evaluation result of the to-be-evaluated text intelligence based on the intelligence feature data and the preset evaluation index system.

[0061] In one implementation, the evaluation index system includes at least one of the uniqueness evaluation index, the objectivity evaluation index, the relevance evaluation index, the intelligence type richness evaluation index, the intrusion index type richness evaluation index, and the relationship richness evaluation index.

[0062] In one implementation, if the evaluation index system includes multiple evaluation indexes, the specific manner of obtaining the evaluation result of the to-be-evaluated text intelligence based on the intelligence feature data and the preset evaluation index system can include S310, S320, and S330, as shown in Figure 3

[0063] S310: Determine all target evaluation indexes from the preset evaluation index system based on the data types included in the intelligence feature data.

[0064] The target evaluation index is an evaluation index that can be evaluated based on the data included in the intelligence feature data.

[0065] For example, when the preset evaluation index system includes the uniqueness evaluation index, the objectivity evaluation index, the relevance evaluation index, the intelligence type richness evaluation index, the intrusion index type richness evaluation index, and the relationship richness evaluation index, if the intelligence feature data includes a summary, the target evaluation index can include the uniqueness evaluation index and the objectivity evaluation index. If the intelligence feature data includes threat content related data, the target evaluation index can include the relevance evaluation index, the intelligence type richness evaluation index, and the relationship richness evaluation index. If the intelligence feature data includes intrusion index information, the target evaluation index can include the intrusion index type richness evaluation index and the relationship richness evaluation index.

[0066] S320: Obtain a score of each target evaluation index based on the specified type of data in the intelligence feature data corresponding to the target evaluation index and the target evaluation index.

[0067] ​In one implementation, the target evaluation index is a uniqueness evaluation index, the uniqueness evaluation index comprises a correspondence between similarity and uniqueness score, the specified type data is a summary of the text information to be evaluated, and the specific process of obtaining the score of the target evaluation index based on the specified type data in the information feature data corresponding to the target evaluation index and the target evaluation index can be: obtaining summaries of other text information, calculating the similarity between the summary of the text information to be evaluated and each obtained summary of other information; and finally obtaining the uniqueness score of the text information to be evaluated based on the similarity and the correspondence between the similarity and the uniqueness score.

[0068] The way of calculating the similarity between two texts is well known to those skilled in the art, and will not be described here for brevity.

[0069] The correspondence between the similarity and the uniqueness score can be in the form of a correspondence table, for example, and the specific type of the correspondence between the output result and the uniqueness score is not limited here.

[0070] The correspondence between the output result and the uniqueness score can comprise a plurality of similarity intervals respectively corresponding to a uniqueness score. The specific values of the uniqueness scores corresponding to different similarity intervals can be set according to actual needs, and are not limited here.

[0071] For example, the uniqueness score corresponding to the similarity interval 90%-100% can be set to 5; the uniqueness score corresponding to the similarity interval 80%-90% can be set to 4; the uniqueness score corresponding to the similarity interval 60%-80% can be set to 3; the uniqueness score corresponding to the similarity interval 40%-60% can be set to 2; the uniqueness score corresponding to the similarity interval 20%-40% can be set to 1; and the uniqueness score corresponding to the similarity interval 0-20% can be set to 0. The examples here are only for ease of understanding and should not be considered as a limitation on the present application.

[0072] In one implementation, the target evaluation index is an objectivity evaluation index, the objectivity evaluation index comprises a pre-trained second large language model and a correspondence between the output result and the objectivity score, the specified type data is a summary of the text information to be evaluated, and the specific process of obtaining the score of the target evaluation index based on the specified type data in the information feature data corresponding to the target evaluation index and the target evaluation index can be: inputting the summary and an instruction representing whether the summary is objective into the pre-trained second large language model to obtain an output result representing whether the summary is objective; and obtaining the objectivity score of the text information to be evaluated based on the output result and the correspondence between the output result and the objectivity score.

[0073] The second large language model can be the same as the first large language model, or the second large language model can be another large language model independent of the first large language model.

[0074] The training method and training data set of the second large language model are the same as those of the first large language model, and will not be described again here for brevity.

[0075] The instruction for determining whether the summary is objective can be, for example, "determine whether the summary is objective", and the example is only for ease of understanding and should not be considered as a limitation of the present application.

[0076] The correspondence between the output result and the objectivity score can be, for example, a correspondence table, and the specific type of the correspondence between the output result and the objectivity score is not limited here.

[0077] The correspondence between the output result and the objectivity score can include an objectivity score corresponding to an output result representing an objective summary, and an objectivity score corresponding to an output result representing a non-objective summary.

[0078] The specific values of the objectivity score corresponding to the output result representing the objective summary and the objectivity score corresponding to the output result representing the non-objective summary can be set according to actual conditions. For example, the specific value of the objectivity score corresponding to the output result representing the objective summary can be set to 1, and the specific value of the objectivity score corresponding to the output result representing the non-objective summary can be set to 0. The example is only for ease of understanding and should not be considered as a limitation of the present application.

[0079] In an implementation, the target evaluation index is a relevance evaluation index, the relevance evaluation index includes intelligence information of an evaluation object itself, the specified type data is threat content related data, and the specific process of obtaining a score of the target evaluation index based on the specified type data in the intelligence feature data corresponding to the target evaluation index and the target evaluation index can be: obtaining a relevance score representing the relevance of the intelligence of the text to be evaluated and the evaluation object based on the threat content related data and the preset intelligence information of the evaluation object itself.

[0080] The intelligence information of the evaluation object itself is the object to be evaluated.

[0081] For example, the evaluation object can be a company, an enterprise, an industry, a country, etc.

[0082] The content of the intelligence information of the evaluation object itself is different according to the different evaluation objects, but the more complete the intelligence information of the evaluation object itself is, the more accurate the final relevance score is.

[0083] Optionally, the specific process of obtaining the relevance score representing the relevance between the to-be-evaluated text intelligence and the evaluation object based on the threat content related data and the preset intelligence information of the evaluation object itself can be: extracting keywords in the threat content related data to obtain a first keyword set; and extracting keywords in the intelligence information of the evaluation object itself to obtain a second keyword set. Counting the number of keywords in the intersection of the first keyword set and the second keyword set, and obtaining the relevance score according to the number.

[0084] The number of keywords in the intersection of the first keyword set and the second keyword set can be obtained in the following manner: obtaining the relevance score of the to-be-evaluated intelligence according to the number and the corresponding relationship between the number of keywords and the relevance score. The corresponding relationship between the number of keywords and the relevance score is preset in the relevance evaluation index.

[0085] The corresponding relationship between the number of keywords and the relevance score can be in the form of a corresponding relationship table, for example, and the specific type of the corresponding relationship between the number of keywords and the relevance score is not limited here.

[0086] The corresponding relationship between the number of keywords and the relevance score can include a plurality of keyword number intervals (each interval can include only one keyword) corresponding to a relevance score. The specific values of the relevance scores corresponding to different keyword number intervals can be set according to actual needs, and are not limited here.

[0087] For example, the relevance score corresponding to a keyword number less than 3 can be set to 0, the relevance score corresponding to a keyword number interval of 3-5 can be set to 1, the relevance score corresponding to a keyword number interval of 6-10 can be set to 2, and the relevance score corresponding to a keyword number greater than 10 can be set to 3. The examples here are only for ease of understanding and should not be considered as a limitation of the present application.

[0088] Optionally, the specific process of obtaining the relevance score representing the relevance between the to-be-evaluated text intelligence and the evaluation object based on the threat content related data and the intelligence information of the evaluation object itself can also be: extracting keywords of the threat content related data to obtain a first keyword set; counting the frequency of each keyword in the first keyword set appearing in the intelligence information of the evaluation object itself to obtain the total frequency of all keywords in the first keyword set appearing in the intelligence information of the evaluation object itself. According to the total frequency and the corresponding relationship between the total frequency and the relevance score, the relevance score of the to-be-evaluated text intelligence is obtained.

[0089] The corresponding relationship between the total frequency and the relevance score can be in the form of a corresponding relationship table, for example, and the specific type of the corresponding relationship between the total frequency and the relevance score is not limited here.

[0090] The correspondence between the total frequency and the correlation score can include a plurality of total frequency intervals respectively corresponding to correlation scores. The specific values of the correlation scores corresponding to different total frequency intervals can be set according to actual needs, and this place does not limit them.

[0091] For example, the correlation score corresponding to the total frequency less than 10 can be set to 0, the correlation score corresponding to the total frequency interval 11-20 can be set to 1, the correlation score corresponding to the total frequency interval 21-40 can be set to 2, and the correlation score corresponding to the total frequency greater than 40 can be set to 3. The examples here are only for easy understanding and should not be regarded as a limitation of the present application.

[0092] In an implementation, the target evaluation index is an intelligence type richness evaluation index, the intelligence type richness evaluation index includes a correspondence between the category number and an intelligence type richness score, the specified type data is threat content related data, and the specific implementation of obtaining the score of the target evaluation index based on the target evaluation index and the specified type data in the intelligence feature data corresponding to the target evaluation index can be: counting the categories of data included in the threat content related data, and obtaining the intelligence type richness score of the text intelligence to be evaluated based on the counted category number and the correspondence between the category number and the intelligence type richness score.

[0093] The categories of data included in the threat content related data are also referred to as data types, such as attack industry, target country, vulnerability, TTP, Malware, and tool. The categories of data included in different text intelligence to be evaluated are different, and the examples here are only for easy understanding and should not be regarded as a limitation of the present application.

[0094] The correspondence between the category number and the intelligence type richness score can be in the form of a correspondence table, and this place does not limit the specific type of the correspondence between the category number and the intelligence type richness score.

[0095] The correspondence between the category number and the intelligence type richness score can include a plurality of category number intervals (an interval can include only one number, that is, one number corresponds to one intelligence type richness score one-to-one) respectively corresponding to intelligence type richness scores. The specific values of the intelligence type richness scores corresponding to different category number intervals can be set according to actual needs, and this place does not limit them.

[0096] For example, the intelligence type richness corresponding to the category number less than 3 can be set to 0, the intelligence type richness corresponding to the category number interval 3-5 can be set to 1, the intelligence type richness corresponding to the category number interval 6-10 can be set to 2, and the intelligence type richness corresponding to the category number greater than 10 can be set to 3. The examples here are only for easy understanding and should not be regarded as a limitation of the present application.

[0097] In an implementation, the target evaluation index is an intrusion indicator type richness evaluation index, the intrusion indicator type richness evaluation index comprises a corresponding relationship between the number of intrusion indicator types and an intrusion indicator type richness score, the specified type data is intrusion indicator information, and the specific manner of obtaining the score of the target evaluation index based on the specified type data in the intelligence feature data corresponding to the target evaluation index and the target evaluation index can be: counting the intrusion indicator types included in the intrusion indicator information, obtaining the intrusion indicator type richness score of the intelligence of the to-be-evaluated text based on the number of intrusion indicator types obtained by counting and the corresponding relationship between the number of intrusion indicator types and the intrusion indicator type richness score.

[0098] The intrusion indicator types included in the intrusion indicator information are types of indicators used to detect and identify network attacks, such as IP, Domain, URL, Hash, and the like. The intrusion indicator types include but are not limited to the types exemplified herein.

[0099] The corresponding relationship between the number of intrusion indicator types and the intrusion indicator type richness score can be in the form of a corresponding relationship table, and the specific type of the corresponding relationship between the number of intrusion indicator types and the intrusion indicator type richness score is not limited herein.

[0100] The corresponding relationship between the number of intrusion indicator types and the intrusion indicator type richness score can comprise a plurality of intrusion indicator type number intervals (each interval can only include one number, that is, one number corresponds to one intrusion indicator type richness score in a one-to-one manner) respectively corresponding to intrusion indicator type richness scores. The specific values of the intrusion indicator type richness scores corresponding to different intrusion indicator type number intervals can be set according to actual needs, and are not limited herein.

[0101] For example, the intrusion indicator type richness corresponding to the number of intrusion indicator types less than 3 can be set to 0, the intrusion indicator type richness corresponding to the number of intrusion indicator types in the interval 3-5 can be set to 1, the intrusion indicator type richness corresponding to the number of intrusion indicator types in the interval 6-10 can be set to 2, and the intrusion indicator type richness corresponding to the number of intrusion indicator types greater than 10 can be set to 3. The examples herein are only for ease of understanding and should not be considered as a limitation on the present application.

[0102] In an implementation form, the target evaluation index is a relationship richness evaluation index, the specified type of data includes at least one of intrusion indicator information and threat content related data, and the specific implementation form of obtaining the score of the target evaluation index based on the specified type of data in the intelligence feature data corresponding to the target evaluation index and the target evaluation index can be: determining whether there is an association relationship between any two data in the intelligence feature data; and obtaining the relationship richness score of the text intelligence to be evaluated based on the number of data pairs in the intelligence feature data that have an association relationship.

[0103] Optionally, the specific implementation form of determining whether there is an association relationship between any two data in the intelligence feature data can be: extracting the keywords of the two data respectively to obtain a third keyword set and a fourth keyword set, and then determining the relationship richness score of the text intelligence to be evaluated based on the number of keywords existing in the intersection of the third keyword set and the fourth keyword set.

[0104] The specific implementation form of determining the relationship richness score of the text intelligence to be evaluated based on the number of keywords existing in the intersection of the third keyword set and the fourth keyword set is the same as the specific implementation form of determining the relevance score of the text intelligence to be evaluated based on the number of keywords existing in the intersection of the first keyword set and the second keyword set, and will not be described here for brevity.

[0105] Alternatively, the specific implementation form of determining whether there is an association relationship between any two data in the intelligence feature data can also be: inputting the two data and the operation instruction representing whether the input data has an association relationship into the first large language model to obtain the output result representing whether the input data has an association relationship.

[0106] After obtaining the number of data pairs in the intelligence feature data that have an association relationship, the implementation form of obtaining the relationship richness score of the text intelligence to be evaluated based on the number of data pairs in the intelligence feature data that have an association relationship can be: multiplying the number of data pairs in the intelligence feature data that have an association relationship by a relationship richness parameter to obtain the relationship richness score.

[0107] The relationship richness evaluation index includes the relationship richness parameter, and the specific value of the relationship richness parameter can be set according to actual conditions, and the specific value is not limited here.

[0108] Alternatively, the implementation manner of obtaining the relationship richness score of the information of the to-be-evaluated text based on the number of data pairs having the correlation relationship in the information feature data can also be: obtaining the relationship richness score based on the number of data pairs having the correlation relationship in the information feature data and the corresponding relationship between the data pair and the relationship richness score.

[0109] The corresponding relationship between the number of data pairs having the correlation relationship and the relationship richness score can be in the form of a corresponding relationship table, for example, and the specific type of the corresponding relationship between the number of data pairs having the correlation relationship and the relationship richness score is not limited here.

[0110] The corresponding relationship between the number of data pairs having the correlation relationship and the relationship richness score can include the relationship richness scores respectively corresponding to a plurality of number intervals of data pairs having the correlation relationship (one interval can only include one number, that is, one number corresponds to one relationship richness score - one-to-one correspondence). The specific values of the relationship richness scores respectively corresponding to different number intervals of data pairs having the correlation relationship can be set according to actual needs, and are not limited here.

[0111] For example, the relationship richness corresponding to the number of data pairs having the correlation relationship less than 3 can be 0, the relationship richness corresponding to the number interval of data pairs having the correlation relationship of 3-5 can be 1, the relationship richness corresponding to the number interval of data pairs having the correlation relationship of 6-10 can be 2, and the relationship richness corresponding to the number of data pairs having the correlation relationship greater than 10 can be 3. The examples here are only for easy understanding and should not be considered as a limitation of the present application.

[0112] It can be understood that in the implementation manner of determining whether any two data in the information feature data have a correlation relationship by using the first large language model, the operation instruction can also be an operation instruction representing the type of the correlation relationship between the input data. At this time, the output result of the first large language model represents the type of the relationship between the two input data. If there is no correlation relationship between the two data, the output result of the first large language model represents that there is no correlation relationship.

[0113] Therefore, according to the output result of the first large language model for each pair of data, the number of data pairs having the correlation relationship in the information feature data and the number of types of correlation relationships can be obtained.

[0114] Correspondingly, the implementation manner of obtaining the relationship richness score of the information of the to-be-evaluated text based on the number of data pairs having the correlation relationship in the information feature data is: obtaining the relationship richness score of the information of the to-be-evaluated text based on the number of data pairs having the correlation relationship in the information feature data and the number of types of correlation relationships.

[0115] At this time, a sum of a product of a number of data pairs having a correlation relationship in the intelligence feature data multiplied by the first relationship richness parameter and a product of the number of correlation relationships multiplied by the second relationship richness parameter can be taken as the relationship richness score.

[0116] The relationship richness evaluation index includes the first relationship richness parameter and the second relationship richness parameter, and specific values of the first relationship richness parameter and the second relationship richness parameter can be set according to actual conditions, which are not limited herein.

[0117] S330: Obtain an evaluation result of the text intelligence to be evaluated based on the obtained scores of each target evaluation index.

[0118] In one implementation, after obtaining the scores of each target evaluation index, a sum of the scores of all target evaluation indexes can be taken as the evaluation result of the text intelligence to be evaluated. Alternatively, an average of the scores of all target evaluation indexes can also be taken as the evaluation result of the text intelligence to be evaluated.

[0119] Optionally, a weight value corresponding to each evaluation index in the evaluation index system can also be set in advance, and after obtaining the scores of each target evaluation index, a product of the score of each target evaluation index and the weight value corresponding to the target evaluation index is calculated, and then a sum or an average of all products is calculated as the evaluation result of the text intelligence to be evaluated.

[0120] The weight value corresponding to each evaluation index can be set according to actual needs, which is not limited herein.

[0121] In one implementation, the evaluation index system further includes an intelligence source credibility evaluation index and an intelligence source intelligence update frequency evaluation index. The intelligence evaluation method further includes: obtaining identity information of a target data source corresponding to the text intelligence to be evaluated and an intelligence update frequency of the target data source.

[0122] Correspondingly, the specific manner of obtaining the evaluation result of the text intelligence to be evaluated based on the intelligence feature data and the preset evaluation index system can be: obtaining the evaluation result of the text intelligence to be evaluated based on the intelligence feature data, the identity information of the target data source, the intelligence update frequency of the target data source, and the evaluation index system.

[0123] Optionally, the target evaluation index is an intelligence source credibility evaluation index, the intelligence source credibility evaluation index comprises a corresponding relationship between intelligence sources and intelligence source credibility scores, the specified type of data is identity information of the target data source, and a specific implementation manner of obtaining the score of the target evaluation index based on the specified type of data in the intelligence feature data corresponding to the target evaluation index and the target evaluation index can be as follows: obtaining the intelligence source credibility score based on the identity information of the target data source and the corresponding relationship between intelligence sources and intelligence source credibility scores.

[0124] The identity information of the target data source can be any information that can uniquely identify the target data source, such as ID, IP, and the like.

[0125] The corresponding relationship between intelligence sources and intelligence source credibility scores can comprise intelligence source credibility scores corresponding to the identity information of each of the plurality of intelligence sources. The intelligence source credibility score corresponding to the identity information of each intelligence source can be set according to actual needs, which is not limited herein.

[0126] Optionally, the target evaluation index is an intelligence source intelligence update frequency evaluation index, the intelligence source intelligence update frequency evaluation index comprises a corresponding relationship between update frequencies and intelligence source intelligence update frequency scores, the specified type of data is an intelligence update frequency of the target data source, and a specific implementation manner of obtaining the score of the target evaluation index based on the specified type of data in the intelligence feature data corresponding to the target evaluation index and the target evaluation index can be as follows: obtaining the intelligence source intelligence update frequency score based on the intelligence update frequency of the target data source and the corresponding relationship between update frequencies and intelligence source intelligence update frequency scores.

[0127] The corresponding relationship between update frequencies and intelligence source update frequency scores can be in a form such as a corresponding relationship table, and the specific type of the corresponding relationship between update frequencies and intelligence source update frequency scores is not limited herein.

[0128] The corresponding relationship between update frequencies and intelligence source update frequency scores can comprise intelligence source update frequency scores corresponding to a plurality of update frequency intervals. The specific values of the intelligence source update frequency scores corresponding to different update frequency intervals can be set according to actual needs, which are not limited herein.

[0129] For example, the information source update frequency score can be set as 0 when the information source update frequency is less than 1 time per month, 1 when the information source update frequency is between 1 time per month and 3 times per month, 2 when the information source update frequency is between 4 times per month and 10 times per month, and 3 when the information source update frequency is more than 10 times per month. The above examples are only for better understanding and should not be considered as a limitation to the present application.

[0130] Based on the same technical concept, the present application also provides an information value evaluation device, as shown in the figure, which comprises an acquisition module 110 and a processing module 120. Figure 4 The acquisition module 110 is configured to acquire information to be evaluated.

[0131] The acquisition module 110 is configured to acquire information to be evaluated.

[0132] The processing module 120 is configured to process the information to be evaluated based on a pre-trained first large language model to obtain information feature data.

[0133] The processing module 120 is further configured to obtain an evaluation result of the information to be evaluated based on the information feature data and a preset evaluation index system.

[0134] In an embodiment, the evaluation index system comprises at least one of a uniqueness evaluation index, an objectivity evaluation index, a relevance evaluation index, an information type richness evaluation index, an intrusion index type richness evaluation index, and a relationship richness evaluation index.

[0135] The evaluation index system comprises a plurality of evaluation indexes, and the processing module 120 is specifically configured to determine all target evaluation indexes from the preset evaluation index system based on the data types included in the information feature data, wherein the target evaluation index is an evaluation index that can be evaluated based on the data included in the information feature data; for each target evaluation index, a score of the target evaluation index is obtained based on the designated type data in the information feature data corresponding to the target evaluation index and the target evaluation index; and an evaluation result of the information to be evaluated is obtained based on the scores of each of the target evaluation indexes.

[0136] In an embodiment, the information feature data comprises at least one type of data selected from the group consisting of a summary of the information to be evaluated, threat content related data, and intrusion index information.

[0137] The target evaluation index is the objectivity evaluation index, the objectivity evaluation index includes a pre-trained second large language model and a corresponding relationship between an output result and an objectivity score, the specified type data is a summary of the text intelligence to be evaluated, and the processing module 120 is specifically configured to input the summary and an instruction indicating whether the summary is objective into the pre-trained second large language model to obtain an output result indicating whether the summary is objective; based on the output result and the corresponding relationship between the output result and the objectivity score, an objectivity score of the text intelligence to be evaluated is obtained.

[0138] The target evaluation index is the intelligence type richness evaluation index, the intelligence type richness evaluation index includes a corresponding relationship between a category number and an intelligence type richness score, the specified type data is the threat content related data, and the processing module 120 is specifically configured to count categories of data included in the threat content related data, and based on the counted category number and the corresponding relationship between the category number and the intelligence type richness score, an intelligence type richness score of the text intelligence to be evaluated is obtained.

[0139] The target evaluation index is the intrusion indicator type richness evaluation index, the intrusion indicator type richness evaluation index includes a corresponding relationship between an intrusion indicator type number and an intrusion indicator type richness score, the specified type data is the intrusion indicator information, and the processing module 120 is specifically configured to count intrusion indicator types included in the intrusion indicator information, and based on the counted intrusion indicator type number and the corresponding relationship between the intrusion indicator type number and the intrusion indicator type richness score, an intrusion indicator type richness score of the text intelligence to be evaluated is obtained.

[0140] The target evaluation index is the relationship richness evaluation index, the specified type data includes at least one of the intrusion indicator information and the threat content related data, and the processing module 120 is specifically configured to determine, for any two pieces of data in the intelligence feature data, whether there is an association relationship between the two pieces of data; based on a number of data pairs in the intelligence feature data that have an association relationship, a relationship richness score of the text intelligence to be evaluated is obtained.

[0141] The evaluation index system further includes an intelligence source credibility evaluation index and an intelligence source intelligence update frequency evaluation index; the acquisition module 110 is further configured to acquire identity information of a target data source corresponding to the text intelligence to be evaluated and an intelligence update frequency of the target data source. Correspondingly, the processing module 120 is specifically configured to obtain an evaluation result of the text intelligence to be evaluated based on the intelligence feature data, the identity information of the target data source, the intelligence update frequency of the target data source, and the evaluation index system.

[0142] The information value evaluation device 100 provided in the embodiments of the present application has the same implementation principle and technical effects as the foregoing information value evaluation method embodiments. For brief description, the part not mentioned in the device embodiment can refer to the corresponding content in the foregoing information value evaluation method embodiments.

[0143] Based on the same technical concept, the present application also provides a model training device, as shown in Figure 5 The model training device 200 includes an acquisition module 210 and a training module 2202.

[0144] The acquisition module 210 is configured to acquire a training data set, wherein the training data set includes a plurality of text information.

[0145] The training module 220 is configured to train an initial large language model based on the training data set to obtain a trained first large language model.

[0146] The model training device 200 provided in the embodiments of the present application has the same implementation principle and technical effects as the foregoing information value evaluation method and model training embodiments. For brief description, the part not mentioned in the device embodiment can refer to the corresponding content in the foregoing information value evaluation method embodiments.

[0147] Please refer to Figure 6 , which is an electronic device 300 provided in the embodiments of the present application. The electronic device 300 includes a processor 310 and a memory 320.

[0148] The memory 320 and the processor 310 are directly or indirectly electrically connected to each other to realize data transmission or interaction. For example, these elements can be electrically connected to each other through one or more communication buses or signal lines. Among them, the memory 320 is used to store computer programs, such as software function modules shown in Figure 4 or Figure 5 , that is, the information value evaluation device 100 or the model training device 200.

[0149] In the information value evaluation device 100, at least one software function module in the form of software or firmware is stored in the memory 320 or is solidified in the operating system (OS) of the electronic device 300. The processor 310 is configured to execute the executable modules stored in the memory 320, such as the software function modules or computer programs included in the information value evaluation device 100. At this time, the processor 310 is configured to obtain text information to be evaluated; process the text information to be evaluated based on a pre-trained first large language model to obtain information feature data; and obtain an evaluation result of the text information to be evaluated based on the information feature data and a preset evaluation index system.

[0150] In the model training device 200, at least one software function module in the form of software or firmware is stored in the memory 320 or is solidified in the operating system (OS) of the electronic device 300. The processor 310 is configured to execute the executable modules stored in the memory 320, such as the software function modules or computer programs included in the model training device 200. At this time, the processor 310 is configured to obtain a training data set, the training data set including a plurality of text information; and train an initial large language model based on the training data set to obtain a trained first large language model.

[0151] The memory 320 can be, but is not limited to, a random access memory (RAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), and the like.

[0152] The processor 310 can be an integrated circuit chip having a processing capability for signals. The processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; or can be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed by the processor. The general-purpose processor can be a microprocessor or the processor 310 can also be any conventional processor.

[0153] The electronic device 300 described above includes, but is not limited to, a personal computer, a server, etc.

[0154] The embodiments of the present application also provide a non-volatile computer readable storage medium (hereinafter referred to as a storage medium) having a computer program stored thereon. When the computer program is run by a computer such as the electronic device 300 described above, the information value evaluation method and / or the model training method shown above are executed. The computer readable storage medium includes a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0155] The above only describes the preferred embodiments of the present application and is not intended to limit the present application. For those skilled in the art, the present application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. An information value evaluation method characterized by comprising: The method comprises: obtaining to-be-evaluated text information; processing the to-be-evaluated text information based on a pre-trained first large language model to obtain information feature data; obtaining an evaluation result of the to-be-evaluated text information based on the information feature data and a preset evaluation index system; wherein processing the to-be-evaluated text information based on the pre-trained first large language model to obtain information feature data comprises: inputting the to-be-evaluated text information and operation instructions into the first large language model for processing to obtain the information feature data, the operation instructions being used to instruct the first large language model to perform different functions to obtain different types of information feature data, wherein the operation instructions include at least one instruction representing generation of an abstract, extraction of feature content of the to-be-evaluated text information, and extraction of intrusion indicator information, and correspondingly, the information feature data includes at least one type of data from among an abstract of the to-be-evaluated text information, threat content related data, and intrusion indicator information, wherein the threat content related data is other feature data of the to-be-evaluated text information except for the abstract and the intrusion indicator information.

2. The method of claim 1, wherein, The evaluation index system includes at least one evaluation index from among uniqueness evaluation index, objectivity evaluation index, relevance evaluation index, information type richness evaluation index, intrusion indicator type richness evaluation index, and relationship richness evaluation index.

3. The method of claim 2, wherein, The evaluation index system includes multiple evaluation indexes, and obtaining an evaluation result of the to-be-evaluated text information based on the information feature data and a preset evaluation index system comprises: determining all target evaluation indexes from the preset evaluation index system based on the data types included in the information feature data, wherein the target evaluation indexes are evaluation indexes that can be evaluated based on the data included in the information feature data; for each target evaluation index, obtaining a score of the target evaluation index based on the specified type data in the information feature data corresponding to the target evaluation index and the target evaluation index; obtaining an evaluation result of the to-be-evaluated text information based on the scores of each target evaluation index.

4. The method of claim 3, wherein, The target evaluation index is the objectivity evaluation index, the objectivity evaluation index includes a pre-trained second large language model and a correspondence between output results and objectivity scores, the specified type data is an abstract of the to-be-evaluated text information, and obtaining a score of the target evaluation index based on the specified type data in the information feature data corresponding to the target evaluation index and the target evaluation index comprises: inputting the abstract and an instruction representing judgment of whether the abstract is objective into the pre-trained second large language model to obtain an output result representing whether the abstract is objective; obtaining an objectivity score of the to-be-evaluated text information based on the output result and the correspondence between output results and objectivity scores.

5. The method of claim 3, wherein, The target evaluation index is the information type richness evaluation index, the information type richness evaluation index includes a corresponding relationship between the category number and the information type richness score, the specified type data is the threat content related data, and the score of the target evaluation index is obtained based on the specified type data corresponding to the target evaluation index in the information feature data and the target evaluation index, including: The number of categories included in the threat content related data is counted, and the information type richness score of the text information to be evaluated is obtained based on the number of categories counted and the corresponding relationship between the number of categories and the information type richness score.

6. The method of claim 3, wherein, The target evaluation index is the intrusion index type richness evaluation index, the intrusion index type richness evaluation index includes a corresponding relationship between the number of intrusion index types and the intrusion index type richness score, the specified type data is the intrusion index information, and the score of the target evaluation index is obtained based on the specified type data corresponding to the target evaluation index in the information feature data and the target evaluation index, including: The number of intrusion index types included in the intrusion index information is counted, and the intrusion index type richness score of the text information to be evaluated is obtained based on the number of intrusion index types counted and the corresponding relationship between the number of intrusion index types and the intrusion index type richness score.

7. The method of claim 3, wherein, The target evaluation index is the relationship richness evaluation index, and the specified type data includes at least one of the intrusion index information and the threat content related data. The score of the target evaluation index is obtained based on the specified type data corresponding to the target evaluation index in the information feature data and the target evaluation index, including: For any two data in the information feature data, it is determined whether there is an association relationship between the two data; Based on the number of data pairs with an association relationship in the information feature data, the relationship richness score of the text information to be evaluated is obtained.

8. The method according to any one of claims 1 to 7, characterized in that, The evaluation index system further includes an information source credibility evaluation index and an information source information update frequency evaluation index. The method further includes: Obtaining identity information of a target data source corresponding to the text information to be evaluated and an information update frequency of the target data source; Correspondingly, based on the information feature data and the preset evaluation index system, an evaluation result of the text information to be evaluated is obtained, including: Based on the information feature data, the identity information of the target data source, the information update frequency of the target data source, and the evaluation index system, an evaluation result of the text information to be evaluated is obtained.

9. An information value evaluation device characterized by comprising: including: An acquisition module is configured to acquire text information to be evaluated; A processing module is configured to process the text information to be evaluated based on a pre-trained first large language model to obtain information feature data; The processing module is further configured to obtain an evaluation result of the text information to be evaluated based on the information feature data and a preset evaluation index system; The processing module is further configured to obtain an evaluation result of the text information to be evaluated based on the information feature data and a preset evaluation index system; The processing module is further configured to obtain an evaluation result of the text information to be evaluated based on the information feature data and a preset evaluation index system; The to-be-evaluated text information and operation instructions are input into the first large language model for processing to obtain the information feature data. The operation instructions are used to instruct the first large language model to perform different functions to obtain different types of information feature data. The operation instructions include at least one instruction of generating an abstract, extracting feature content of the to-be-evaluated text information, and extracting intrusion indicator information. Correspondingly, the information feature data includes at least one type of data of the abstract, threat content related data, and intrusion indicator information of the to-be-evaluated text information. The threat content related data is other feature data of the to-be-evaluated text information except the abstract and the intrusion indicator information.

10. An electronic device, comprising: Comprise: a memory and a processor, the memory and the processor are connected; the memory is used for storing a program; the processor is used for calling the program stored in the memory to execute the method in any one of claims 1-8.

11. A computer readable storage medium, characterized in that, A computer program is stored thereon, and the computer program is run by a computer to execute the method in any one of claims 1-8.

Images