Item scoring method, apparatus, electronic device, and storage medium

Abstract

The application relates to the technical field of project scoring, and provides a project scoring method, device, electronic equipment and storage medium, the method comprises the following steps: determining a calculation formula of an objective scoring item and scoring logic of a subjective scoring item in a scoring file; extracting the assignment of each parameter in the calculation formula from a project file, and determining the objective score of the objective scoring item based on the assignment of each parameter; extracting associated elements from the project file, and determining the subjective score of the subjective scoring item based on the associated elements and the scoring logic, wherein the associated elements refer to elements with a semantic correlation greater than a threshold value with the subjective scoring item; and determining the project score based on the objective score and the subjective score. The application realizes the automation, objectivity and high efficiency of project scoring, improves the project scoring efficiency, and reduces the project scoring cost.

Description

[0001] The present application claims priority to the patent application with the application date of May 9, 2025, the application number of 2025105940651, and the invention name of "Project scoring method, device, electronic equipment and storage medium". TECHNICAL FIELD

[0002] The present application relates to the technical field of project scoring, in particular to a project scoring method, device, electronic equipment and storage medium. BACKGROUND

[0003] Project scoring refers to the process of comprehensively evaluating a project according to its importance, feasibility, and expected benefits in multiple dimensions during the project approval process of government informationization projects.

[0004] Currently, the scoring of government informationization projects is mainly completed manually. Usually, the staff of the government approval department reviews a large amount of project declaration materials, review rules, and third-party evaluation reports, and then scores the project based on their own experience and understanding. However, manually reviewing and analyzing a large amount of materials is time-consuming and labor-intensive, affecting the approval efficiency, and the scoring results are easily affected by the subjective factors of the staff. SUMMARY

[0005] The present application provides a project scoring method, device, electronic equipment and storage medium to solve the defects in the prior art.

[0006] The present application provides a project scoring method, comprising the following steps:

[0007] Determine the calculation formula of the objective scoring item and the scoring logic of the subjective scoring item in the scoring file;

[0008] Extract the assignment of each parameter in the calculation formula from the project file, and determine the objective score of the objective scoring item based on the assignment of each parameter;

[0009] Extract the associated elements from the project file, and determine the subjective score of the subjective scoring item based on the associated elements and the scoring logic, wherein the associated elements refer to elements with a semantic correlation greater than a threshold value with the subjective scoring item;

[0010] Determine the project score based on the objective score and the subjective score.

[0011] According to the project scoring method provided by the present application, the associated elements are extracted from the project file, comprising:

[0012] Based on the project file and the subjective scoring item, a prompt text is constructed;

[0013] Based on the element extraction model, the prompt text is applied to extract the associated elements from the project file.

[0014] According to the project scoring method provided by the application, the prompt text is constructed based on the project file and the subjective scoring item, which comprises:

[0015] Based on the type of the project file, a plurality of candidate historical project files are determined;

[0016] The candidate historical project file with the highest project scoring accuracy is taken as the target project file;

[0017] The target prompt text of the target project file is taken as a template, and the prompt text is constructed based on the project file and the subjective scoring item.

[0018] According to the project scoring method provided by the application, the prompt text is constructed based on the project file and the subjective scoring item, which comprises:

[0019] Based on the subjective scoring item, a target keyword is extracted from the project file;

[0020] The keyword in the target prompt text of the target project file is replaced by the target keyword to obtain the prompt text.

[0021] According to the project scoring method provided by the application, the prompt text is applied to extract the associated elements from the project file based on the element extraction model, which comprises:

[0022] Based on the element extraction model, the prompt text is applied to extract a plurality of initial associated elements from the project file;

[0023] The problem text of the plurality of initial associated elements is received;

[0024] Based on the element extraction model, the problem text and historical project information are applied to screen the associated elements from the plurality of initial associated elements.

[0025] According to the project scoring method provided by the application, the associated elements are extracted from the project file, which comprises:

[0026] The project file is respectively input into a plurality of expert models to obtain the initial associated elements output by each expert model;

[0027] Based on the weight of each expert model, the initial associated elements output by each expert model are fused to obtain the associated elements.

[0028] According to the project scoring method provided by the application, each expert model is trained based on the following steps:

[0029] Obtaining a sample project file and a sample associated element related to a subjective scoring item in the sample project file;

[0030] Based on the weakness of each expert model, the sample project file is disturbed to obtain an adversarial sample project file, and the weakness of each expert model is used to represent the key elements that are easily confused by each expert model;

[0031] Based on the initial model of each expert model, a first associated element is extracted from the sample project file, and a second associated element is extracted from the adversarial sample project file;

[0032] Based on the difference between the first associated element and the sample associated element, and the difference between the second associated element and the sample associated element, the parameters of each expert model are updated.

[0033] According to the project scoring method provided by the application, the scoring logic is determined based on the following steps:

[0034] Extracting a description text from the scoring file, the description text is used to describe the scoring requirements of the subjective scoring item;

[0035] Based on the keywords in the description text and the logical relationship between the keywords, the scoring logic is determined.

[0036] According to the project scoring method provided by the application, the description text is extracted from the scoring file, which includes:

[0037] In the case that the preset word exists in the scoring file, the initial description text is extracted from the scoring file based on the position information of the preset word in the scoring file;

[0038] Based on the semantic association degree between each word in the initial description text and the preset word, the description text is determined from the initial description text.

[0039] According to the project scoring method provided by the application, the project score is determined based on the objective score and the subjective score, which includes:

[0040] In the case that the target scoring item is included in the subjective scoring item, the subjective score of the target scoring item is determined;

[0041] In the case that the subjective score of the target scoring item is less than a threshold value, the subjective score of the target scoring item is taken as the project score;

[0042] In a case where the subjective score of the target score item is greater than or equal to the threshold value, the item score is determined based on the objective score and the subjective score.

[0043] According to the item score determination method provided in the present application, the item score is determined based on the objective score and the subjective score, which includes:

[0044] The item score is determined by fusing the objective score and the subjective score based on a score item weight, the score item weight being determined based on the importance of the subjective score item and the objective score item.

[0045] The present application further provides an item score determination device, which includes the following modules:

[0046] A first determination unit is configured to determine a calculation formula of an objective score item and a score logic of a subjective score item in a score file;

[0047] A second determination unit is configured to extract the assignment of each parameter in the calculation formula from a project file and determine the objective score of the objective score item based on the assignment of each parameter;

[0048] A third determination unit is configured to extract a related element from the project file and determine the subjective score of the subjective score item based on the related element and the score logic, the related element being an element with a semantic correlation greater than a threshold value with the subjective score item;

[0049] An item score determination unit is configured to determine an item score based on the objective score and the subjective score.

[0050] The present application further provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the item score determination method according to any one of the above embodiments when executing the program.

[0051] The present application further provides a non-transitory computer readable storage medium, which stores a computer program executable by a processor to implement the item score determination method according to any one of the above embodiments.

[0052] The present application further provides a computer program product, which includes a computer program executable by a processor to implement the item score determination method according to any one of the above embodiments.

[0053] The project scoring method, device, electronic equipment and storage medium provided by the application determine the calculation formula of the objective scoring item and the scoring logic of the subjective scoring item in the scoring file, extract the assignment of each parameter in the calculation formula from the project file to determine the objective score of the objective scoring item, extract the associated elements from the project file and determine the subjective score of the subjective scoring item based on the associated elements and the scoring logic, and then determine the project score based on the objective score and the subjective score, realizing the automation, objectivity and efficiency of the project scoring, improving the project scoring efficiency and reducing the project scoring cost. BRIEF DESCRIPTION OF DRAWINGS

[0054] In order to more clearly illustrate the technical solutions in the application or prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative effort.

[0055] Figure 1 is one of the flowcharts of the project scoring method provided by the application.

[0056] Figure 2 is the second flowchart of the project scoring method provided by the application.

[0057] Figure 3 is the third flowchart of the project scoring method provided by the application.

[0058] Figure 4 is the fourth flowchart of the project scoring method provided by the application.

[0059] Figure 5 is the fifth flowchart of the project scoring method provided by the application.

[0060] Figure 6 is the sixth flowchart of the project scoring method provided by the application.

[0061] Figure 7 is the seventh flowchart of the project scoring method provided by the application.

[0062] Figure 8 is the eighth flowchart of the project scoring method provided by the application.

[0063] Figure 9 is the flowchart of the scoring file processing method provided by the application.

[0064] Figure 10 is the structural diagram of the project scoring device provided by the application.

[0065] Figure 11 is a structural schematic diagram of an electronic device provided by the present application. DETAILED DESCRIPTION

[0066] To make the objectives, technical solutions and advantages of the present application clearer, the technical solutions of the present application will be described below in conjunction with the accompanying drawings in the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0067] At present, the scoring of government informationization projects mainly depends on manual work. Usually, the staff of the government approval department reviews a large number of project declaration materials, review rules and third-party evaluation reports, and then scores the project according to their own experience and understanding. However, manually reviewing and analyzing a large number of materials is time-consuming and labor-intensive, which affects the approval efficiency, and the scoring results are easily affected by the subjective factors of the staff, and it is difficult to guarantee the objectivity and fairness of the scoring. In addition, different staff may have different scoring results for the same project, which is difficult to guarantee the consistency of the scoring, and a large number of approval personnel need to be invested to score, which increases the labor cost.

[0068] To this end, the present application provides a project scoring method, which aims to automatically, objectively and efficiently score the project, so as to improve the project scoring efficiency and reduce the project scoring cost. Among them, the method can be applied to scoring government informationization projects, can be applied to scoring scientific and technological innovation projects, and can also be applied to scoring other projects (such as information technology projects, infrastructure projects, environmental protection projects, social service projects, etc.). In order to facilitate the understanding of the technical solutions of the present application, the following embodiments are described by taking the application to government informationization projects as an example.

[0069] Among them, Figure 1 is one of the flowcharts of the project scoring method provided by the present application, as shown in Figure 1 The method comprises steps 110, 120, 130 and 140.

[0070] Step 110, determining the calculation formula of the objective scoring item and the scoring logic of the subjective scoring item in the scoring file.

[0071] Here, the score file refers to a file that standardizes the project scoring process, which defines scoring items, scoring standards, scoring weights, and the like. Among them, the score file here can be an examination rule, which is usually a specific provision and standard for guiding the project approval, acceptance, and the like, such as a specific provision and standard for guiding the approval, acceptance, and the like of the government informationization project. Among them, the score file can be obtained by expert review, such as organizing an expert group to discuss and demonstrate various scoring items of the government informationization project, and finally determining the scoring standards and weights. The score file can also be obtained by data mining and analyzing historical project data, such as using a machine learning algorithm to analyze historical government informationization project data, extracting key scoring items related to project success rate, social benefit, and the like, and determining the score file based on the key scoring items.

[0072] In addition, the scoring item in the score file refers to a specific indicator for measuring the quality and value of the project, and the scoring item usually includes objective scoring items and subjective scoring items. The objective scoring item refers to an indicator that can be quantified by a specific numerical value or standard, for example, the project budget amount and the project completion time are quantifiable data indicators, and thus the project budget amount and the project completion time are objective scoring items. The subjective scoring item refers to an indicator that is difficult to quantify by a specific numerical value or standard, for example, project innovation, social influence, and technical risk are unquantifiable, and thus project innovation, social influence, and technical risk are subjective scoring items.

[0073] For objective scoring items, they can be quantified by a specific numerical value or standard, and the calculation formula in the score file is a mathematical expression for calculating the score of the objective scoring item, which contains one or more parameters corresponding to specific data items in the project file. For example, for the objective scoring item—project budget amount, the corresponding calculation formula can be project budget amount score = (total budget amount - actual expenditure) / total budget amount x weight.

[0074] For subjective scoring items, considering that they cannot be quantified by a specific numerical value or standard, the embodiments of the present application realize quantitative evaluation of subjective scoring items by means of the scoring logic of subjective scoring items. Among them, the scoring logic can be understood as a set of rules or conditions transformed according to the subjective scoring item, which is used for quantitative scoring of the subjective scoring item.

[0075] For example, for the subjective scoring item of project innovation, the relevant description in the scoring file is "whether the project has significant innovation in technology, application or business model, and whether it can solve existing technical problems or meet new market demand". According to the description, the scoring logic of project innovation can be determined as extracting the description of technical scheme, application scenario and business model in the project file, judging whether it contains new technology, method or idea, and whether it solves existing problems or meets new demand, that is, based on the scoring logic, the innovation can be scored according to the innovation-related description in the project file, combined with the experience of experts, for example, divided into "high", "medium" and "low" three levels, and assigned with corresponding scores.

[0076] Step 120, extract the assignment of each parameter in the calculation formula from the project file, and determine the objective score of the objective scoring item based on the assignment of each parameter.

[0077] Specifically, the project file refers to the document set submitted by the project reporting unit, containing detailed information of the project, such as project declaration, feasibility study report, technical scheme, etc. The scoring file is used to evaluate and score the project file. For example, the scoring file is the pre-set government informationization project review rules, and the project file can include project application, project feasibility study report, project technical scheme, project budget table, etc.

[0078] In addition, the calculation formula usually includes one or more parameters, which correspond to specific data items in the project file, and the assignment of these parameters can be directly obtained from the project file, or obtained by simple calculation on the data in the project file. After extracting the assignment of each parameter from the project file, the assignment of each parameter can be substituted into the corresponding calculation formula, and then the objective score of the corresponding objective scoring item can be obtained. Generally, the higher the objective score, the better the performance of the corresponding objective scoring item, and the more obvious the advantages of the project.

[0079] As an optional embodiment, the assignment of each parameter in the calculation formula can be extracted from the project file by regular expression matching, keyword extraction and other technical means, for example, the regular expression is used to extract the specific value of the budget amount from the "total budget" field of the project declaration.

[0080] Step 130, extract the associated elements from the project file, and determine the subjective score of the subjective scoring item based on the associated elements and the scoring logic, the associated elements refer to the elements with a semantic correlation greater than a threshold with the subjective scoring item.

[0081] Specifically, the associated element refers to a piece of information in the project file related to a specific subjective scoring item, which can be a sentence, a paragraph or a specific part of the document, and the semantic correlation between the associated element and the subjective scoring item is greater than a threshold. The semantic correlation can be understood as the matching degree between the meaning expressed by the associated element and the content investigated by the subjective scoring item. For example, for the subjective scoring item "project innovation", the element 1 "the project adopts a brand-new blockchain technology to realize data security sharing and privacy protection", and the element 2 "the project uses a mature technology widely used in the market", the element 1 is used to describe the technical solution adopted by the project, and the element 2 is used to describe the maturity of the technical solution of the project. Obviously, the semantic correlation between element 1 and the subjective scoring item "project innovation" is greater, and element 1 is used as the associated element corresponding to the subjective scoring item "project innovation".

[0082] In addition, the associated element is a piece of original information in the project file, and the project file includes project details, that is, the project file is used to represent the specific performance and characteristics of various aspects of the project, so the associated element from the original text of the project file is also used to represent the specific performance and characteristics of the project in a specific aspect, and the project score can be evaluated based on the associated element.

[0083] Further, considering that the subjective scoring item cannot be directly quantified by an explicit numerical value or standard, it needs to be quantified by scoring logic. On this basis, the embodiments of the present application combine the associated element and the scoring logic to determine the subjective score of the subjective scoring item, wherein the associated element is used to provide the basis and material for evaluation, and the scoring logic is used to guide how to score based on these basis and material, and then the combination of the two can be used for comprehensive analysis and judgment to determine the subjective score of the subjective scoring item. The higher the subjective score is, the better the project is in this aspect, and the more it meets the scoring standard.

[0084] For example, the associated element is "the project adopts a new artificial intelligence-based algorithm that can significantly improve data processing efficiency", and the scoring logic of the subjective scoring item "technological innovation" is "if the project adopts new technology, new method or new idea, and can significantly improve efficiency or performance, the score of technological innovation is higher". The process of scoring the subjective scoring item "technological innovation" in combination with the two can be: first, analyze the associated element, and it can be known that the project adopts "an artificial intelligence-based algorithm"; then, according to the scoring logic, it is judged whether the algorithm belongs to new technology and whether it can "significantly improve data processing efficiency"; if the judgment results are both positive, a higher score of technological innovation can be given, for example, 90 points.

[0085] Step 140, determining the project score based on the objective score and the subjective score.

[0086] Specifically, the objective score is used to represent the quantitative result of the objective score item, the subjective score is used to represent the comprehensive evaluation of the subjective score item, and then the overall quality and value of the project can be comprehensively determined based on the objective score and the subjective score, that is, the project score is obtained, that is, the project score is used to evaluate the comprehensive strength and investment value of the project.

[0087] As an optional embodiment, the objective score and the subjective score can be weighted and added based on the score item weight to obtain the project score. The score item weight can be pre-set in the score file, or can be determined based on the importance of the subjective score item and the objective score item, for example, a higher score item weight is given to a score item with higher importance, and a lower score item weight is given to a score item with lower importance.

[0088] As an optional embodiment, the steps 110-140 can be implemented by a large language model, that is, the score file and the project file are input into the large language model, and the large language model executes the steps 110-140 to output the project score.

[0089] The large language model (LLM) is a natural language processing (NLP) model with a large number of parameters, and the number of model parameters and / or the complexity of the model structure exceeds a preset threshold. The model processes large-scale text data during training and has the ability to understand and generate natural language. For example, the large language model can include the BERT series model, the Wenxin Yiyang model, etc.

[0090] The project scoring method provided by the embodiments of the present application determines the calculation formula of the objective score item and the scoring logic of the subjective score item in the score file, extracts the assignment of each parameter in the calculation formula from the project file to determine the objective score of the objective score item, extracts the associated elements from the project file and determines the subjective score of the subjective score item based on the associated elements and the scoring logic, and then determines the project score based on the objective score and the subjective score. The automation, objectivity and efficiency of the project scoring are realized, the project scoring efficiency is improved, and the project scoring cost is reduced.

[0091] Based on any of the above embodiments, Figure 2 is a flowchart of the project scoring method provided by the present application, as Figure 2 shown, the method comprises:

[0092] Step 210, determining the calculation formula of the objective score item and the scoring logic of the subjective score item in the score file.

[0093] Specifically, the scoring file is the key to standardize the project scoring process, which defines the scoring items, scoring criteria and weights. The scoring file can be the review rules, obtained by expert review or data mining historical project data.

[0094] The scoring items are divided into objective and subjective categories. Objective scoring items can be quantified, such as project budget amount and project completion time, which are calculated by preset calculation formulas. Subjective scoring items are difficult to quantify, such as project innovation, social impact and technical risk, which are quantitatively evaluated by means of scoring logic.

[0095] Scoring logic is a set of rules or conditions for converting subjective scoring items to quantify them. For example, for project innovation, the scoring logic is to extract relevant descriptions in the project file, determine whether it contains new technology, method or idea, and whether it solves problems or meets needs, and grade the score accordingly.

[0096] Step 220, extract the assignment of each parameter in the calculation formula from the project file, and determine the objective score of the objective scoring item based on the assignment of each parameter.

[0097] Specifically, the project file is a collection of detailed information documents submitted by the project reporting unit, such as the declaration, report, plan, etc. The scoring file is used to evaluate the project file, which is the preset review rules.

[0098] The parameters in the calculation formula correspond to the data in the project file, which can be directly extracted or simply calculated. After extracting the parameters and substituting them into the calculation formula, the objective score can be obtained. The higher the objective score, the more obvious the project advantage.

[0099] Among them, regular expression matching, keyword extraction and other technical means can be used to extract the parameter assignment required by the calculation formula from the project file. For example, regular expressions are used to extract the total budget from the project declaration.

[0100] Step 230, based on the project file and the subjective scoring item, construct a prompt text; based on the element extraction model, apply the prompt text to extract the associated elements from the project file, and based on the associated elements and the scoring logic, determine the subjective score of the subjective scoring item, the associated elements refer to the elements whose semantic relevance to the subjective scoring item is greater than a threshold.

[0101] Specifically, the prompt text refers to the natural language description or instruction used to guide the element extraction model to extract the associated elements, which can be a prompt text. After constructing the prompt text, input the prompt text into the element extraction model, and based on the key information and instructions in the prompt text, the element extraction model extracts the associated elements from the project file based on its own knowledge and understanding.

[0102] Among them, the prompt text is as follows:

[0103] "Please extract the associated elements of the subjective scoring item 'project innovation' from the following project file. The associated elements refer to the novelty, breakthrough and uniqueness of the project in terms of technology, application or business model, including but not limited to the application of new technology, the adoption of new methods, the proposal of new ideas and innovative solutions to existing problems.

[0104] {project file}".

[0105] As an optional embodiment, the element extraction model can be trained based on sample prompt text and corresponding sample associated elements, and the sample prompt text is determined based on the sample project file and the sample subjective scoring item.

[0106] Step 240, based on the objective score and the subjective score, determining the project score.

[0107] Specifically, the objective score quantifies the objective scoring item, and the subjective score evaluates the subjective scoring item. The combination of the two determines the overall quality and value of the project, and obtains the project score, which is used to evaluate the comprehensive strength and investment value of the project.

[0108] As an optional embodiment, the objective score and the subjective score can be weighted and summed based on the scoring item weight to obtain the project score.

[0109] Based on any of the above embodiments, based on the project file and the subjective scoring item, a prompt text is constructed, including:

[0110] Based on the type of the project file, a plurality of candidate historical project files are determined;

[0111] The candidate historical project file with the highest project score accuracy is selected as the target project file;

[0112] The target prompt text of the target project file is used as a template, and the prompt text is constructed based on the project file and the subjective scoring item.

[0113] Specifically, the type of the project file refers to the field or theme to which the project belongs. In the application scenario (government informationization) of the embodiments of the present application, the type of the project file can include government informationization projects. Among them, the type of the project file can be determined based on the project name or key words in the project declaration materials; the type of the project file can also be determined based on the classification standard of the project approval department.

[0114] The candidate historical project file refers to a historical project file of the same type as the current project file. For example, if the current project file is in the field of government informationization project, the candidate historical project file should also be in the field of government informationization project. The candidate historical project file can be determined from the historical project database or archive from the historical project file of the same type as the current project file.

[0115] The project scoring accuracy can be understood as the accuracy of scoring the current project file using the prompt text constructed by the candidate historical project file. The higher the project scoring accuracy, the more suitable the prompt file of the corresponding candidate historical project file for the current project file, the more accurately reflecting the quality of the project, and thus more effectively guiding the model to score. The candidate historical project file with the highest project scoring accuracy has a higher quality and applicability of the corresponding prompt text, so as to be used as the target project file.

[0116] In addition, the target prompt text refers to the prompt text of the target project file, that is, the text used to guide the model to extract the associated elements from the target project file. Since the target project file has a high scoring accuracy, it indicates that the prompt text can effectively guide the model to extract key information. The target prompt text of the target project file is used as a template, so that the structure and content of the target prompt text can be referred to, and the project file and the subjective scoring item are combined to construct the prompt text of the project file. The prompt text can more accurately guide the model to extract the associated elements, improve the accuracy and efficiency of project scoring, thereby optimizing the construction process of the prompt text and improving the scoring quality.

[0117] As an optional embodiment, a pre-trained prompt text generation model can be used, and the target prompt text is used as a guide to generate a new prompt text to construct the prompt text.

[0118] As can be seen, the embodiments of the present application optimize the construction of the prompt text by using the information of the historical project file, thereby improving the accuracy and efficiency of project scoring.

[0119] Based on any of the above embodiments, the target prompt text of the target project file is used as a template, and the prompt text is constructed based on the project file and the subjective scoring item, which includes:

[0120] Based on the subjective scoring item, the target key word is extracted from the project file;

[0121] The key word of the target project file in the target prompt text is replaced by the target key word to obtain the prompt text.

[0122] Specifically, the target keyword refers to a key word closely related to the subjective scoring item and capable of reflecting the core content of the project file, which is used to represent the specific performance or characteristics of the project in a specific subjective scoring item. For example, the subjective scoring item is "innovation", and the target keyword can be "new technology application", "original design", and "breakthrough progress". Among them, the target keyword can be extracted from the project file using a keyword extraction algorithm (such as TF-IDF, TextRank).

[0123] The keyword in the target prompt text refers to a word in the target prompt text that is used to guide the model to focus on a specific information segment, and is usually related to the characteristics of the target project file. In constructing the prompt text, in order to ensure that the prompt text is consistent with the content and theme of the project file and improve the pertinence and effectiveness of the prompt, the embodiments of the present application replace the keywords in the target prompt text with the target keywords, so that the prompt text is more in line with the actual situation of the current project file, better guides the model to extract information related to the subjective scoring item, and finally improves the accuracy of the score.

[0124] Among them, after constructing the prompt text, the prompt text can be optimized, for example, using natural language processing techniques for grammar correction, semantic optimization, etc., to improve the quality and readability of the prompt text; the prompt text can also be adjusted and improved through manual review to ensure that it can accurately express the scoring intention and effectively guide the model to score.

[0125] Based on any of the above embodiments, Figure 3 is a third flowchart of the project scoring method provided by the present application, as shown in Figure 3 , the method comprises:

[0126] Step 310, determining the calculation formula of the objective scoring item and the scoring logic of the subjective scoring item in the scoring file.

[0127] Specifically, the scoring file is a key to standardize the project scoring process, which defines the scoring items, scoring standards and weights. The scoring file can be a review rule, which is obtained through expert review or data mining of historical project data.

[0128] The scoring items are divided into objective and subjective categories. The objective scoring item can be quantified, such as the project budget amount and the project completion time, which is calculated by a preset calculation formula. The subjective scoring item is difficult to quantify, such as the project innovation, social influence and technical risk, which is quantitatively evaluated by means of scoring logic.

[0129] The scoring logic is a set of rules or conditions converted from the subjective scoring item, which is used for quantitative scoring.

[0130] Step 320, extract the assignment of each parameter in the calculation formula from the project file, and determine the objective score of the objective scoring item based on the assignment of each parameter.

[0131] Specifically, the project file is a collection of detailed information documents submitted by the project reporting unit, such as the declaration, report, scheme, etc. The scoring file is used to evaluate the project file, which is the pre-set review rules.

[0132] The parameters in the calculation formula correspond to the data in the project file, which can be directly extracted or simply calculated. After extracting the parameters and substituting them into the calculation formula, the objective score can be obtained.

[0133] Step 330, based on the project file and the subjective scoring item, construct a prompt text; based on the element extraction model, apply the prompt text to extract a plurality of initial associated elements from the project file; receive a plurality of initial associated elements of the problem text; based on the element extraction model, apply the problem text and the historical project information to filter the associated elements from the plurality of initial associated elements, and based on the associated elements and the scoring logic, determine the subjective score of the subjective scoring item, the associated elements refer to the elements with a semantic correlation greater than a threshold with the subjective scoring item.

[0134] Specifically, considering that the initial associated elements extracted by the element extraction model based on the prompt text may have redundant information or low relevance to the subjective scoring item due to the limitations of the model itself or the inaccuracy of the prompt text. Based on this, the embodiment of the present application adopts a dialogue mode to further screen the extracted initial associated elements, and then accurately obtains the associated elements corresponding to the subjective scoring item.

[0135] Among them, the initial associated elements refer to a plurality of information segments preliminarily extracted from the project file by the element extraction model based on the prompt text, without screening, and the problem text refers to a question asked to confirm its relevance to the subjective scoring item.

[0136] For example, the subjective scoring item is "project innovation", and the plurality of initial associated elements extracted by the element extraction model include: "the project adopts blockchain technology", "the project team members have rich project management experience", and "the project can improve the efficiency of government services". It can be seen that "the project team members have rich project management experience" and "the project can improve the efficiency of government services" in the plurality of initial associated elements have low relevance to the subjective scoring item. At this time, the problem text "whether these elements are all directly related to the innovation of the project, please filter out the elements with low relevance" can be input into the element extraction model, and the element extraction model can screen the plurality of initial associated elements, and finally output the associated elements with high relevance to the subjective scoring item.

[0137] The question text can be a question raised by a user for the plurality of initial associated elements to further clarify the relevance of the elements to the subjective scoring item, or can be a standard question set in advance, for example, "please judge whether these elements directly describe the innovation of the project and give reasons", and the embodiment of the application does not make specific limitations.

[0138] Therefore, the embodiment of the application extracts the associated elements related to the subjective scoring item more accurately through the process of "extraction-questioning-screening", thereby improving the accuracy of the project score.

[0139] Step 340, determining the project score based on the objective score and the subjective score.

[0140] Specifically, the objective score quantifies the objective scoring item, and the subjective score comprehensively evaluates the subjective scoring item. The combination of the two determines the overall quality and value of the project to obtain the project score, which is used to evaluate the comprehensive strength and investment value of the project.

[0141] Based on any of the above embodiments, the associated elements are extracted from the project file, including:

[0142] The project file is input into a plurality of expert models respectively to obtain the initial associated elements output by each expert model;

[0143] The initial associated elements output by each expert model are fused based on the weight of each expert model to obtain the associated elements.

[0144] Specifically, the expert model refers to a machine learning model trained for a specific field or task, which is used to extract a specific type of associated element from the project file, and each expert model has different knowledge background and strong field. For example, the expert model 1 focuses on the technical details and innovation points of the project, thereby being able to extract elements related to technical feasibility and innovation. The expert model 2 focuses on the market prospect and economic benefit of the project, thereby being able to extract elements related to market demand and profitability. The expert model 3 focuses on the social influence and environmental impact of the project, thereby being able to extract elements related to social responsibility and sustainable development.

[0145] Considering that the project file contains complex and diverse information, a single model may be difficult to extract all key elements comprehensively and accurately, and the embodiment of the application inputs the project file into each expert model respectively, so that each expert model can analyze the project file from its own strong field to extract more comprehensive and professional initial associated elements, thereby obtaining the corresponding initial associated elements.

[0146] In view of the possibility of repetition, conflict or importance difference of the initial correlation elements output by each expert model, in order to avoid information redundancy and deviation, and ensure the accuracy and representativeness of the final extracted correlation elements, the embodiments of the present application fuse the initial correlation elements based on the weights of each expert model to obtain the correlation elements. For example, if the expert model 1 has higher accuracy in technical innovation, the elements related to technical innovation output by the expert model 1 will be given higher weight in the fusion process.

[0147] Among them, the weight of each expert model can be determined based on the historical performance (such as accuracy, recall rate, etc.) of the expert model. For example, the higher the accuracy of the expert model, the higher the weight. The weight of each expert model can be determined based on the relevance of the professional field of the expert model and the project file. For example, if the project file belongs to the field of information technology, the weight of the expert model skilled in the field of information technology is higher.

[0148] As an optional embodiment, attention mechanism can be used to dynamically adjust the weight of each expert model according to the importance of each initial correlation element, and fuse the initial correlation elements output by each expert model to obtain the correlation elements. For example, if an initial correlation element is closely related to multiple subjective scoring items, the importance of the element is higher, and the weight of the corresponding expert model will also be increased accordingly. At the same time, the weight of the attention mechanism can also be adjusted by introducing an artificial feedback mechanism, to further improve the accuracy of element fusion.

[0149] Based on any of the above embodiments, each expert model is trained based on the following steps:

[0150] Obtain a sample project file and sample correlation elements related to the subjective scoring items in the sample project file;

[0151] Based on the weaknesses of each expert model, perturb the sample project file to obtain an adversarial sample project file, and the weaknesses of each expert model are used to represent the key elements that each expert model is easy to confuse;

[0152] Based on the initial model of each expert model, extract a first correlation element from the sample project file, and extract a second correlation element from the adversarial sample project file;

[0153] Based on the difference between the first correlation element and the sample correlation element, and the difference between the second correlation element and the sample correlation element, update the parameters of each expert model.

[0154] Specifically, the sample item file refers to an item file that has been annotated and used to train the expert model, and the sample associated element refers to an element related to the subjective scoring item in the sample item file and serving as a training target. The sample associated element can be manually annotated by a domain expert. For example, the domain expert annotates technical details, original designs, etc. related to "innovation" in the sample item file. The sample associated element can also be automatically extracted using an existing knowledge base or dataset. For example, patent information related to the sample item file is automatically extracted as the sample associated element using a patent database.

[0155] The weakness of each expert model refers to a place where the expert model is prone to errors or biases when extracting associated elements, which is used to represent key elements that are easily confused by each expert model. That is, different expert models can have different weaknesses. For example, some models can have weak processing capabilities for long texts, and other models can not accurately understand professional terms. For example, an expert model is prone to misjudging some non-key terms as key elements when processing item files containing a large number of professional terms.

[0156] Since the weaknesses of each expert model can lead to inaccurate or incomplete extraction of associated elements, affecting the final scoring results, in order to ensure that the expert model can better identify and extract key elements and improve the robustness and generalization ability of the model, the embodiments of the present application perturb the sample item file when training the expert model to obtain an adversarial sample item file. The adversarial sample item file refers to a sample that is slightly modified from the original sample item file to cause misjudgment or bias in the expert model. Perturbing the sample item file refers to making subtle modifications to the text content while keeping the semantics of the sample item file unchanged. For example, some words in the sample item file can be replaced with synonyms, or some irrelevant interference information can be added. The adversarial sample item file can be obtained using a generative adversarial network (GAN). For example, the sample item file can be input into the generative adversarial network to obtain the adversarial sample item file.

[0157] In addition, the first associated element refers to the associated element extracted by the expert model from the original sample item file, and the difference between the first associated element and the sample associated element is used to represent the performance of the expert model on the original sample. When updating the expert model parameters based on the difference, the model parameters can be adjusted to more accurately extract the associated elements in the original sample.

[0158] The second associated element refers to the associated element extracted by the expert model from the adversarial sample item file, and the difference between the second associated element and the sample associated element is used to represent the robustness of the expert model after being perturbed. When updating the expert model parameters based on the difference, the resistance of the model to the adversarial sample can be improved, and the generalization performance of the model can be enhanced.

[0159] If the parameters of the expert model are updated only based on the difference between the first correlation element and the sample correlation element, the model can be over-fitted to the original sample and perform poorly on new or perturbed samples. If the parameters of the expert model are updated only based on the difference between the second correlation element and the sample correlation element, the model can pay too much attention to the adversarial samples and ignore the learning of the original samples.

[0160] Based on this, the embodiments of the present application update the parameters of each expert model based on the difference between the first correlation element and the sample correlation element, and the difference between the second correlation element and the sample correlation element, so as to take into account the performance of the model on the original sample and the adversarial sample, and improve the accuracy and robustness of the model.

[0161] As an optional embodiment, the first loss value can be determined based on the difference between the first correlation element and the sample correlation element, and the second loss value can be determined based on the difference between the second correlation element and the sample correlation element, and the training loss value can be determined in combination with the first loss value and the second loss value, and finally the parameters of the expert model are updated based on the training loss value.

[0162] Based on any of the above embodiments, Figure 4 is a fourth flowchart of the project scoring method provided by the present application, as shown in Figure 4 The method comprises the following steps.

[0163] In step 410, the calculation formula of the objective scoring item and the scoring logic of the subjective scoring item in the scoring file are determined. The scoring logic is determined based on the following steps: extracting the description text from the scoring file, and the description text is used to describe the scoring requirements of the subjective scoring item; based on the keywords in the description text and the logical relationship between the keywords, the scoring logic is determined.

[0164] Specifically, the description text is used to describe the scoring requirements of the subjective scoring item. The scoring requirements here can include scoring standards, scoring details, and scoring emphases, etc. For example, the subjective scoring item is "project innovation", and its description text in the scoring file can be "whether the project has significant innovation in technology, application or business model, whether it uses new technology, new method or new idea, whether it can solve existing technical problems or meet new market demand, and whether it has independent intellectual property rights".

[0165] As an optional embodiment, the description text can be determined based on the following steps: first, text analysis is performed on the scoring file to extract the paragraphs or sentences related to the subjective scoring item; then, the extracted text is cleaned and preprocessed to remove irrelevant information; finally, the processed text is taken as the description text.

[0166] After extracting the description text from the scoring file, considering that the description text contains various descriptions of the scoring requirements of the subjective scoring item, the embodiments of the present application determine the scoring logic based on the keywords in the description text and the logical relationship between the keywords. Here, the keywords refer to words in the description text that have representative meanings and can reflect the core evaluation criteria of the subjective scoring item.

[0167] For example, the description text is "The project adopts a new algorithm developed independently in technology, which can effectively improve data processing efficiency, and combines actual business scenarios in application to solve the bottleneck problem of existing systems", the extracted keywords include "independent research and development", "new algorithm", "improve efficiency", "combine actual", "solve the bottleneck", and the logical relationship between the keywords is "the project must adopt an independently developed new algorithm, and can improve efficiency, and needs to combine actual in application to solve existing problems", on this basis, the scoring logic can be determined as "if the project adopts an independently developed new algorithm, and can effectively improve data processing efficiency, and combines actual business scenarios in application to solve the bottleneck problem of existing systems, the project scores higher in this subjective scoring item, otherwise scores lower".

[0168] As an optional embodiment, considering that the frequently occurring words in the description text are closely related to the definition and evaluation criteria of the subjective scoring item, and have a low frequency of occurrence in the entire scoring file, which means that these words are not general words or common concepts, but are more specific to the subjective scoring item, which can effectively distinguish different scoring items. Based on this, the high-frequency words in the description text but low-frequency words in the entire scoring file can be used as keywords, and the logical relationship between the keywords can be determined, and then the keywords and the logical relationship between the keywords can be input into a large language model to extract the scoring logic of the subjective scoring item.

[0169] Step 420, extract the assignment of each parameter in the calculation formula from the project file, and determine the objective score of the objective scoring item based on the assignment of each parameter.

[0170] Specifically, the project file is a collection of detailed information documents submitted by the project reporting unit, such as the declaration, report, scheme, etc. The scoring file is used to evaluate the project file, which is a pre-set review rule.

[0171] The parameters in the calculation formula correspond to the data in the project file, which can be directly extracted or simply calculated. After extracting the parameters and substituting them into the calculation formula, the objective score can be obtained.

[0172] Step 430, extract the associated elements from the project file, and determine the subjective score of the subjective scoring item based on the associated elements and the scoring logic, the associated elements refer to elements with a semantic correlation greater than a threshold with the subjective scoring item.

[0173] Specifically, the correlation element is a piece of information in the project file related to a specific subjective scoring item, and the meaning expressed by the piece of information has a high degree of matching with the content investigated by the subjective scoring item.

[0174] The correlation element is derived from the original text of the project file, and is used to represent the specific performance and characteristics of the project in a specific aspect, and then evaluate the score of the project. In combination with the scoring logic, the subjective score of the subjective scoring item is determined. The correlation element provides the basis for evaluation, and the scoring logic guides the scoring. The two are combined to analyze and judge, determine the subjective score, and represent the performance level of the project in the scoring item. The higher the subjective score, the better the project in this aspect.

[0175] Step 440, based on the objective score and the subjective score, determining the project score.

[0176] Specifically, the objective score quantifies the objective scoring item, and the subjective score comprehensively evaluates the subjective scoring item. In combination, the overall quality and value of the project are determined, and the project score is obtained, which is used to evaluate the comprehensive strength and investment value of the project.

[0177] Based on any of the above embodiments, Figure 5 is a fifth flowchart of the project scoring method provided by the present application, as Figure 5 shown, the method comprises:

[0178] Step 510, determining the calculation formula of the objective scoring item and the scoring logic of the subjective scoring item in the scoring file. The scoring logic is determined based on the following steps: in the case that there is a preset word in the scoring file, based on the position information of the preset word in the scoring file, extracting an initial description text from the scoring file; based on the semantic association degree between each word in the initial description text and the preset word, determining a description text from the initial description text; based on the keywords in the description text and the logical relationship between the keywords, determining the scoring logic.

[0179] Specifically, the preset word here can be understood as a word that points to the subjective scoring item, which is used to assist in locating the description text related to the subjective scoring item. For example, the preset words can include "innovation", "benefit", "risk", "feasibility", "demand", etc. Since the preset word usually appears directly in the description text related to the subjective scoring item, there is usually description text near the preset word.

[0180] Based on this, the embodiment of the present application extracts an initial description text from the scoring file according to the position information of the preset word in the scoring file, such as taking the preset word as the center, extending N sentences forward and M sentences backward, and taking the extended text as the initial description text. For example, the sentence or paragraph containing the preset word can also be directly extracted as the initial description text.

[0181] Considering that the content of the score file is usually complex and contains a large amount of irrelevant information, the initial description text can contain noise or content with low relevance to the subjective score item. In order to further improve the extraction accuracy of the description text, the embodiment of the application determines the description text from the initial description text based on the semantic correlation degree between each word in the initial description text and the preset word. The semantic correlation degree refers to the similarity or correlation degree between each word and the preset word in semantics. The semantic correlation degree between the two can be calculated using word vectors, knowledge graphs and other technologies. The higher the semantic correlation degree, the higher the relevance of the corresponding word in the initial description text to the score standard or score requirement indicated by the preset word.

[0182] As an optional embodiment, the words with a semantic correlation degree greater than a threshold value can be selected from the initial description text as candidate words, and the description text can be constructed according to the logic between each candidate word.

[0183] As can be seen, the embodiment of the application further filters the initial description text, so that the description text not only accurately describes the score requirements of the subjective score item, but also effectively removes irrelevant information, improving the accuracy of subsequent score logic determination.

[0184] Step 520, extract the assignment of each parameter in the calculation formula from the project file, and determine the objective score of the objective score item based on the assignment of each parameter.

[0185] Specifically, the project file is a collection of detailed information documents submitted by the project reporting unit, such as a declaration, a report, a plan, etc. The score file is used to evaluate the project file, which is a preset review rule.

[0186] The parameters in the calculation formula correspond to the data in the project file, which can be directly extracted or simply calculated. After extracting the parameters and substituting them into the calculation formula, the objective score can be obtained.

[0187] Step 530, extract the associated elements from the project file, and determine the subjective score of the subjective score item based on the associated elements and the score logic. The associated elements refer to elements with a semantic correlation degree greater than a threshold value with the subjective score item.

[0188] Specifically, the associated elements are information segments in the project file related to a specific subjective score item, and the meaning expressed by the associated elements has a high degree of matching with the content investigated by the subjective score item.

[0189] The association element is derived from the original text of the project file, and is used to represent the specific performance and characteristics of the project in a specific aspect, and then evaluate the project score. The association element is combined with the scoring logic to determine the subjective score of the subjective scoring item. The association element provides the basis for evaluation, and the scoring logic guides the scoring. The two are combined to analyze and determine the subjective score, representing the performance level of the project in this scoring item. The higher the subjective score, the better the project in this aspect.

[0190] Step 540, determining the project score based on the objective score and the subjective score.

[0191] Specifically, the objective score quantifies the objective scoring item, and the subjective score comprehensively evaluates the subjective scoring item. The combination of the two determines the overall quality and value of the project, and obtains the project score, which is used to evaluate the comprehensive strength and investment value of the project.

[0192] Based on any of the above embodiments, Figure 6 is a sixth flowchart of the project scoring method provided by the present application, as shown in the figure, the method comprises: Figure 6

[0193] Step 610, determining the calculation formula of the objective scoring item and the scoring logic of the subjective scoring item in the scoring file.

[0194] Specifically, the scoring file is the key to standardize the project scoring process, which defines the scoring items, scoring standards and weights. The scoring file can be a review rule, which is obtained by expert review or data mining historical project data.

[0195] The scoring items are divided into objective and subjective types. The objective scoring item can be quantified, such as the project budget amount and the project completion time, which is calculated by a preset calculation formula. The subjective scoring item is difficult to quantify, such as the project innovation, social influence and technical risk, which is quantitatively evaluated by means of scoring logic.

[0196] The scoring logic is a set of rules or conditions according to the subjective scoring item, which is used to quantitatively score it.

[0197] Step 620, extracting the assignment of each parameter in the calculation formula from the project file, and determining the objective score of the objective scoring item based on the assignment of each parameter.

[0198] Specifically, the project file is a set of detailed information documents submitted by the project reporting unit, such as the declaration, report and scheme. The scoring file is used to evaluate the project file, which is a preset review rule.

[0199] The parameters in the calculation formula correspond to the data in the project file, which can be directly extracted or simply calculated. After extracting the parameters and substituting them into the calculation formula, the objective score can be obtained.

[0200] ​Step 630, extracting the associated elements from the project file, and determining the subjective score of the subjective score item based on the associated elements and the scoring logic, the associated elements refer to the elements with a semantic correlation greater than a threshold with the subjective score item.

[0201] Specifically, the associated elements are information segments related to a specific subjective score item in the project file, and the meanings expressed by the information segments have a high degree of matching with the content investigated by the subjective score item.

[0202] The associated elements are derived from the original text of the project file, and are used to represent the specific performance and characteristics of the project in a specific aspect, and then evaluate the score of the project. The associated elements and the scoring logic are combined to determine the subjective score of the subjective score item. The associated elements provide the basis for evaluation, and the scoring logic guides the scoring. The two are combined to analyze and determine the subjective score, which represents the performance level of the project in the scoring item. The higher the subjective score, the better the project in this aspect.

[0203] Step 640, in the case that the target score item is included in the subjective score item, determining the subjective score of the target score item; in the case that the subjective score of the target score item is less than a threshold, taking the subjective score of the target score item as the project score; in the case that the subjective score of the target score item is greater than or equal to the threshold, determining the project score based on the objective score and the subjective score.

[0204] Specifically, the target score item is a specific score item that has a decisive role in the project score, and its result directly affects whether the project passes or not. For example, the target score item can be understood as a mandatory score item, that is, the target score item has a veto power. For example, if the score of the target score item does not meet the requirements, it can be directly determined that the project does not pass, such as directly outputting the project score as 0. If the score of the target score item meets the requirements, the scores of the remaining target score items can be further determined.

[0205] It should be noted that since the target score item usually involves the evaluation of the fundamental problems of the project, such as policy compliance, safety, etc., these problems are difficult to be directly quantified by objective data, so the target score item usually appears in the subjective score item, but not in the objective score item. Among them, the target score item can be pre-marked by experts, or can be learned from historical data by machine learning algorithm, for example, by analyzing the historical projects that are rejected, finding out the key factors that lead to rejection, and setting the score items corresponding to these factors as target score items, and the embodiments of the present application do not make specific limitations.

[0206] The subjective score of the target score item is used to represent the performance level of the target score item in the item. If the subjective score is less than a threshold value, it indicates that the item has serious defects in the key aspect and cannot meet the basic requirements, that is, the item fails, and at this time, the subjective score of the target score item can be directly used as the project score without considering other scores. If the subjective score of the target score item is greater than or equal to the threshold value, it indicates that the item meets the basic requirements in the key aspect and has the qualification for further evaluation, and at this time, the project score can be determined by comprehensively considering the objective score and the subjective score.

[0207] Based on any of the above embodiments, Figure 7 is a seventh flowchart of the project scoring method provided by the application, as shown in the figure, the method comprises: Figure 7

[0208] Step 710, determining the calculation formula of the objective score item and the scoring logic of the subjective score item in the scoring file.

[0209] Specifically, the scoring file is a key to standardize the project scoring process, which defines the score item, the score standard and the weight. The scoring file can be an evaluation rule, which is obtained by expert evaluation or data mining historical project data.

[0210] The score item is divided into two categories: objective and subjective. The objective score item can be quantified, such as the project budget amount and the project completion time, and the score is calculated by a preset calculation formula. The subjective score item is difficult to quantify, such as the project innovation, social influence and technical risk, and a set of rules or conditions is used to realize quantitative evaluation.

[0211] The scoring logic is a set of rules or conditions for converting the subjective score item, which is used for quantitative scoring.

[0212] Step 720, extracting the assignment of each parameter in the calculation formula from the project file, and determining the objective score of the objective score item based on the assignment of each parameter.

[0213] Specifically, the project file is a set of detailed information documents submitted by the project declaration unit, such as declaration, report and scheme. The scoring file is used to evaluate the project file, which is a preset evaluation rule.

[0214] The parameters in the calculation formula correspond to the data in the project file, which can be directly extracted or simply calculated. After extracting the parameters and substituting them into the calculation formula, the objective score can be obtained.

[0215] Step 730, extracting the associated elements from the project file, and determining the subjective score of the subjective score item based on the associated elements and the scoring logic, the associated elements refer to the elements with a semantic correlation greater than a threshold value with the subjective score item.

[0216] ​Specifically, the correlation element is a piece of information in the project file related to a specific subjective scoring item, and the meaning expressed by the piece of information matches the content investigated by the subjective scoring item to a high degree.

[0217] The correlation element is derived from the original text of the project file, and is used to represent the specific performance and characteristics of the project in a specific aspect, and to evaluate the score of the project. In combination with the scoring logic, the subjective score of the subjective scoring item is determined. The correlation element provides the basis for evaluation, and the scoring logic guides the scoring. The two are combined to analyze and determine the subjective score, representing the performance level of the project in the scoring item. The higher the subjective score, the better the project in this aspect.

[0218] Step 740, in the case of including a target scoring item in the subjective scoring item, determining the subjective score of the target scoring item; in the case that the subjective score of the target scoring item is less than a threshold value, taking the subjective score of the target scoring item as the project score; in the case that the subjective score of the target scoring item is greater than or equal to the threshold value, based on a scoring item weight, fusing the objective score and the subjective score to determine the project score, the scoring item weight being determined based on the importance of the subjective scoring item and the objective scoring item.

[0219] Specifically, the scoring item weight refers to the proportion of each scoring item, which is used to measure the contribution of the scoring item to the overall quality and value of the project. Considering that different scoring items have different overall impacts on the project, some scoring items may be more critical and directly related to the success or failure of the project. After obtaining the objective score and the subjective score, the embodiments of the present application fuse the objective score and the subjective score based on the scoring item weight to obtain the project score.

[0220] In the fusion of the objective score and the subjective score, the objective score and the subjective score can be weighted and added based on the scoring item weight to obtain the project score, or the objective score and the subjective score can be nonlinearly weighted according to the mutual relationship between the scoring items to obtain the project score. For example, assuming that the scoring item A (technological innovation) and the scoring item B (risk controllability) are included, the score of the scoring item A is score1, and the corresponding scoring weight is 0.6, the score of the scoring item B is score2, and the corresponding scoring weight is 0.4. Considering the mutual relationship between the scoring item A and the scoring item B, if the risk is uncontrollable, it may lead to the failure of the project, that is, the score of the scoring item B will affect the score of the scoring item A. On this basis, a nonlinear function can be introduced: project score = 0.6 × score1 × (score2 / 100) + 0.4 × score2.

[0221] The score item weight is determined based on the importance of the subjective score item and the objective score item, that is, the higher the importance of any score item, the greater the corresponding score item weight. As an optional embodiment, the importance of each score item can be determined by expert scoring method, or the influence of each score item on the project result can be determined by analyzing the score data and project results of historical projects, and the influence degree is taken as the importance of the score item.

[0222] Based on any of the above embodiments, Figure 8 is the eighth flowchart of the project scoring method provided by the present application, as shown in the figure, the method comprises: Figure 8

[0223] First, the project file is obtained, wherein the project file includes different formats (such as Word, PDF, Image, etc.), and the project file can include construction background, file basis, fund composition, construction period, construction content, architecture design, data resource, component condition, project budget, etc.

[0224] Then, the project file is parsed to obtain the vectorized representation of the project file, and the vectorized representation is stored in the local vector database. The project file can be parsed by python-docx, pdf2docx, pdfplumber, PyMuPDF, PaddleOCR, etc., and the parsed data can be cleaned, such as invalid header page number, watermark, etc. in the document, which can be cleaned by OpenCV, regular expression, etc.

[0225] At the same time, the scoring file is processed, Figure 9 is the flowchart of the scoring file processing method provided by the present application, as shown in the figure, Figure 9 determine whether each score item in the scoring file is quantifiable, the quantifiable score item is the objective score item, the corresponding calculation formula is extracted, and the objective score item and the corresponding calculation formula are stored in the local knowledge base. The unquantifiable score item is the subjective score item, the description text of the subjective score item is input into the large language model, the scoring logic of the subjective score item is extracted by the large language model, and the subjective score item and the scoring logic are stored in the local knowledge block.

[0226] ​In addition, other files are also stored in the local knowledge base, including expert experience files, historical project files, etc. The content in the local knowledge base is converted into structured vectors through multi-modal encoding technology and stored in the local vector database. For example, text generates high-dimensional semantic vectors through the BERT model, tables generate mixed vectors through the fusion of table header semantics (TextCNN), row and column relationships (GAT network), and numerical features, etc., and pictures output high-dimensional feature vectors through the CLIP model. At the same time, cross-modal indexes of text description and corresponding tables, pictures, etc. are established through the Attention mechanism. The local vector database configuration can adopt Milvus, and private deployment is realized through KubeSphere.

[0227] Based on the information in the local vector database, a prompt text is constructed, and the prompt text is input into a large language model, and the final project score is output by the large language model. Among them, the subjective scoring items are divided into mandatory scoring items (M, provided with m items) and non-mandatory scoring items (N, provided with n items), if the score of the mandatory scoring item is 0, the project score is directly output as “not passed”, and the corresponding mandatory scoring item is displayed, if the score of the mandatory scoring item is not 0, the scores of the remaining scoring items are combined to determine the project score.

[0228] The project scoring device provided by the present application is described below, and the project scoring device described below can be correspondingly referred to the project scoring method described above.

[0229] Based on any of the above embodiments, Figure 10 is a structural schematic diagram of the project scoring device provided by the present application, as Figure 10 shown, the device comprises:

[0230] The first determination unit 1010 is configured to determine the calculation formula of the objective scoring item and the scoring logic of the subjective scoring item in the scoring file.

[0231] The second determination unit 1020 is configured to extract the assignment of each parameter in the calculation formula from the project file, and determine the objective score of the objective scoring item based on the assignment of each parameter.

[0232] The third determination unit 1030 is configured to extract the associated elements from the project file, and determine the subjective score of the subjective scoring item based on the associated elements and the scoring logic, the associated elements refer to the elements with a semantic correlation greater than a threshold with the subjective scoring item.

[0233] The project scoring unit 1040 is configured to determine the project score based on the objective score and the subjective score.

[0234] Figure 11 is a structural schematic diagram of the electronic device provided by the present application, as Figure 11As shown, the electronic device can include a processor 1110, a communications interface 1120, a memory 1130, and a communications bus 1140, wherein the processor 1110, the communications interface 1120, and the memory 1130 complete communications with each other through the communications bus 1140. The processor 1110 can invoke a logical instruction in the memory 1130 to execute the item scoring method.

[0235] In addition, the logical instruction in the memory 1130 described above can be implemented in the form of a software functional unit and sold or used as an independent product, and can be stored in a computer-readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0236] On the other hand, the present application also provides a computer program product, which includes a computer program, the computer program can be stored on a non-transitory computer readable storage medium, and the computer program is executed by a processor, and the computer can execute the item scoring method provided by the above-mentioned methods.

[0237] In yet another aspect, the present application also provides a non-transitory computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the item scoring method provided by the above-mentioned methods.

[0238] The device embodiments described above are only schematic, wherein the units shown as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, i.e., they can be located in one place, or distributed on multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the present embodiment. Those skilled in the art can understand and implement it without creative labor.

[0239] Those skilled in the art can clearly understand the technical solutions of the various embodiments from the above description of the embodiments, and the various embodiments can be implemented by means of software with the necessary general hardware platforms, and of course, can also be implemented by hardware. Based on such understanding, the above technical solutions, essentially or in other words, the part of the prior art that makes a contribution, can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, and the like, and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0240] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, rather than limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for some technical features therein; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A project scoring method, characterized in that, include: Determine the calculation formulas for objective rating items and the scoring logic for subjective rating items in the rating document; Extract the values ​​of each parameter in the calculation formula from the project file, and determine the objective score of the objective scoring item based on the values ​​of each parameter. The associated elements are extracted from the project file, and the subjective score of the subjective score item is determined based on the associated elements and the scoring logic. The associated elements refer to elements whose semantic relevance to the subjective score item is greater than a threshold. The project score is determined based on the objective score and the subjective score. The step of extracting related elements from the project file includes: The project file is input into multiple expert models to obtain the initial related elements output by each expert model; based on the weights of each expert model, the initial related elements output by each expert model are fused to obtain the related elements; wherein, the weights of each expert model are dynamically adjusted according to the importance of each initial related element using an attention mechanism. Each expert model is trained based on the following steps: The process involves: acquiring sample project files and sample association elements related to subjective rating items within these files; perturbing the sample project files based on the weaknesses of each expert model to obtain adversarial sample project files, where the weaknesses of each expert model characterize key elements that are easily confused by each model; extracting a first association element from the sample project files and a second association element from the adversarial sample project files based on the initial models of each expert model; updating the parameters of each expert model based on the differences between the first association element and the sample association elements, and the differences between the second association element and the sample association elements; wherein, the adversarial sample project files refer to samples that, through minor modifications to the original project files, cause misjudgments or biases in the expert models, and perturbing the sample project files refers to making minor modifications to the text content while maintaining the semantics of the sample project files.

2. The project scoring method according to claim 1, characterized in that, The extraction of related elements from the project file includes: Based on the project file and the subjective rating items, construct the prompt text; Based on the feature extraction model, the prompt text is used to extract related features from the project file.

3. The project scoring method according to claim 2, characterized in that, The step of constructing prompt text based on the project file and the subjective rating items includes: Based on the type of the project file, multiple candidate historical project files were identified; Use the candidate historical project file with the highest project scoring accuracy as the target project file; Using the target prompt text of the target project file as a template, the prompt text is constructed based on the project file and the subjective rating item.

4. The project scoring method according to claim 3, characterized in that, The step of constructing the prompt text based on the target project file and the subjective rating item, using the target prompt text as a template, includes: Based on the subjective rating items, target keywords are extracted from the project documents; The target project file is replaced with the target keyword in the target prompt text to obtain the prompt text.

5. The project scoring method according to claim 2, characterized in that, The feature extraction model, using the prompt text, extracts related features from the project file, including: Based on the aforementioned feature extraction model and the aforementioned prompt text, multiple initial related features are extracted from the project file. Receive the question text of the multiple initial associated elements; Based on the aforementioned element extraction model, the related elements are obtained by filtering from the multiple initial related elements using the question text and historical project information.

6. The project scoring method according to any one of claims 1 to 5, characterized in that, The scoring logic is determined based on the following steps: Extract descriptive text from the rating file, the descriptive text being used to describe the rating requirements of the subjective rating item; The scoring logic is determined based on the keywords in the descriptive text and the logical relationships between the keywords.

7. The project scoring method according to claim 6, characterized in that, The step of extracting descriptive text from the rating file includes: If preset words exist in the scoring file, the initial description text is extracted from the scoring file based on the position information of the preset words in the scoring file; The description text is determined from the initial description text based on the semantic correlation between each word segment in the initial description text and the preset word.

8. The project scoring method according to any one of claims 1 to 5, characterized in that, The process of determining the project score based on the objective score and the subjective score includes: If the subjective rating item includes the target rating item, determine the subjective rating of the target rating item; If the subjective score of the target rating item is less than the threshold, the subjective score of the target rating item shall be used as the project score. If the subjective score of the target rating item is greater than or equal to the threshold, the item score is determined based on the objective score and the subjective score.

9. The project scoring method according to claim 8, characterized in that, Determining the project score based on the objective score and the subjective score includes: The project score is determined by integrating the objective score and the subjective score based on the weights of the scoring items, wherein the weights of the scoring items are determined based on the importance of the subjective score items and the objective score items.

10. A project scoring device, characterized in that, include: The first determining unit is used to determine the calculation formula of the objective scoring items and the scoring logic of the subjective scoring items in the scoring file; The second determining unit is used to extract the values ​​of each parameter in the calculation formula from the project file, and determine the objective score of the objective scoring item based on the values ​​of each parameter. The third determining unit is used to extract related elements from the project file and determine the subjective score of the subjective scoring item based on the related elements and the scoring logic. The related elements refer to elements whose semantic relevance to the subjective scoring item is greater than a threshold. The project scoring unit is used to determine the project score based on the objective score and the subjective score. The step of extracting related elements from the project file includes: The project file is input into multiple expert models to obtain the initial related elements output by each expert model; based on the weights of each expert model, the initial related elements output by each expert model are fused to obtain the related elements; wherein, the weights of each expert model are dynamically adjusted according to the importance of each initial related element using an attention mechanism. Each expert model is trained based on the following steps: The process involves: acquiring sample project files and sample association elements related to subjective rating items within these files; perturbing the sample project files based on the weaknesses of each expert model to obtain adversarial sample project files, where the weaknesses of each expert model characterize key elements that are easily confused by each model; extracting a first association element from the sample project files and a second association element from the adversarial sample project files based on the initial models of each expert model; updating the parameters of each expert model based on the differences between the first association element and the sample association elements, and the differences between the second association element and the sample association elements; wherein, the adversarial sample project files refer to samples that, through minor modifications to the original project files, cause misjudgments or biases in the expert models, and perturbing the sample project files refers to making minor modifications to the text content while maintaining the semantics of the sample project files.

11. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the item scoring method as described in any one of claims 1 to 9.

12. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the item scoring method as described in any one of claims 1 to 9.

Images