Generation system of intelligent scoring model

Abstract

The embodiment of the invention relates to the technical field of artificial intelligence, in particular to a generation system of an intelligent scoring model, and the generation system of the intelligent scoring model provided by the embodiment of the invention identifies and classifies the professional field of the intelligent scoring model by using a configuration management module, and manages and stores the configuration information of the intelligent scoring model. And calling the large language model by using a model generation module to generate original data about the intelligent scoring model, processing the original data by using a data processing module, outputting tree structure data with an association relationship, and finally obtaining the intelligent scoring model. The professional field and standardized intelligent scoring model can be quickly generated by using the large language model, time and labor are saved, and the accuracy of the intelligent scoring model in professional field knowledge is enhanced.

Description

TECHNICAL FIELD

[0001] The embodiment of the present application relates to the technical field of artificial intelligence, and particularly relates to a generation system of an intelligent scoring model. BACKGROUND

[0002] With the development of Internet technology and artificial intelligence, artificial intelligence models serve more and more technical personnel in various fields to assist people in quickly completing work and tasks. In the related art, in the generation process of an intelligent scoring model, there are problems such as time-consuming and labor-consuming, lack of standardization, strong subjectivity, and inaccurate domain knowledge in artificially designing a scoring model. SUMMARY

[0003] Therefore, one of the purposes of the embodiment of the present application is to provide a generation system of an intelligent scoring model, aiming to improve the problems such as time-consuming and labor-consuming, lack of standardization, strong subjectivity, and inaccurate domain knowledge in the generation process of an intelligent scoring model in the related art.

[0004] To solve the above technical problems, the embodiment of the present application provides the following technical solutions. In a first aspect, the embodiment of the present application provides a generation system of an intelligent scoring model, the generation system of the intelligent scoring model being in communication connection with a large language model, and the generation system of the intelligent scoring model comprising: a configuration management module configured to, in response to an input of a user, identify and classify a professional field of a required intelligent scoring model based on a natural language processing technology, and manage and store configuration information of the intelligent scoring model; a model generation module connected with the configuration management module and configured to call the large language model based on the configuration information by using a preset format template, so as to generate original data about the intelligent scoring model by using the large language model; a data processing module connected with the model generation module and configured to process the original data, so as to output tree structure data with an association relationship, wherein the tree structure data with the association relationship is the intelligent scoring model, and the intelligent scoring model serves as a reference standard for the user to score a to-be-scored object in the professional field.

[0005] In some embodiments, the configuration management module comprises an identification and classification unit, a management unit, and a storage unit, and the management unit is connected with the identification and classification unit and the storage unit respectively; the identification and classification unit is configured to, in response to an input of a user, identify and classify a target professional field of a required intelligent scoring model based on a natural language processing technology; the management unit is configured to, based on the target professional field, call target configuration information corresponding to the target professional field, and configure model information of the intelligent scoring model as the target configuration information; The storage unit is configured to store target configuration information of the intelligent scoring model.

[0006] In some embodiments, the model generation module comprises a prompt word construction unit and a model calling unit, the prompt word construction unit being connected to the model calling unit. The prompt word construction unit is configured to obtain a preset format module, a user input parameter, and target configuration information, generate a target prompt word based on the format module, the user input parameter, and the target configuration information, and use the target prompt word to prompt a large language model to output raw data about the intelligent scoring model. The model calling unit is configured to call the large language model based on the target prompt word to generate the raw data about the intelligent scoring model.

[0007] In some embodiments, the model generation module further comprises a multi-level template processing unit, the multi-level template processing unit being connected to the model calling unit. The multi-level template processing unit is configured to obtain the raw data about the intelligent scoring model, perform multi-level layout on the raw data based on a predefined hierarchical structure template, and output the raw data after the multi-level layout.

[0008] In some embodiments, the raw data is data after the multi-level layout, the data processing module comprises a data parsing unit, a hierarchical structure conversion unit, and an identifier allocation unit, the hierarchical structure conversion unit being connected to the data parsing unit and the identifier allocation unit. The data parsing unit is configured to obtain the raw data after the multi-level layout, and perform cleaning and formatting processing on the raw data after the multi-level layout to obtain reference data. The hierarchical structure conversion unit is configured to obtain the reference data, and convert the reference data into tree structure data having an association relationship. The identifier allocation unit is configured to obtain the tree structure data, and allocate a unique reference identifier to each data node in the tree structure data based on the association relationship.

[0009] In some embodiments, the intelligent scoring model generation system further comprises a state tracking module, the state tracking module being connected to the configuration management module, the model generation module, and the data processing module. The state tracking module comprises a generation record unit, the generation record unit being configured to obtain working states of the configuration management module, the model generation module, and the data processing module in the process of generating the intelligent scoring model, determine a generation state of the intelligent scoring model based on the working states, and store and manage the generation state of the intelligent scoring model.

[0010] In some embodiments, the state tracking module further comprises a performance monitoring unit connected with the record generating unit, the performance monitoring unit is configured to acquire performance index information of the configuration management module, the model generating module and the data processing module in the process of generating the intelligent scoring model, and store and manage the performance index information, the performance index information comprising generation time, error output information and resource consumption information.

[0011] In some embodiments, the intelligent scoring model generation system further comprises a knowledge enhancement module connected with the data processing module. The knowledge enhancement module is configured to construct a structured knowledge database based on expert knowledge of the professional field input by the user, and store and manage the structured knowledge database. The knowledge enhancement module is further configured to acquire tree structure data with an association relationship, and supplement and optimize data of data nodes in the tree structure data based on the structured knowledge database, to obtain target data with an association relationship.

[0012] In some embodiments, the pre-set format template is a JSON format template.

[0013] In some embodiments, the association relationship is a parent-child association relationship.

[0014] The embodiments of the present application have the following beneficial effects: Unlike the prior art, the intelligent scoring model generation system provided by the embodiments of the present application is in communication connection with a large language model, comprising a configuration management module, a model generating module and a data processing module, wherein the configuration management module is configured to respond to the input of the user, identify and classify the professional field of the intelligent scoring model to be generated based on natural language processing technology, and manage and store the configuration information of the intelligent scoring model, the model generating module is connected with the configuration management module and is configured to call the large language model based on the configuration information using a pre-set format template, to generate original data about the intelligent scoring model using the large language model, the data processing module is connected with the model generating module and is configured to process the original data to output tree structure data with an association relationship, the tree structure data with an association relationship being the intelligent scoring model, the intelligent scoring model serving as a reference standard for the user to score the to-be-scored object in the professional field.

[0015] The intelligent scoring model generation system provided in this embodiment of the invention utilizes a configuration management module to identify and classify the professional fields of the intelligent scoring model, manage and store the configuration information of the intelligent scoring model, and uses a model generation module to call a large language model to generate raw data about the intelligent scoring model. The data processing module processes the raw data and outputs tree-structured data with correlation relationships. Finally, the intelligent scoring model is obtained. In this way, a professional and standardized intelligent scoring model can be quickly generated using a large language model, saving time and effort and enhancing the accuracy of the intelligent scoring model in professional field knowledge. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the prior art or embodiments will be briefly introduced below. Obviously, the drawings described below only show some embodiments of the present invention and should not be considered as limiting the scope of protection. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1a This is a schematic diagram illustrating the application scenario of the intelligent scoring model generation system in some embodiments of the present invention; Figure 1b This is a schematic diagram illustrating an application scenario of the intelligent scoring model generation system provided in other embodiments of the present invention; Figure 2a This is a schematic diagram of the structure of the intelligent scoring model generation system in some embodiments of the present invention; Figure 2b This is a schematic diagram of the tree structure data of the intelligent scoring model in some embodiments of the present invention; Figure 3 This is a schematic diagram of the configuration management module in some embodiments of the present invention; Figure 4a This is a schematic diagram of the structure of the model generation module in some embodiments of the present invention; Figure 4b This is a schematic diagram of the structure of the model generation module in some other embodiments of the present invention; Figure 5 These are schematic diagrams of the data processing module in some embodiments of the present invention; Figure 6 This is a schematic diagram of the structure of the intelligent scoring model generation system in some other embodiments of the present invention; Figure 7 yes Figure 6 A schematic diagram of the state tracking module in the intelligent scoring model generation system shown in the embodiment; Figure 8 This is a schematic diagram of the structure of the intelligent scoring model generation system in some embodiments of the present invention.

[0018] Explanation of reference numerals in the attached figures: 100. Intelligent scoring model generation system; 110. Configuration Management Module; 111. Identification and Classification Unit; 112. Management Unit; 113. Storage Unit; 120. Model Generation Module; 121. Prompt Word Construction Unit; 122. Model Calling Unit; 123. Multi-level Template Processing Unit; 130. Data processing module; 131. Data parsing unit; 132. Hierarchical structure conversion unit; 133. Identifier allocation unit; 140. Status tracking module; 141. Record generation unit; 142. Performance monitoring unit; 150. Knowledge Enhancement Module. Detailed Implementation

[0019] To make the objectives and advantages of the embodiments of the present invention more readily understood, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. The detailed description of the embodiments of the present invention in the accompanying drawings is not intended to limit the scope of protection claimed by the present invention, but only to illustrate selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] It should be noted that, unless there is a conflict, the various technical features involved in the embodiments of the present invention described below can be combined with each other, and all are within the protection scope of the present invention. Furthermore, although functional modules are divided in the device or structural schematic diagram and a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than the module division in the device or the order in the flowchart. In addition, the terms "first," "second," "third," and other similar expressions used herein do not limit the data or execution order, but are only for illustrative purposes and to distinguish identical or similar items with substantially the same function and effect, and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features.

[0021] Unless otherwise defined, the technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention. It should be understood that the term "and / or" as used herein includes any and all combinations of one or more of the listed items.

[0022] Please see Figure 1a , Figure 1a The illustration shows a schematic diagram of an application scenario of the intelligent scoring model generation system in some embodiments of the present invention.

[0023] Specifically, such as Figure 1a As shown, this application scenario includes electronic device 200, server 300, and mobile terminal 400, wherein electronic device 200 communicates with server 300 and mobile terminal 400 via a network. Clearly, examples of networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0024] For example, electronic device 200 is configured with intelligent scoring model generation system 100, and server 300 is configured with at least one large language model, for example, see Figure 1a As shown, server 300 is configured with three large language models: large language model A1, large language model A2, and large language model A3. After electronic device 200 establishes a communication connection with server 300 via a network, the intelligent scoring model generation system 100 in electronic device 200 communicates with the large language models in server 300. The intelligent scoring model generation system 100 can call the large language models to generate data about the intelligent scoring model, for example... Figure 1a As shown, the intelligent scoring model generation system 100, based on the user's input of prompts regarding the intelligent scoring model, invokes large language models A1, A2, or A3 to generate data about the intelligent scoring model. The intelligent scoring model generation system 100 processes the data generated by the large language models and outputs a tree-structured data with relationships, thus obtaining the intelligent scoring model required by the user.

[0025] In some embodiments, when a user needs to generate an intelligent scoring model for a specific professional field, a mobile terminal 400 (e.g., a laptop 410, tablet 420, or smartphone 430) connects to the electronic device 200 via a network. The user inputs relevant prompts about the intelligent scoring model to be generated on the mobile terminal 400, such as the professional field, model purpose, and applicable scope. The mobile terminal 400 transmits the prompts to the intelligent scoring model generation system 100 configured on the electronic device 200 via the network. The intelligent scoring model generation system 100 processes the prompts, identifies and classifies the professional field of the intelligent scoring model to be generated, sets and manages the configuration information of the intelligent scoring model, and uses a pre-defined format template to call a large language model based on the configuration information of the intelligent scoring model. The system then uses the large language model to generate data about the intelligent scoring model. For example, the intelligent scoring model generation system 100 calls a large language model A1 based on the configuration information of the intelligent scoring model and uses the large language model A1 to generate data about the intelligent scoring model.

[0026] After the large language model generates data about the intelligent scoring model, the intelligent scoring model generation system 100 processes the data about the intelligent scoring model and finally obtains the intelligent scoring model required by the user.

[0027] It should be understood that Figure 1a This is merely an illustrative representation of the prompts for the user to input the intelligent scoring model to be generated on the mobile terminal 400. The intelligent scoring model generation system 100 in the electronic device 200 processes the prompts to call the large language model to generate data about the intelligent scoring model, and processes the data about the intelligent scoring model to finally obtain the intelligent scoring model. Figure 1a The illustrated electronic device 200 is a desktop computer, and the mobile terminal 400 includes a laptop 410, a tablet 420, and a smartphone 430, as well as a large language model configured on the server 300. This does not limit the structure, type, or quantity of the electronic device 200, server 300, and mobile terminal 400 in other application scenarios or embodiments. For example, in some other application scenarios or embodiments, the electronic device 200 can also be a server, a laptop, or other suitable type of device. Users can also directly operate the electronic device 200 to input prompts related to the intelligent scoring model to be generated, and can also directly configure the large language model on the electronic device 200. Alternatively, please refer to... Figure 1bAlternatively, the intelligent scoring model generation system 100 can be directly configured on the server 300. The mobile terminal 400 communicates with the server 300 via the network. After the user inputs the prompt information related to the intelligent scoring model to be generated on the mobile terminal 400, the intelligent scoring model generation system 100 on the server 300 processes the prompt information input by the user and finally generates the intelligent scoring model.

[0028] Please see Figure 2a , Figure 2a A schematic diagram of the structure of the intelligent scoring model generation system in some embodiments of the present invention is shown.

[0029] like Figure 2a As shown, the intelligent scoring model generation system 100 includes a configuration management module 110, a model generation module 120, and a data processing module 130. The model generation module 120 is communicatively connected to the configuration management module 110 and the data processing module 130, respectively.

[0030] Specifically, the configuration management module 110 is configured to respond to user input, identify and classify the professional domain of the intelligent scoring model to be generated based on natural language processing (NLP) technology, and manage and store the configuration information of the intelligent scoring model. That is, the configuration management module 110 receives the user's requirement description for the intelligent scoring model through a natural language interaction interface. This interface supports multiple input formats such as text and voice, and uses NLP technology for semantic parsing to accurately extract the user's core requirements for the intelligent scoring model, including but not limited to key information such as the type of scoring object, scoring scenario, accuracy requirements, and model purpose. Then, based on a pre-trained domain classification model, the configuration management module 110 performs domain matching on the parsed user requirements. This domain classification model covers multiple fields such as health, education, medical care, and catering, automatically determining the professional category to which the intelligent scoring model belongs, and loading the corresponding professional vocabulary and rule base according to domain characteristics to determine the professional domain of the intelligent scoring model. Next, the configuration management module 110 constructs a structured configuration template. This template includes basic parameters of the intelligent scoring model (such as the number of scoring dimensions, weight allocation method, and scoring level division) and domain-specific parameters (such as knowledge point weights in education and risk factors in health). It provides a visual configuration interface, allowing users to manually adjust parameters, supports configuration version control, and records the history of each configuration change. The configuration management module 110 encrypts the configuration information of the intelligent scoring model and stores it in a distributed database, establishing an indexing mechanism for fast querying. It also provides standardized interfaces for other modules to access the configuration information data and supports the export and import of configuration information, thereby enabling the management and storage of the intelligent scoring model's configuration information.

[0031] For example, the model generation module 120 is configured to call a large language model based on the configuration information of the intelligent scoring model using a preset format template, so as to generate raw data about the intelligent scoring model using the large language model. In some embodiments, the preset format template is a JSON format template. The model generation module 120 maintains multiple preset format template libraries, providing dedicated format templates for different professional fields and scoring scenarios (e.g., percentage-based assessment templates, healthy catering rating templates). The templates include elements such as scoring indicator frameworks, logical relationship definitions, and output format specifications. Based on the parameters transmitted by the configuration management module 110, the model generation module 120 automatically generates prompts that meet the input requirements of the large language model. The prompts include domain information, scoring objectives, output format constraints, etc. The model generation module 120 establishes a large language model call adaptation layer, supporting integration with large language models from different vendors (such as the GPT series and Tongyi Qianwen), enabling dynamic adjustment of call parameters. The model generation module 120 receives the raw output returned by the large language model, initially filters out invalid information (such as duplicate content, format errors, etc.), retains key data such as core scoring indicators, weight suggestions, and scoring logic, and forms the raw data document.

[0032] The data processing module 130 is configured to process the raw data to output a tree-structured data with relationships. It should be understood that this tree-structured data with relationships constitutes the intelligent scoring model, which serves as a reference standard for users to score objects within their professional field. The data processing module 130 performs structured processing on the raw data, including removing redundant information, standardizing indicator naming conventions, and correcting logical inconsistencies (such as the sum of indicator weights exceeding a certain threshold). The data processing module 130 converts unstructured text into parsable intermediate data formats (such as JSON and XML) based on domain knowledge graphs. It identifies hierarchical relationships (e.g., primary and secondary indicators) and causal relationships (e.g., the impact path of an abnormal indicator on the total score) between various indicators, and uses graph algorithms to construct a relational network to quantify indicator weights. Combining suggestions from a large language model with user configuration, it generates a reasonable weight allocation scheme using algorithms such as the Analytic Hierarchy Process (AHP). The processed indicator data is organized into a multi-branch tree structure according to the relationships. Each node contains attributes such as indicator name, weight value, scoring criteria, and data source description, and supports CRUD operations on nodes. The tree structure data is validated using preset rules (e.g., indicator coverage completeness, logical consistency, etc.), and a validation report is output for user review. The data processing module 130 provides multiple output formats (such as a visualized scoring system graph, an executable scoring rule engine script, and an Excel spreadsheet) to output an intelligent scoring model. This intelligent scoring model can be directly connected to a scoring system or used directly for scoring reference, supporting automatic or semi-automatic scoring of the objects to be scored.

[0033] For example, the generated intelligent scoring model 500 is shown in Table 1 below: Table 1:

[0034] For example, after generating the intelligent scoring model 500 for goose meat cooking in the catering industry, as shown in Table 1 above, the intelligent scoring model 500 serves as a reference standard for goose meat cooking in the catering industry. Users can use the intelligent scoring model 500 to give an overall score to the cooking ingredients, cooking process, and finished product of goose meat cooking provided by themselves or others, thus obtaining a goose meat cooking score. It is understood that the intelligent scoring model 500 shown in Table 1 is merely illustrative and does not impose any limitations on other intelligent scoring models. Those skilled in the art can also use the intelligent scoring model generation system 100 provided in this embodiment of the invention to generate any suitable intelligent scoring model.

[0035] Understandably, the intelligent scoring model 500 for goose meat cooking in the catering industry, as shown in Table 1, includes five scoring indicators: ingredient preparation, heat control, flavor and seasoning, appearance and presentation, and finished product effect. Each scoring indicator is associated with two sub-indicators (e.g., ingredient preparation is associated with goose meat processing and ingredient pairing). Each sub-indicator has corresponding attribute information, including score, type, and reference standard. The attribute information of the sub-indicators, the sub-indicators themselves, and the scoring indicators to which the sub-indicators belong form a tree-like data structure with relationships. All the tree-like data with relationships among the scoring indicators constitute the intelligent scoring model. For example, please refer to...Figure 2b , Figure 2b The illustration schematically shows a portion of the tree structure data in the intelligent scoring model provided by some embodiments of the present invention. Figure 2b The attribute information of the indicator sub-items is not shown.

[0036] In some embodiments, the association relationship is a parent-child association relationship, that is, a parent class data in the tree structure data is associated with one or more child class data to form a parent-child association relationship, resulting in a tree structure data of the association relationship. It should be understood that a data item can be both a parent class data item of a child class data item and a child class data item of a parent class data item. For example, please refer to [link to relevant documentation]. Figure 2b The intelligent scoring model 500 is the parent data. The sub-data associated with the intelligent scoring model 500 includes ingredient preparation, cooking temperature control, flavoring, appearance and finished product effect. The sub-data associated with ingredient preparation includes goose meat processing and ingredient matching. Obviously, the intelligent scoring model 500 is only the parent data of the sub-data. Ingredient preparation is both the parent data of the sub-data (i.e. goose meat processing and ingredient matching) and the sub-data of the parent data (i.e. intelligent scoring model 500).

[0037] The intelligent scoring model generation system provided in this invention utilizes a configuration management module to identify and classify the professional fields of intelligent scoring models, manage and store the configuration information of intelligent scoring models, uses a model generation module to call a large language model to generate raw data about intelligent scoring models, and uses a data processing module to process the raw data, outputting tree-structured data with correlation relationships, and finally obtaining the intelligent scoring model. In this way, it can quickly generate professional and standardized intelligent scoring models using a large language model, saving time and effort and enhancing the accuracy of intelligent scoring models in professional field knowledge.

[0038] Please see Figure 3 In some embodiments, the configuration management module 110 includes an identification and classification unit 111, a management unit 112, and a storage unit 113, with the management unit 112 being communicatively connected to the identification and classification unit 111 and the storage unit 113, respectively.

[0039] Specifically, the identification and classification unit 111 is configured to respond to user input and identify and classify the target professional field of the intelligent scoring model to be generated based on natural language processing technology.

[0040] The identification and classification unit 111 is the "front-end perception center" of the configuration management module 110. Its core function is to transform the user's input natural language requirements into accurate professional domain labels, providing a foundation for subsequent configuration information retrieval. The identification and classification unit 111 supports multiple input formats, including text and voice (after text-to-text processing). It uses Natural Language Processing (NLP) technology to clean the input content (removing redundant words and correcting typos) and perform semantic analysis to extract core requirement elements (e.g., from "I need a scoring standard for a goose cooking competition," extracting key information such as "goose cooking" and "competition scoring"). The identification and classification unit 111 constructs a domain intent recognition model, distinguishing between basic user needs (e.g., "generate a new model") and advanced needs (e.g., "modify the scoring dimensions of an existing model"), providing a basis for determining subsequent processing paths.

[0041] The identification and classification unit 111, based on a pre-trained domain classification model (e.g., a BERT-tuned multi-label classification model), performs domain matching on the parsed user needs, covering primary domains such as education, catering, finance, and healthcare, and further subdividing them into secondary sub-domains (e.g., "Chinese cooking" and "Western baking" under the catering domain, and "essay scoring" and "experimental operation assessment" under the education domain). The identification and classification unit 111 introduces a domain confidence mechanism; when the confidence of the identification result falls below a threshold (e.g., ...), the domain confidence level is adjusted accordingly. When a second confirmation process is triggered (e.g., "Do you need to generate a scoring model for the 'Chinese cuisine cooking' domain? Optional domains: ① Chinese cuisine cooking ② Western cuisine cooking"), it avoids domain misjudgment.

[0042] For the target professional field of the identified intelligent scoring model, the identification and classification unit 111 automatically extracts core features (such as features in the "goose meat cooking" field, including "ingredient processing," "heat control," and "seasoning"), and generates standardized domain labels (such as "catering - Chinese cuisine - poultry cooking - scoring model"), which serve as index keys for subsequent retrieval of configuration information. The identification and classification unit 111 establishes a domain feature library and dynamically updates emerging elements in each domain (such as adding the feature "healthy nutrition ratio" to the catering field) to ensure the timeliness of identification and classification.

[0043] The management unit 112 is configured to be a target professional field based on the intelligent scoring model, retrieve the target configuration information corresponding to the target professional field, and configure the model information of the intelligent scoring model as the target configuration information.

[0044] Specifically, the management unit 112 is the "core scheduling center" of the configuration management module 110, responsible for retrieving, adapting, and managing configuration information based on the identified target professional field, ensuring that the configuration of the intelligent scoring model conforms to the characteristics of the field. Based on the field tags output by the identification and classification unit 111, the management unit 112 retrieves the corresponding basic configuration template from the storage unit 113 (for example, the basic template for the "Chinese cuisine" field includes a framework of primary indicators such as "ingredient preparation," "flavoring," and "finished product effect," as well as default score allocation rules). Combined with personalized parameters input by the user (e.g., "total score 100 points, 'taste and flavor' weight should be emphasized"), the basic template is dynamically adjusted, with indicators added or deleted: for example, if the user requests the addition of the "innovation and creativity" dimension, this item is added to the primary indicators; weight adaptation: if the weight of "taste and flavor" needs to be increased, the score is adjusted from other scoring indicators (e.g., 5 points are transferred from "appearance and styling" to "taste and flavor"); rule customization: for example, if the user specifies "goose meat without removing the fishy smell will be deducted 10 points," this rule is embedded into the scoring criteria of the corresponding indicator.

[0045] The management unit 112 establishes a configuration verification rule base to perform logical verification on the adjusted target configuration information. This includes: completeness verification (ensuring no missing indicators at any level, such as "cooking score" needing to include core dimensions like ingredient preparation, flavor and seasoning, and finished product effect); consistency verification (ensuring the total score matches the set total score, such as the total score of all indicators in a 100-point model needing to be 100); and rationality verification (avoiding imbalanced indicator weights, such as the percentage of a single secondary indicator's score not exceeding that of its primary indicator). For configuration information that fails verification, optimization suggestions are generated (e.g., "The score of 'Dish Color' is 15 points, which exceeds 60% of the total score of 'Appearance' of 20 points. It is recommended to adjust it to 12 points or 10 points"), and users can manually confirm or automatically correct it.

[0046] Management unit 112 generates a unique version number for each configuration information, records the time, modifier, and changes for each modification (e.g., "Adjustment to the scoring criteria / reference criteria for 'ingredient pairing' on August 1, 2025"), and supports version rollback (e.g., restoring to the configuration state 3 days ago). Management unit 112 establishes a configuration information reuse mechanism, marking frequently used domain configurations (e.g., "monthly cooking competition standards") as "common templates" or "high-frequency templates," which users can directly reuse and quickly fine-tune the next time they call them, reducing the cost of repetitive configuration.

[0047] Specifically, storage unit 113 is configured to store target configuration information for the intelligent scoring model.

[0048] Storage unit 113 serves as the "data storage hub" of configuration management module 110, responsible for securely and efficiently storing and managing the configuration information of all intelligent scoring models to provide data support for the entire system. Storage unit 113 employs a distributed database (e.g., MySQL cluster + MongoDB) to construct a hierarchical storage structure: a relational database stores structured configuration data (e.g., indicator ID, name, score, and parent indicator relationships), while a document-oriented database stores unstructured configuration content (e.g., detailed scoring criteria text, domain feature descriptions). An index table (domain tag → configuration ID → version number) is established to enable millisecond-level querying of configuration information.

[0049] Storage unit 113 encrypts the stored configuration information (e.g., field-level AES encryption) to prevent the leakage of sensitive data (e.g., scoring rules for internal enterprise assessments). Storage unit 113 employs role-based access control (RBAC): administrators can modify configurations for all domains, while ordinary users can only view and use configurations for authorized domains (e.g., users in the catering domain cannot access scoring configurations in the medical domain), and all data access logs are recorded (e.g., "User A queried the 'Goose Meat Cooking' configuration V2.1 at 10:00 AM on August 1, 2025"). Storage unit 113 implements a strategy of scheduled full backups (e.g., daily at midnight) + incremental backups (e.g., hourly), with backup data stored off-site to ensure rapid recovery in case of data loss. A configuration information lifecycle management mechanism is established: configurations unused for more than one year are marked as "dormant," and those unused for more than three years are automatically archived (retaining metadata, deleting detailed content), freeing up storage space while ensuring historical records are traceable.

[0050] Please see Figure 4a In some embodiments, the model generation module 120 includes a prompt word construction unit 121 and a model invocation unit 122, with the prompt word construction unit 121 and the model invocation unit 122 being communicatively connected.

[0051] For example, the prompt word construction unit 121 is configured to acquire a preset format module, user input parameters, and target configuration information, and generate target prompt words based on the format module, user input parameters, and target configuration information. The target prompt words are used to prompt the large language model to output raw data about the intelligent scoring model. The prompt word construction unit 121 acquires a preset format template (including a domain-specific output framework, indicator hierarchy structure template, etc.), user input parameters (such as scoring object characteristics, scoring scenario constraints, and accuracy requirements), and the target configuration information of the intelligent scoring model (domain classification results, weight allocation rules, scoring division standards, etc. synchronized from the configuration management module 110). The prompt word construction unit 121 performs structured parsing of the input data, extracts key elements (such as identifying core parameters like "text type," "number of scoring dimensions," and "misspelling deduction rules" in the "education domain - essay scoring" scenario), and verifies the completeness of the input data. If any data is missing, a completion mechanism is automatically triggered (e.g., calling the default configuration for completion or prompting the user to supplement). The prompt word construction unit 121 constructs a hierarchical prompt logic based on domain characteristics: The first layer is the "task definition layer", which clearly tells the core goal of the large language model (e.g., "generate a goose meat cooking scoring model suitable for Chinese chefs' skill assessment"). The second layer is the "constraint layer", which limits the output range (e.g., "the scoring indicators need to include dimensions such as taste, seasoning, and finished product effect, with a total score of 100 points"). The third layer is the "format specification layer", which specifies the output structure (e.g., "presented in a tree structure, each indicator needs to include three elements: name, score, and scoring criteria"). The fourth layer is the "professional adaptation layer", which injects domain knowledge (e.g., "it needs to reflect the exclusive evaluation dimensions of goose meat cooking such as "removing fishy smell" and "meat tenderness").

[0052] The prompt word construction unit 121 automatically adjusts the prompt word style based on the characteristics of different large language models (e.g., the GPT series emphasizes logical coherence, while Tongyi Qianwen emphasizes Chinese domain adaptability). For example, it adds "step-by-step reasoning" guidance for logic-based models and enhances the accuracy of professional terminology for Chinese models. It also supports prompt word version management, recording historical iterations and allowing for backtracking to the optimal prompt word template based on the generated results. The prompt word construction unit 121 combines the parsed elements according to the logical structure to generate natural language prompt words (i.e., target prompt words) that conform to the input specifications of large language models. The length of the prompt words is controlled within the model context window threshold (e.g., for a 4096-token model, the prompt word length does not exceed 3000 tokens). The validity of the target prompt words is verified through a rule engine (e.g., avoiding vague expressions and ensuring no conflicting constraints). If unqualified, it is automatically corrected (e.g., optimizing "the scoring criteria should be as detailed as possible" to "each scoring indicator should include 3-5 specific deduction rules").

[0053] Specifically, the model invocation unit 122 is configured to invoke a large language model based on the target prompt word to generate raw data about the intelligent scoring model. The model invocation unit 122 maintains a model adaptation library, with built-in API interface parameter specifications, authentication methods, and invocation limits (e.g., number of requests per minute and token limit) for mainstream large language models (such as GPT-4, Claude, ERNIE, etc.). Based on the characteristics of the target prompt word (such as length and domain specialization) and system load, the model invocation unit 122 automatically selects the optimal model (e.g., prioritizing models supporting 16k tokens for long prompt words, and prioritizing domestic large language models for Chinese domain tasks), and also allows users to manually specify the model. It automatically adjusts model parameters based on the generation task requirements. For example, for strong logic tasks like scoring criteria, a low temperature coefficient (e.g., 0.2-0.4) is set to ensure output stability; for generating innovative scoring dimensions, the temperature coefficient is increased (e.g., 0.6-0.8) to enhance diversity; and generation length constraints are set (e.g., "output content not exceeding 2000 characters") to avoid redundant information.

[0054] The model invocation unit 122 receives the raw data (usually unstructured text or semi-structured JSON) returned by the large language model, removes irrelevant information (such as redundant explanations generated by the model and formatting errors), and extracts the core content (such as indicator names, scores, and scoring criteria). It then converts the filtered raw data into a system intermediate format (such as standardized JSON), appends metadata (such as generation time, model used, and token consumption statistics), and transmits the converted raw data to the data processing module 130 via an interface.

[0055] Please see Figure 4b In some embodiments, the model generation module 120 further includes a multi-level template processing unit 123, which is communicatively connected to the model calling unit 122.

[0056] The multi-level template processing unit 123 is configured to acquire the raw data of the intelligent scoring model and perform multi-level formatting on the raw data based on a predefined hierarchical structure template, outputting the multi-level formatted raw data. The multi-level template processing unit 123 is a key intermediate link connecting the model calling unit 122 and the data processing module 130, transforming the unstructured raw data generated by the large language model into structured content that conforms to the preset hierarchical specifications, providing standardized input for subsequent tree structure construction.

[0057] In this embodiment of the invention, the multi-level template processing unit 123 maintains a multi-dimensional, multi-domain hierarchical structure template library. The templates are classified into three levels: "domain-scenario-granularity" (e.g., "catering domain-goose meat cooking score-refined level", "education domain-essay score-basic level"). Each template defines a fixed hierarchical framework (e.g., "first-level indicator → second-level indicator → scoring details → deduction standard"), field specifications for each hierarchical framework (e.g., indicator name, score range, data type), and layout format (e.g., indentation rules and numbering logic).

[0058] For example, after receiving the raw data transmitted by the model calling unit 122, the multi-level template processing unit 123 extracts domain tags (such as "cooking" and "education") and scene features (such as "skills assessment" and "work evaluation") from the raw data. It then uses a template matching algorithm (e.g., based on keyword matching and cosine similarity calculation) to select the optimal hierarchical template from the template library. If no matching template is found, a default general template (e.g., a three-level structure of "total item → sub-item → specific standard") is called. The multi-level template processing unit 123 performs deep analysis on the raw data output by the large language model (usually natural language text, semi-structured lists, or mixed-format content). Using regular expressions and NLP entity recognition technology, it extracts key information units, including hierarchical entities: identifying first-level indicators (e.g., "finished product effect"), second-level indicators (e.g., "taste and texture"), and other hierarchical tags; attribute information: extracting the score, scoring standard / reference standard, data type (e.g., "normal item", "deducted item"), and association rules (e.g., "if a certain item fails to meet the standard, the total score will be deducted"). Logical relationships: Identify the inclusion relationship between indicators (e.g., "taste" belongs to "finished product effect") and the parallel relationship (e.g., "taste" and "overall flavor" are at the same level).

[0059] For ambiguous information that appears during the parsing process (such as indicators without a clear level), the multi-level template processing unit 123 completes the information by using preset rules (such as defaulting to the lowest level) or initiates a secondary verification (such as prompting manual confirmation of whether "heat control" belongs to a sub-item under "finished product effect"). The multi-level template processing unit 123 performs hierarchical mapping and layout on the parsed structured data based on the matched hierarchical template. Hierarchical mapping: The extracted indicators are nested in sequence according to "Level 1 → Level 2 → Level 3", for example, "Finished product effect (20 points) → Taste (10 points) → Reference standard: ...". Format uniformity: The numbering rules of each level are unified (for example, Level 1 uses "I, II, III", Level 2 uses "1, 2, 3"), indentation distance (for example, Level 2 indicators are indented by 2 characters, Level 3 indicators are indented by 4 characters), and field display order (for example, "Indicator name [score]: Scoring standard"). Numerical verification: Ensure that the scores of each level are logically consistent (for example, the sum of the scores of Level 2 indicators equals the score of the corresponding Level 1 indicator, and the sum of all Level 1 indicators conforms to the total score setting). If there is a conflict (for example, the sum of the scores of the sub-items under a certain level indicator exceeds the limit), the error is automatically marked and a correction suggestion is prompted.

[0060] Specifically, the multi-level template processing unit 123 converts the raw data after multi-level layout into a system-compatible intermediate format (such as structured JSON, XML, or text with hierarchical tags), which includes complete hierarchical relationships, attribute information, and layout metadata (such as template ID and layout time), and transmits it to the data processing module 130 through a standardized interface.

[0061] The entire typesetting process is recorded, supporting version rollback (e.g., retaining "initial typesetting version" and "revised version"). A typesetting report is generated, including template matching results, data parsing success rate, and anomaly correction records. This report provides data support for system optimization (e.g., when the parsing success rate of a certain type of template is lower than...). (At that time, the template iteration mechanism is triggered).

[0062] For example, in some embodiments, the original data is data that has been formatted through multiple levels; please refer to [link to relevant documentation]. Figure 5 The data processing module 130 includes a data parsing unit 131, a hierarchical structure conversion unit 132, and an identifier allocation unit 133. The hierarchical structure conversion unit 132 is communicatively connected to the data parsing unit 131 and the identifier allocation unit 133, respectively.

[0063] The data parsing unit 131 is configured to acquire the original data after multi-level layout, and to clean and format the original data after multi-level layout to obtain reference data.

[0064] Among them, the data parsing unit 131 is the "data preprocessing center" of the data processing module 130, responsible for converting the original data after multi-level typesetting into standardized and clean reference data to provide high-quality input for subsequent structure conversion.

[0065] The data parsing unit 131 receives the original data after multi-level typesetting (usually text with hierarchical marks, structured JSON or XML format), and identifies the hierarchical symbols (such as "1.", "1.", "(1)"), indentation rules and field delimiters (such as "[score]", ":") in the typesetting through the parsing engine, and accurately extracts the core information of each level, basic attributes: index name (such as "finished product effect"), score (such as "20 points"), reference standard / grading standard (such as "deduction of 3-5 points for too hard texture"); hierarchical relationship clues: identify "parent-child relationship" and "sibling relationship" through the typesetting position (for example, the secondary index is on the next line of the primary index and indented); constraint conditions: such as "deduct until exhausted", "if this item does not meet the standard, the total score will be downgraded" and other rules.

[0066] The data parsing unit 131 performs multi-dimensional cleaning operations to eliminate redundant information: delete the auxiliary text left during typesetting (such as "Note: The following are the grading rules"), duplicate content (such as the same index appearing twice); format standardization: convert non-uniform expressions into the system's common format (such as converting "10 points full score", "total score 10" into "score: 10"; converting the "√ / ×" mark into "qualified / unqualified"); logical contradiction correction: identify and process obvious errors (such as "the score of a secondary index > the total score of its primary index", "conflict between 'deduct 5 points' and 'deduct 10 points' in the grading standard"), automatically correct the repairable errors (such as adjusting the sub-item scores proportionally), and mark and prompt for manual intervention for the irreparable errors.

[0067] The data parsing unit 131 uses a standardized data model (such as a JSON object containing fields such as "id temporary storage area", "name", "score", "parent identifier", "grading standard", "rule type", etc.) to integrate the cleaned information into structured reference data, and performs integrity verification (such as ensuring that each index must contain "name" and "score" fields) and format verification (such as the score must be a non-negative integer) on the reference data through the verification rule library. After passing the verification, it is output to the hierarchical structure conversion unit 132, and if it fails, it returns to the cleaning link for reprocessing.

[0068] The hierarchical structure conversion unit 132 is configured to obtain reference data and convert the reference data into tree-structured data with associated relationships.

[0069] The hierarchical structure conversion unit 132 is the "structure construction core" of the data processing module 130. It is responsible for converting the flat reference data into a tree structure with clear relationships, reflecting the hierarchical logic of the intelligent scoring model.

[0070] The hierarchical structure conversion unit 132 constructs a model of relationships between indicators based on hierarchical clues in the reference data (such as "temporary parent identifier value" and "formatting indentation level"). Parent-child relationships include: for example, "taste" (secondary level) belongs to "finished product effect" (primary level); parallel relationships include: for example, "taste" and "overall flavor" are both secondary indicators under "finished product effect"; and dependency relationships include: for example, "if an indicator fails to meet the standard, the associated indicator will automatically deduct 2 points" (the logical rules need to be extracted from the scoring criteria). For fuzzy relationships (such as indicators without a clearly defined parent), a domain knowledge graph is used to assist in the judgment (e.g., "pot temperature control" in the cooking domain is implicitly subordinate to "heat control").

[0071] Then, the hierarchical structure conversion unit 132 adopts a multi-branch tree data structure (each node contains a "list of child nodes" and a "pointer to the parent node") to construct a tree structure layer by layer according to the hierarchical relationship. The root node corresponds to the overall item of the intelligent scoring model (e.g., "goose meat cooking scoring system", with a score of 100 points); first-level nodes correspond to the major items of each core indicator (e.g., "taste and seasoning" and "finished product effect"); second-level and lower-level nodes correspond to the sub-items of the detailed indicators (e.g., "saltiness" and "aroma presentation"). It supports dynamic adjustment of the node hierarchy. For example, if the user requests that "aroma presentation" be upgraded from a second-level indicator to a first-level indicator, the tree structure is automatically reconstructed and the score proportion of the associated nodes is recalculated.

[0072] The hierarchical structure conversion unit 132 verifies the logical consistency and hierarchical integrity of the tree structure: ensuring no gaps from the root node to the leaf node (the finest granularity index) (e.g., a first-level index must contain at least one second-level index); score correlation: verifying that the sum of the scores of all child nodes equals the score of the parent node (e.g., "Flavor and Seasoning" has a total score of 20 points, under which "Saltiness" has 10 points + "Aroma Presentation" has 10 points = 20 points); rule compatibility: checking whether there are conflicts in the dependency rules between nodes (e.g., "If A fails to meet the standard, B will be penalized" and "If B fails to meet the standard, A will be penalized" form a circular dependency). For structures with abnormal verification, adjustment suggestions are generated (e.g., "It is recommended to adjust the score of 'Saltiness' from 12 points to 10 points so that the total score of all sub-items under 'Flavor and Seasoning' matches 20 points").

[0073] The identifier allocation unit 133 is configured to acquire tree structure data with relationships, and assign a unique reference identifier to each data node in the tree structure data based on the relationships.

[0074] The identifier allocation unit 133 is the "node identity management center" of the data processing module 130. It assigns a unique identifier to each node in the tree structure to achieve refined management of the intelligent scoring model and support for subsequent applications.

[0075] The identifier allocation unit 133 establishes standardized unique identifier (ID) generation rules, including hierarchical coding: using numbers or letters to represent node levels (e.g., "1" represents a first-level node, "1-1" represents the first second-level node under a first-level node); domain prefix: embedding domain tags (e.g., "CY-" represents the catering domain, "JY-" represents the education domain); version suffix: marking the model version (e.g., "V2.0" represents version 2); check digit: generating the last character through a hash algorithm to ensure the uniqueness of the identifier (e.g., "CY-1-1-V2.0-5"). The identifier allocation unit 133 maintains an identifier rule library, supporting dynamic adjustment of rules according to domain and model type (e.g., risk level coding needs to be added to a financial domain scoring model).

[0076] The identifier allocation unit 133 traverses each node in the tree structure (from the root node to the leaf node) and generates a unique identifier according to preset rules. The root node is assigned a fixed domain prefix + "ROOT" + version (e.g., "CY-ROOT-V1"). First-level nodes are identified by adding a hierarchy number to the root node identifier (e.g., "CY-1-V1", "CY-2-V1"). Child nodes are identified by appending a child item number to the parent node identifier (e.g., "CY-1-1-V1" represents the first second-level node under first-level node 1). When generating unique identifiers, it ensures that the identifier is bound to the node attribute (e.g., "1-1" in the identifier corresponds to "Goose Meat Processing" under "Ingredient Preparation"), so that nodes can be quickly located by referencing the identifier.

[0077] The identifier allocation unit 133 establishes an identifier association table to record the mapping relationship between the reference identifier of each node and the reference identifier of its parent / child nodes (e.g., the parent identifier of "CY-1-1-V1" is "CY-1-V1"). It also supports tracing the position of nodes in the tree structure through identifiers. The identifier allocation unit 133 synchronously updates identifiers for node addition, deletion, and modification operations in the tree structure. When a new node is added, a new identifier is generated according to rules; when a node is deleted, the identifier is marked as "invalid"; when the node level is modified, the identifier is regenerated and the association table is updated, ensuring that the identifier system is synchronized with the tree structure in real time.

[0078] Please see Figure 6 as well as Figure 7 In some embodiments, the intelligent scoring model generation system 100 further includes a state tracking module 140, which is communicatively connected to the configuration management module 110, the model generation module 120, and the data processing module 130.

[0079] The status tracking module 140 includes a generation record unit 141, which is configured to acquire the working status of the configuration management module 110, the model generation module 120 and the data processing module 130 during the generation of the intelligent scoring model, determine the generation status of the intelligent scoring model based on the working status, and store and manage the generation status of the intelligent scoring model.

[0080] The status tracking module 140 is the "full-process monitoring hub" of the intelligent scoring model generation system 100. By capturing the running status of each module in real time, it realizes the visual tracking and management of the entire life cycle of intelligent scoring model generation, ensuring that the generation process is traceable and anomalies can be located.

[0081] The status tracking module 140 establishes a real-time communication interface (e.g., a status push mechanism based on a message queue) with the configuration management module 110, model generation module 120, and data processing module 130. It collects the working status data of each module at a preset frequency (e.g., every 2 seconds), including: configuration management module 110: user input processing progress (e.g., "parsing in progress," "domain identification completed"), configuration information retrieval status (e.g., "template matching successful," "parameter verification failed"); model generation module 120: prompt word construction progress (e.g., "template integration in progress," "prompt word generation completed"), large language model call status (e.g., "model response in progress," "raw data returned successfully"), multi-level layout status (e.g., "template matching in progress," "layout error"); data processing module 130: data cleaning progress (e.g., "cleaning completed"). The statuses include: “Format standardization failed”, tree structure data conversion status (e.g., “Hierarchical relationship modeling in progress”, “Structure verification passed”), and identifier allocation status (e.g., “Identifier generation in progress”, “Allocation completed”).

[0082] The status tracking module 140 standardizes the collected status data, unifying the format (e.g., timestamp + module name + status code + description information) and filters out duplicate or invalid data (e.g., empty status when the module is not running). Based on the working status of each core module, the status tracking module 140 constructs a "status flowchart" for the intelligent scoring model, intuitively displaying the current stage (e.g., "configuration stage → model generation stage → data processing stage → completion") and specific progress (e.g., "data processing stage:..."). The status tracking module 140 establishes a status early warning mechanism, which marks abnormal states (such as "model call timeout" and "data verification failure") in real time (e.g., red alerts) and associates them with abnormal details (e.g., "large language model API response timed out, already retried 2 times"). It supports pushing alarm information to administrators through the system console or visual interface.

[0083] For example, the status tracking module 140 establishes inter-module status linkage rules, such as "Configuration management module 110 completes → triggers model generation module 120 to start", "Model generation module 120 encounters an error → suspends data processing module 130", ensuring consistent status among modules. It records the complete trajectory of status changes (e.g., "Configuration management module 110 enters 'Domain Identification' at 10:00 on August 1, 2025 → Enters 'Configuration Information Retrieval Successfully' at 10:01 on August 1, 2025"), and supports querying historical statuses by model ID, time range, etc., providing a basis for troubleshooting (e.g., when a model generation fails, it can trace back to the specific time point of "Data parsing unit 131 cleaning error").

[0084] The generation record unit 141 is the "data management core" of the state tracking module 140. It is responsible for the structured storage, systematic management and efficient querying of the generated state, and is a key component supporting the traceability of the intelligent scoring model generation process.

[0085] The generation record unit 141 establishes a standardized generation status system, classified into three levels: "stage - status type - result". Stage division: corresponding to the core process of the system (such as "configuration stage", "model generation stage", "data processing stage", "completion stage"); Status type: distinguishing between running status (such as "processing", "waiting"), result status (such as "success", "failure"), and abnormal status (such as "timeout", "format error"); Result coding: assigning a unique code to each status (for example, "C001" represents the configuration stage as successful, and "M002" represents the model generation stage as timed out), to facilitate computer identification and processing.

[0086] For each intelligent scoring model, the generation record unit 141 generates a unique "generation record ID," which serves as the core index for linking the entire process status. The generation record unit 141 employs a storage architecture combining a time-series database (such as InfluxDB) and a relational database: the time-series database stores real-time data on status changes (such as timestamps, status codes, and progress values), supporting high-concurrency writes and time-series queries; the relational database stores structured records (such as generation record ID, model name, domain, final status, and start / end time), and links them to user information (such as creator and operation logs).

[0087] The generation record unit 141 implements a real-time status update mechanism. When the status of a module changes (such as from "processing" to "success"), it automatically triggers a storage update and records the triggering conditions for the status change (such as "because the data parsing unit 131 returns 'cleaning complete', the status of the data processing stage is updated to 'structure conversion in progress'").

[0088] The generation record unit 141 provides multi-dimensional record management functions, including filtering and querying: it supports filtering by model name, domain, generation time, final status (such as "all failed records"), and combines pagination and sorting functions to quickly locate target records; export and statistics: it supports exporting generated records to Excel / CSV format and generating statistical reports (such as "15 models were successfully generated today, 3 models failed to be generated, failure rate..."). Archiving and Cleanup: Archive historical records exceeding a preset period (e.g., 90 days), retaining core information (e.g., generated record ID, final status), deleting redundant detailed data, and optimizing storage performance. The generated record unit 141 provides reliable data support for system optimization by analyzing high-frequency abnormal states in the generated records (e.g., "large language model call timeout percentage"). This drives module iteration (e.g., optimizing model call retry mechanisms).

[0089] For example, please refer to Figure 7 In some embodiments, the state tracking module 140 further includes a performance monitoring unit 142, which is communicatively connected to the generation record unit 141. The performance monitoring unit 142 is configured to acquire performance indicator information of the configuration management module 110, the model generation module 120, and the data processing module 130 during the generation of the intelligent scoring model, and to store and manage the performance indicator information, which includes generation time, error output information, and resource consumption information.

[0090] Specifically, the performance monitoring unit 142 is the "performance analysis core" of the status tracking module 140. By quantitatively collecting, storing and analyzing the key performance indicators of each module, it enables efficiency evaluation, bottleneck location and optimization support for the intelligent scoring model generation process.

[0091] The performance monitoring unit 142 designs a fine-grained performance indicator collection scheme for the configuration management module 110, model generation module 120, and data processing module 130, covering three dimensions: "time efficiency," "quality stability," and "resource consumption." It generates time indicators including: module-level time consumption: recording the total time from startup to completion for each module (e.g., "configuration management module 110 time consumption 12 seconds," "model generation module 120 time consumption 45 seconds"); unit-level time consumption: breaking down the processing time of each unit within the module (e.g., "prompt word construction unit 121 time consumption 8 seconds," "large language model call time consumption 32 seconds," and "multi-level template processing unit 123 time consumption 5 seconds" in model generation module 120); and stage time consumption percentage: calculating the percentage of each module's time consumption in the total process time (e.g., "time consumption percentage of model generation module 120"). ”, to identify time bottlenecks in the process.

[0092] The performance monitoring unit 142 performs error type classification: recording error types by module (e.g., "failure to identify professional domain" in configuration management module 110, "timeout for large language model call" in model generation module 120, and "tree structure verification conflict" in data processing module 130); error frequency statistics: recording the number of times the same error occurs within a unit of time (e.g., "3 times in 1 hour for 'incorrect prompt word format'"); and error context: associating the specific parameters at the time of the error (e.g., model type and prompt word length when "timeout for large language model call") to provide a basis for root cause analysis.

[0093] The performance monitoring unit 142 collects resource consumption information during the generation of the intelligent scoring model, including computing resources: CPU utilization of each module during runtime (e.g., peak CPU utilization of the data parsing unit 131). ) and memory consumption (e.g., "peak memory usage of hierarchical structure conversion unit 132 is 200MB"); network resources: network latency when model generation module 120 calls large language models (e.g., "average response latency is 800ms"), data transmission volume (e.g., "single prompt word transmission is 1.2KB"); interface resources: success rate of interface calls between modules (e.g., "success rate of data transmission from configuration management module 110 to model generation module 120"). The performance monitoring unit 142 uses a combination of real-time sampling (high-frequency indicators such as CPU utilization are sampled every 100ms) and event triggering (such as recording immediately when an error occurs) to ensure the timeliness and completeness of the indicators.

[0094] The performance monitoring unit 142 constructs a performance indicator data model, converting the collected status data into a structured format, including basic dimensions: indicator name (e.g., "time taken to call a large language model"), value (e.g., "32 seconds"), and unit (e.g., "seconds"). ), collection timestamp; associated dimensions: model ID (e.g., “Model-20250701-001”), module (e.g., “Model Generation Module 120”), unit (e.g., “Model Calling Unit 122”), and operating environment (e.g., “Server Node A”).

[0095] The performance monitoring unit 142 uses a time-series database (such as Prometheus) to store performance metrics, supporting efficient queries by time series (such as "querying the average time spent by the model generation module 120 over the past 7 days"), and simultaneously establishes an associated index with the generation record unit 141 (associating performance metrics with generation status through model ID), realizing the linkage analysis of "status-performance" data (such as "when a model generation fails, the time spent calling its large language model abnormally increases to 180 seconds").

[0096] The performance monitoring unit 142 provides real-time monitoring and alerts, allowing for the setting of performance thresholds (e.g., "Alarm triggered when the total time of model generation module 120 exceeds 60 seconds" or "Alarm triggered when the error frequency exceeds 5 times within 10 minutes"). Key metrics (such as CPU usage trends and module time rankings) are displayed in real-time through a visual dashboard. For metrics exceeding thresholds, tiered alerts (e.g., warning, critical) are automatically triggered and pushed to the system administrator (e.g., via email, SMS, or console notification), along with details of the abnormal metrics (e.g., "The memory usage of server node B continuously exceeds..."). This affects the performance of data processing module 130.

[0097] For example, the performance monitoring unit 142 generates performance reports on a daily / weekly / monthly basis, and calculates the average, maximum, and volatility of core indicators (e.g., "The average time for model generation this week was 35 seconds, a decrease compared to last week"). ") etc.; based on historical data, identify performance bottlenecks (e.g., "the time consumed by calling large language models has long exceeded the standard"). The system generates optimization suggestions (e.g., "Increase the number of concurrent calls to the large language model" or "Optimize the length of prompt words to reduce transmission time"); tracks the effects of optimization measures (e.g., after implementing "prompt word compression," the average model call time decreased from 32 seconds to 25 seconds), forming a closed loop of "monitoring-analysis-optimization-verification." The performance monitoring unit 142 establishes performance baselines for each module (e.g., "The normal time range for configuration management module 110 is 5-15 seconds"), dynamically adjusting them based on domain and model complexity (e.g., "The baseline for the generation time of complex models in the financial domain is higher than that of basic models in the catering domain"); when the indicator deviates from the performance baseline by more than a preset range (e.g., ... When the error occurs, it is marked as "performance anomaly" and the root cause analysis process is triggered (e.g., "the configuration time of a certain education model suddenly increased to 30 seconds, and after investigation it was found that the retrieval efficiency decreased after the domain feature library was updated").

[0098] Please see Figure 8 In some embodiments, the intelligent scoring model generation system 100 further includes a knowledge enhancement module 150, which is communicatively connected to the data processing module 130.

[0099] The knowledge enhancement module 150 is configured to build a structured knowledge database based on expert knowledge in the user's field of expertise, and to store and manage the structured knowledge database.

[0100] The knowledge enhancement module 150 is also configured to acquire tree-structured data with relationships, and supplement and optimize the data nodes in the tree-structured data based on a structured knowledge database to obtain target data with relationships.

[0101] Specifically, the knowledge enhancement module 150 is the "professional knowledge empowerment hub" of the intelligent scoring model generation system 100. By integrating domain expert knowledge and applying it to the intelligent scoring model, it improves the model's professionalism, accuracy, and industry adaptability, making the generated scoring standards / reference standards more in line with actual application scenarios.

[0102] Among them, the knowledge enhancement module 150 performs the construction and management of a structured knowledge database, collects and standardizes expert knowledge from multiple sources, and supports multiple forms of expert knowledge input: receiving user-uploaded domain expert documents (such as "Chinese Cuisine Cooking Scoring Standards" and "Financial Risk Assessment Guidelines"), structured tables (such as "Weight Table of Key Indicators for Goose Meat Cooking"), natural language descriptions (such as expert oral descriptions of "Key Points for Scoring the Removal of Fishy Smell from Braised Goose"), and rule-based knowledge (such as "If the freshness of the ingredients does not meet the standard, deduct directly"). (Base score).

[0103] The knowledge enhancement module 150 performs structured parsing of input knowledge. For unstructured text, it extracts core knowledge units (such as "indicator name", "scoring threshold", "industry practice") using NLP technology. For example, it extracts knowledge elements such as "ingredient type (old goose / young goose)" and "cooking time difference (+20 minutes)" from "old goose meat is tougher and requires 20 minutes longer cooking time than young goose". For semi-structured data, it converts table and list format data into a unified triple structure (such as "indicator-relationship-standard": taste-related factors-cooking time). For rule-based knowledge, it converts it into executable logical expressions (such as "IF ingredient freshness < 60 points THEN total score = total score × 0.7").

[0104] The knowledge enhancement module 150 constructs a layered structured knowledge database, building a multi-level knowledge system according to "domain-subdomain-knowledge type". The domain layer distinguishes between primary domains such as catering, education, and finance, storing general knowledge across subdomains (e.g., "all scoring models must satisfy the sum of indicator weights of..."). Sub-domain layer: refined to specific scenarios (e.g., "poultry cooking" and "baking" in the catering field), storing sub-domain-specific knowledge (e.g., "goose cooking requires a focus on evaluating the removal of fishy smell, while pork cooking pays more attention to the lean-to-fat ratio"); Knowledge type layer: stored by function, including an indicator library (e.g., "10 core evaluation indicators for poultry cooking") and a weight rule library (e.g., "hygiene indicators should account for no less than a certain percentage in catering scores"). ), an anomaly handling library (e.g., "If a food safety problem occurs, it will be directly judged as unqualified"), and an industry standard library (e.g., "Complies with the scoring details for poultry meat processing in the National Occupational Standard for Chinese Chefs").

[0105] The knowledge enhancement module 150 dynamically manages and updates the knowledge database, establishing a knowledge version control mechanism to record the source of each piece of knowledge (e.g., "Source: Executive Chef of a five-star hotel"), effective date, and update log (e.g., "Revised 'Goose Meat Tenderness Scoring Standard' on 2025-08-01"), supporting knowledge backtracking and version comparison. It provides knowledge review and verification functions: newly entered knowledge must be manually reviewed by domain experts or verified by system rules (e.g., no conflict with existing knowledge) before it can take effect, avoiding erroneous knowledge from affecting model quality. It supports automatic knowledge update triggering: when users input new expert knowledge or the system detects changes in domain standards (e.g., "The catering industry adds a 'low-carbon cooking' evaluation dimension"), the corresponding entries in the database are automatically updated and marked as "pending application."

[0106] For example, the knowledge enhancement module 150 associates and matches tree-structured data with a knowledge database. It receives tree-structured data (such as the indicator tree of a "goose meat cooking scoring model") output by the data processing module 130, and uses a semantic matching algorithm (such as BERT-based similarity calculation) to match related knowledge in the knowledge database for each data node (such as the "taste" indicator). This includes indicator name matching (e.g., the "removing fishy smell" node matching "3 standard methods and scoring criteria for removing fishy smell from goose meat" in the database); hierarchical relationship matching (e.g., the first-level node "operating specifications" matching "general framework requirements for operating specifications in the catering industry"); and scoring rule matching (e.g., the "deduction items" node matching "industry practices for common deduction items in cooking scoring (e.g., double deduction for hygiene issues)"). For matching scores below a threshold (e.g., ...), ... The node is marked as "Knowledge to be supplemented" to prompt the user to upload relevant expert knowledge or call the default domain knowledge.

[0107] Knowledge Enhancement Module 150 enhances tree-structured data in multiple dimensions based on matching knowledge, supplementing indicator dimensions: When the knowledge database shows that a certain domain has core indicators that are not included (e.g., "the goose meat cooking scoring model lacks the 'ingredient traceability' indicator, while new regulations in the catering industry require this indicator to account for a certain percentage"), The following suggestions are made: Add new nodes and assign reasonable scores; refine the scoring criteria: optimize vague scoring rules (e.g., "deduct 3-5 points for poor taste") into specific rules based on expert knowledge (e.g., "deduct 3 points for tough meat, deduct 5 points for overly dry meat, deduct 4 points for fishy smell"); adjust the weighting rationality: correct unreasonable weighting in the tree structure based on industry practices in the knowledge database (e.g., "in financial risk assessment, the weight of the 'solvency' indicator should be higher than that of 'operational capability'") (e.g., adjust "solvency" from 20 points to 30 points); supplement association rules: add domain-specific dependencies to nodes (e.g., "if 'ingredient freshness' does not meet the standard, then the highest score for 'finished product flavor' is..."). This enhances the logical rigor of the model.

[0108] The knowledge enhancement module 150 integrates the supplemented and optimized tree-structured data into "target data with related relationships," preserving the hierarchical relationships of the original structure while adding knowledge enhancement tags (e.g., "The scoring criteria for this indicator comes from the National Occupational Standard for Chinese Chefs"). The knowledge enhancement module 150 performs knowledge consistency checks on the target data, checking the compatibility of new content with existing indicators (e.g., after adding the "low-carbon cooking" indicator, the total score remains at 100 points); verifying whether the scoring rules conform to domain knowledge (e.g., whether the "heat control" scoring criteria for goose meat cooking is consistent with the expert-provided knowledge of "high-temperature stir-frying is suitable for tender goose"); and generating adjustment suggestions for data with verification anomalies (e.g., "Expert knowledge A suggests 'weighting the removal of fishy smell'"). 'Knowledge B suggests' 'This can be referenced from the industry average.' (This will take effect upon user confirmation.)

[0109] In summary, the intelligent scoring model generation system provided in this embodiment of the invention utilizes a configuration management module to identify and classify the professional fields of the intelligent scoring model, manage and store the configuration information of the intelligent scoring model, uses a model generation module to call a large language model to generate raw data about the intelligent scoring model, and uses a data processing module to process the raw data, outputting a tree-structured data with correlation relationships, ultimately obtaining the intelligent scoring model. In this way, a professional and standardized intelligent scoring model can be quickly generated using a large language model, saving time and effort and enhancing the accuracy of the intelligent scoring model in professional field knowledge.

[0110] Those skilled in the art will understand that the embodiments provided by this invention are merely illustrative. The order in which the steps in the methods of the embodiments are written does not imply a strict execution order and does not constitute any limitation on the implementation process. The order can be adjusted, merged, and deleted according to actual needs. Modules or sub-modules, units or sub-units in the apparatus or system of the embodiments can be merged, divided, and deleted according to actual needs. For example, the division of units is only a logical functional division, and there may be other division methods in actual implementation. For another example, multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed.

[0111] Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a general-purpose hardware platform, and of course, it can also be implemented using hardware. Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This computer program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. It should be understood that the storage medium can be flash memory, hard disk, optical disk, register, magnetic surface memory, removable disk, CD-ROM, random access memory (RAM), read-only memory (ROM), electrically programmable ROM, and electrically erasable programmable ROM, etc.

[0112] It should be noted that the above embodiments are for illustrating the technical concept and features of the present invention, and are intended to enable those skilled in the art to understand the content of the present invention and implement it accordingly. They should not be construed as limiting the scope of protection of the present invention. Those skilled in the art can understand that all or part of the processes of the above embodiments can be implemented by modifying the technical solutions described in the embodiments of the present invention, or by making equivalent substitutions for some of the technical features. It is understood that these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention, and should be considered as equivalent changes and modifications made based on the embodiments of the present invention, all of which should fall within the scope of the claims of the present invention.

Claims

1. A system for generating an intelligent scoring model, characterized in that, The intelligent scoring model generation system is communicatively connected to the large language model, and the intelligent scoring model generation system includes: The configuration management module is configured to respond to user input, identify and classify the professional fields of the intelligent scoring model to be generated based on natural language processing technology, and manage and store the configuration information of the intelligent scoring model; The model generation module, connected to the configuration management module, is configured to use a preset format template to call the large language model based on the configuration information, so as to use the large language model to generate raw data about the intelligent scoring model; The data processing module, connected to the model generation module, is configured to process the raw data to output a tree-structured data with relationships, wherein the tree-structured data with relationships is the intelligent scoring model, which serves as a reference standard for users to score objects within a professional field.

2. The intelligent scoring model generation system according to claim 1, characterized in that, The configuration management module includes an identification and classification unit, a management unit, and a storage unit, wherein the management unit is connected to the identification and classification unit and the storage unit, respectively. The identification and classification unit is configured to respond to user input and, based on natural language processing technology, identify and classify the target professional field of the intelligent scoring model to be generated. The management unit is configured to retrieve target configuration information corresponding to the target professional field based on the target professional field, and configure the model information of the intelligent scoring model as the target configuration information; The storage unit is configured to store the target configuration information of the intelligent scoring model.

3. The intelligent scoring model generation system according to claim 2, characterized in that, The model generation module includes a prompt word construction unit and a model invocation unit, wherein the prompt word construction unit is connected to the model invocation unit; The prompt word construction unit is configured to acquire a preset format module, user input parameters and target configuration information, and generate target prompt words based on the format module, the user input parameters and the target configuration information. The target prompt words are used to prompt the large language model to output raw data about the intelligent scoring model. The model invocation unit is configured to invoke the large language model based on the target prompt word to generate raw data about the intelligent scoring model.

4. The intelligent scoring model generation system according to claim 3, characterized in that, The model generation module further includes a multi-level template processing unit, which is connected to the model calling unit; The multi-level template processing unit is configured to acquire the original data of the intelligent scoring model, perform multi-level layout on the original data based on a predefined hierarchical structure template, and output the original data after multi-level layout.

5. The intelligent scoring model generation system according to any one of claims 1-4, characterized in that, The original data is data after multi-level typesetting. The data processing module includes a data parsing unit, a hierarchical structure conversion unit, and an identifier allocation unit. The hierarchical structure conversion unit is connected to the data parsing unit and the identifier allocation unit, respectively. The data parsing unit is configured to acquire the original data after the multi-level layout, and to clean and format the original data after the multi-level layout to obtain reference data. The hierarchical structure conversion unit is configured to acquire the reference data and convert the reference data into a tree-structured data with relationships. The identifier allocation unit is configured to acquire the tree structure data and assign a unique reference identifier to each data node in the tree structure data based on the association relationship.

6. The intelligent scoring model generation system according to any one of claims 1-4, characterized in that, The intelligent scoring model generation system also includes a state tracking module, which is connected to the configuration management module, the model generation module, and the data processing module. The status tracking module includes a generation record unit, which is configured to acquire the working status of the configuration management module, the model generation module, and the data processing module during the generation of the intelligent scoring model, determine the generation status of the intelligent scoring model based on the working status, and store and manage the generation status of the intelligent scoring model.

7. The intelligent scoring model generation system according to claim 6, characterized in that, The status tracking module further includes a performance monitoring unit, which is connected to the generation record unit. The performance monitoring unit is configured to acquire performance indicator information of the configuration management module, the model generation module, and the data processing module during the generation of the intelligent scoring model, and to store and manage the performance indicator information, which includes generation time, error output information, and resource consumption information.

8. The intelligent scoring model generation system according to any one of claims 1-4, characterized in that, The intelligent scoring model generation system also includes a knowledge enhancement module, which is connected to the data processing module. The knowledge enhancement module is configured to construct a structured knowledge database based on expert knowledge in the user's professional field, and to store and manage the structured knowledge database. The knowledge enhancement module is also configured to acquire the tree-structured data with relationships, and supplement and optimize the data nodes in the tree-structured data based on the structured knowledge database to obtain target data with relationships.

9. The intelligent scoring model generation system according to any one of claims 1-4, characterized in that, The preset format template is a JSON format template.

10. The system for generating the intelligent scoring model according to any one of claims 1-4, characterized in that, The relationship described is a parent-child relationship.

Images