Vehicle knowledge quality evaluation method, server and computer readable storage medium

Abstract

The invention discloses a vehicle knowledge quality evaluation method, a server and a computer readable storage medium. The method comprises the steps of obtaining vehicle knowledge information; and based on a preset vehicle knowledge evaluation dimension, according to the vehicle knowledge information and a preset evaluation rule associated with the preset vehicle knowledge evaluation dimension, performing quality evaluation on the vehicle knowledge information. Thus, through automatic quality evaluation, high-quality vehicle knowledge information is obtained, a reliable knowledge base can be constructed, high-quality data is provided for training and application of a large language model in the vehicle-mounted field, the reasoning ability of the large language model is improved, and therefore the user experience is improved.

Description

Technical Field

[0001] The present invention relates to the field of knowledge quality assessment, and particularly to a method for assessing the quality of vehicle knowledge, a server, and a computer-readable storage medium. Background Art

[0002] In the related art, in order to enable an in-vehicle intelligent system to provide a natural and fluent interaction experience through a large language model, vehicle knowledge information is usually used to train the large language model. However, the quality of this vehicle knowledge information, which usually comes from the Internet, varies greatly. Directly using this vehicle knowledge information to train the large language model may lead to a decline in model performance, incorrect inference results, and affect the user experience. Summary of the Invention

[0003] This application provides a method for assessing the quality of vehicle knowledge, a server, and a computer-readable storage medium.

[0004] An embodiment of this application provides a method for assessing the quality of vehicle knowledge, the method comprising:

[0005] Obtain vehicle knowledge information;

[0006] Based on a preset vehicle knowledge assessment dimension, according to the vehicle knowledge information and a preset assessment rule associated with the preset vehicle knowledge assessment dimension, perform a quality assessment on the vehicle knowledge information.

[0007] In this way, the server obtains vehicle knowledge information. Then, based on the preset vehicle knowledge assessment dimension, the server performs a quality assessment on the vehicle knowledge information according to the vehicle knowledge information and the preset assessment rule associated with the preset vehicle knowledge assessment dimension. In this way, through automated quality assessment, high-quality vehicle knowledge information is obtained, a reliable knowledge base can be constructed, high-quality data can be provided for the training and application of large language models in the in-vehicle field, the inference ability of the large language model can be improved, and thus the user experience can be enhanced.

[0008] In some embodiments, the preset vehicle knowledge assessment dimension includes multiple preset sub-dimensions, the preset assessment rule includes multiple groups of preset sub-assessment rules, each of the preset sub-dimensions is associated with a group of the preset sub-assessment rules, and the performing a quality assessment on the vehicle knowledge information based on the preset vehicle knowledge assessment dimension, according to the vehicle knowledge information and the preset assessment rule associated with the preset vehicle knowledge assessment dimension, includes:

[0009] According to the vehicle knowledge information and the preset sub-assessment rule, determine a sub-assessment result of the vehicle knowledge information in each of the preset sub-dimensions;

[0010] Based on a preset quality assessment rule, the vehicle knowledge information is quality-assessed according to the multiple sub-assessment results.

[0011] In this way, the server determines the sub-assessment results of the vehicle knowledge information in each preset sub-dimension according to the vehicle knowledge information and the preset sub-assessment rules. Then, based on the preset quality assessment rule, the server quality-assesses the vehicle knowledge information according to the multiple sub-assessment results. In this way, through different sub-dimensions and sub-assessment rules, the quality of all aspects of the vehicle knowledge information can be evaluated in detail, improving the accuracy and comprehensiveness of the assessment. In addition, the detailed assessment results can provide specific guidance for the optimization and update of the vehicle knowledge information, and targeted optimization helps to continuously improve the quality of the vehicle knowledge information.

[0012] In some embodiments, the preset vehicle knowledge assessment dimension includes a first preset sub-dimension, the preset assessment rule includes a first preset sub-assessment rule corresponding to the first preset sub-dimension, and the determining of the sub-assessment results of the vehicle knowledge information in each preset sub-dimension according to the vehicle knowledge information and the preset sub-assessment rules includes:

[0013] Construct a first prompt word according to the first preset assessment rule, where the first prompt word is configured to guide a first preset large language model to determine the existence of factual errors in the vehicle knowledge information;

[0014] Based on the first prompt word, evaluate the vehicle knowledge information through the first preset large language model to determine a first sub-assessment result of the vehicle knowledge information in the first preset sub-dimension, where the first sub-assessment result includes a first dimension quality score and a first score description associated with the first dimension quality score.

[0015] In this way, the server constructs a first prompt word according to the first preset assessment rule, and the first prompt word is configured to guide the first preset large language model to determine the existence of factual errors in the vehicle knowledge information. Then, based on the first prompt word, the server evaluates the vehicle knowledge information through the first preset large language model to determine a first sub-assessment result of the vehicle knowledge information in the first preset sub-dimension, where the first sub-assessment result includes a first dimension quality score and a first score description associated with the first dimension quality score. In this way, by constructing a specific first prompt word, the large language model can be guided to utilize its own powerful semantic understanding and reasoning capabilities to accurately identify factual errors in the vehicle knowledge information, give a quality score, and provide a detailed score description to help users understand the assessment results.

[0016] In some embodiments, the preset vehicle knowledge evaluation dimension further includes a second preset sub-dimension, the preset evaluation rule further includes a second preset sub-evaluation rule corresponding to the second preset sub-dimension, and the determining of the sub-evaluation results of the vehicle knowledge information in each of the preset sub-dimensions according to the vehicle knowledge information and the preset sub-evaluation rule further includes:

[0017] Construct a second prompt word according to the second preset evaluation rule, the second prompt word is configured to guide the first preset large language model to determine the existence of user subjective feeling information of the vehicle knowledge information;

[0018] Based on the second prompt word, evaluate the vehicle knowledge information through the first preset large language model to determine the second sub-evaluation result of the vehicle knowledge information in the second preset sub-dimension, and the second sub-evaluation result includes a second dimension quality score and a second score description associated with the second dimension quality score.

[0019] In this way, the server constructs a second prompt word according to the second preset evaluation rule, and the second prompt word is configured to guide the first preset large language model to determine the existence of user subjective feeling information of the vehicle knowledge information. Then, based on the second prompt word, the server evaluates the vehicle knowledge information through the first preset large language model to determine the second sub-evaluation result of the vehicle knowledge information in the second preset sub-dimension, and the second sub-evaluation result includes a second dimension quality score and a second score description associated with the second dimension quality score. In this way, by constructing a specific second prompt word, the large language model can be guided to utilize its own powerful semantic understanding and reasoning capabilities to accurately identify the user subjective feeling information in the vehicle knowledge information, give a quality score, and provide a detailed score description to help the user understand the evaluation result.

[0020] In some embodiments, the preset vehicle knowledge evaluation dimension further includes a third preset sub-dimension, the preset evaluation rule further includes a third preset sub-evaluation rule corresponding to the third preset sub-dimension, and the determining of the sub-evaluation results of the vehicle knowledge information in each of the preset sub-dimensions according to the vehicle knowledge information and the preset sub-evaluation rule further includes:

[0021] Construct a third prompt word according to the third preset evaluation rule, the third prompt word is configured to guide the first preset large language model to determine the existence of functional description elements of the vehicle knowledge information;

[0022] Based on the third prompting word, the vehicle knowledge information is evaluated by the first pre-set large language model to determine the third sub-evaluation result of the vehicle knowledge information in the third pre-set sub-dimension, and the third sub-evaluation result includes a third dimension quality score and a third score description associated with the third dimension quality score.

[0023] In this way, the server constructs a third prompting word according to the third pre-set evaluation rule, and the third prompting word is configured to guide the first pre-set large language model to determine the existence of functional description elements of the vehicle knowledge information. Then, based on the third prompting word, the server evaluates the vehicle knowledge information through the first pre-set large language model to determine the third sub-evaluation result of the vehicle knowledge information in the third pre-set sub-dimension, and the third sub-evaluation result includes a third dimension quality score and a third score description associated with the third dimension quality score. In this way, by constructing a specific third prompting word, it is possible to guide the large language model to utilize its own powerful semantic understanding and reasoning capabilities to accurately identify the functional description element information in the vehicle knowledge information, give a quality score, and provide a detailed score description to help users understand the evaluation result.

[0024] In some embodiments, the pre-set vehicle knowledge evaluation dimension further includes a fourth pre-set sub-dimension, the pre-set evaluation rule further includes a fourth pre-set sub-evaluation rule corresponding to the fourth pre-set sub-dimension, and determining the sub-evaluation result of the vehicle knowledge information in each pre-set sub-dimension according to the vehicle knowledge information and the pre-set sub-evaluation rule further includes:

[0025] Construct a fourth prompting word according to the fourth pre-set evaluation rule, and the fourth prompting word is configured to guide the first pre-set large language model to determine the existence of semantic ambiguity information of the vehicle knowledge information;

[0026] Based on the fourth prompting word, the vehicle knowledge information is evaluated by the first pre-set large language model to determine the fourth sub-evaluation result of the vehicle knowledge information in the fourth pre-set sub-dimension, and the fourth sub-evaluation result includes a fourth dimension quality score and a fourth score description associated with the fourth dimension quality score.

[0027] Thus, the server constructs a fourth prompt word according to the fourth preset evaluation rule, and the fourth prompt word is configured to guide the first preset large language model to determine the existence of semantic ambiguity information in the vehicle knowledge information. Then, based on the fourth prompt word, the server evaluates the vehicle knowledge information through the first preset large language model to determine the fourth sub-evaluation result of the vehicle knowledge information in the fourth preset sub-dimension. The fourth sub-evaluation result includes a fourth dimension quality score and a fourth score description associated with the fourth dimension quality score. In this way, by constructing a specific fourth prompt word, the large language model can be guided to utilize its own powerful semantic understanding and reasoning capabilities to accurately identify the semantic ambiguity information in the vehicle knowledge information, give a quality score, and provide a detailed score description to help the user understand the evaluation result.

[0028] In some embodiments, the preset vehicle knowledge evaluation dimension further includes a fifth preset sub-dimension, the preset evaluation rule further includes a fifth preset sub-evaluation rule corresponding to the fifth preset sub-dimension, and the step of determining the sub-evaluation result of the vehicle knowledge information in each preset sub-dimension according to the vehicle knowledge information and the preset sub-evaluation rule further includes:

[0029] Construct a fifth prompt word according to the fifth preset evaluation rule, and the fifth prompt word is configured to guide the first preset large language model to determine the existence of redundant information in the vehicle knowledge information;

[0030] Based on the fifth prompt word, evaluate the vehicle knowledge information through the first preset large language model to determine the fifth sub-evaluation result of the vehicle knowledge information in the fifth preset sub-dimension. The fifth sub-evaluation result includes a fifth dimension quality score and a fifth score description associated with the fifth dimension quality score.

[0031] Thus, the server constructs a fifth prompt word according to the fifth preset evaluation rule, and the fifth prompt word is configured to guide the first preset large language model to determine the existence of redundant information in the vehicle knowledge information. Then, based on the fifth prompt word, the server evaluates the vehicle knowledge information through the first preset large language model to determine the fifth sub-evaluation result of the vehicle knowledge information in the fifth preset sub-dimension. The fifth sub-evaluation result includes a fifth dimension quality score and a fifth score description associated with the fifth dimension quality score. In this way, by constructing a specific fifth prompt word, the large language model can be guided to utilize its own powerful semantic understanding and reasoning capabilities to accurately identify the redundant information in the vehicle knowledge information, give a quality score, and provide a detailed score description to help the user understand the evaluation result.

[0032] In some embodiments, the method further includes:

[0033] When the quality evaluation result of the vehicle knowledge information is to be optimized, based on the second preset large language model, the vehicle knowledge information is optimized to determine the optimized vehicle knowledge information.

[0034] Based on the preset vehicle knowledge evaluation dimension, according to the optimized vehicle knowledge information and the preset evaluation rule, the quality of the optimized vehicle knowledge information is evaluated.

[0035] In this way, when the quality evaluation result of the vehicle knowledge information is to be optimized, based on the second preset large language model, the server optimizes the vehicle knowledge information to determine the optimized vehicle knowledge information. Then, based on the preset vehicle knowledge evaluation dimension, the server evaluates the quality of the optimized vehicle knowledge information according to the optimized vehicle knowledge information and the preset evaluation rule. In this way, by optimizing the vehicle knowledge information to be optimized and re - evaluating its quality, the number of vehicle knowledge information that meets the quality requirements can be increased.

[0036] In some embodiments, the optimizing the vehicle knowledge information based on the second preset large language model to determine the optimized vehicle knowledge information includes:

[0037] Performing fact - error removal processing, rewriting processing, and / or information combination processing on the vehicle knowledge information to determine the optimized vehicle knowledge information.

[0038] In this way, the server performs fact - error removal processing, rewriting processing, and / or information combination processing on the vehicle knowledge information to determine the optimized vehicle knowledge information. In this way, by using the second preset large language model to identify the content to be optimized in the vehicle knowledge information and optimizing the content to be optimized, the quality of the vehicle knowledge information can be improved.

[0039] An embodiment of the present application provides a server, which includes a processor and a memory. A computer program is stored on the memory. When the computer program is executed by the processor, the above - mentioned method is implemented.

[0040] An embodiment of the present application provides a computer - readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above - mentioned method are implemented.

[0041] The additional aspects and advantages of the embodiments of the present application will be partially given in the following description, partially become obvious from the following description, or be understood through the practice of the embodiments of the present application. Description of the Drawings

[0042] The above - mentioned and / or additional aspects and advantages of the present application will become obvious and easy to understand from the description of the embodiments in conjunction with the following drawings, where:

[0043] Figure 1 is one of the flow schematic diagrams of the vehicle knowledge quality assessment method according to some embodiments of the present application;

[0044] Figure 2 is the second of the flow schematic diagrams of the vehicle knowledge quality assessment method according to some embodiments of the present application;

[0045] Figure 3 is the third of the flow schematic diagrams of the vehicle knowledge quality assessment method according to some embodiments of the present application;

[0046] Figure 4 is the fourth of the flow schematic diagrams of the vehicle knowledge quality assessment method according to some embodiments of the present application;

[0047] Figure 5 is the fifth of the flow schematic diagrams of the vehicle knowledge quality assessment method according to some embodiments of the present application;

[0048] Figure 6 is the sixth of the flow schematic diagrams of the vehicle knowledge quality assessment method according to some embodiments of the present application;

[0049] Figure 7 is the seventh of the flow schematic diagrams of the vehicle knowledge quality assessment method according to some embodiments of the present application;

[0050] Figure 8 is the eighth of the flow schematic diagrams of the vehicle knowledge quality assessment method according to some embodiments of the present application;

[0051] Figure 9 is the ninth of the flow schematic diagrams of the vehicle knowledge quality assessment method according to some embodiments of the present application. Detailed Embodiments

[0052] The following details the embodiments of the present application. The examples of the embodiments are shown in the accompanying drawings, where the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary only for explaining the embodiments of the present application and should not be construed as limiting the embodiments of the present application.

[0053] Currently, large language models (LLMs) are being increasingly widely applied in the vehicle field. Through large language models, in-vehicle intelligent systems can provide a natural and fluent voice interaction experience, enabling drivers to control various functions of the vehicle through voice commands, such as navigation, music playback, phone calls, etc. To improve the performance of large language models and enable them to accurately recognize and understand the intentions of users, large language models are usually trained and optimized with a large amount of vehicle knowledge information.

[0054] However, this vehicle knowledge information mainly comes from news, blogs, public documents, etc. on the Internet. The quality of vehicle knowledge information is uneven. Some vehicle knowledge information may be inaccurate, some may be outdated, and some may be repetitive. If these vehicle knowledge information are not evaluated and directly used to train the large language model, it may lead to a decline in the performance of the large language model and even produce incorrect prediction results, affecting the user experience.

[0055] Based on the above problems, please refer to Figure 1 , an embodiment of the present application provides a vehicle knowledge quality assessment method, which includes:

[0056] 01: Obtain vehicle knowledge information;

[0057] 02: Based on a preset vehicle knowledge evaluation dimension, according to the vehicle knowledge information and a preset evaluation rule associated with the preset vehicle knowledge evaluation dimension, perform a quality assessment on the vehicle knowledge information.

[0058] An embodiment of the present application also provides a server, including a memory and a processor. The vehicle knowledge quality assessment method of the embodiment of the present application can be implemented by the server of the embodiment of the present application. Specifically, a computer program is stored in the memory, and the processor is used to obtain vehicle knowledge information. And based on a preset vehicle knowledge evaluation dimension, according to the vehicle knowledge information and a preset evaluation rule associated with the preset vehicle knowledge evaluation dimension, perform a quality assessment on the vehicle knowledge information.

[0059] An embodiment of the present application also provides a quality assessment device. The vehicle knowledge quality assessment method of the embodiment of the present application can be implemented by the quality assessment device of the embodiment of the present application. Specifically, the quality assessment device includes an acquisition module and a quality assessment module. The acquisition module is used to obtain vehicle knowledge information. The quality assessment module is used to perform a quality assessment on the vehicle knowledge information based on a preset vehicle knowledge evaluation dimension, according to the vehicle knowledge information and a preset evaluation rule associated with the preset vehicle knowledge evaluation dimension.

[0060] Specifically, vehicle knowledge information refers to various types of knowledge information related to automobiles, which are sourced from news, blogs, public documents, etc. on the Internet. Their quality varies, and they mainly cover aspects such as vehicle functions, vehicle equipment, vehicle technologies, and practical knowledge related to vehicles. In terms of vehicle functions, it includes descriptions, usage methods, and effects of functions such as steering wheel heating, rapid air-conditioning cooling, navigation systems, music playback, and phone calls. In terms of vehicle equipment, it includes introductions, working principles, and usage methods of equipment such as air conditioners, stereos, and navigators. In terms of vehicle technologies, it includes the principles, development, and applications of technologies such as autonomous driving technology, battery technology, and engine technology. In terms of practical vehicle knowledge, it includes practical knowledge related to automobiles such as vehicle maintenance, driving skills, and traffic rules.

[0061] The preset vehicle knowledge evaluation dimensions refer to multiple aspects used to measure the quality of vehicle knowledge information, including authenticity, objectivity, comprehensiveness, clarity, conciseness, and timeliness, etc. Authenticity refers to whether the vehicle knowledge information is true and reliable and whether there are factual errors. Objectivity refers to whether the vehicle knowledge information is objective and fair and whether there are subjective evaluations, rhetorical devices, or subjective emotions. Comprehensiveness refers to whether the vehicle knowledge information is comprehensive and complete and whether it covers content such as relevant equipment, usage effects, option differences, usage scenarios, and taboo scenarios. Clarity refers to whether the vehicle knowledge information is clear and easy to understand and whether there are situations of ambiguous limitations or ambiguous information. Conciseness refers to whether the vehicle knowledge information is concise and clear and whether there is irrelevant content or redundant nonsense. Timeliness refers to whether the vehicle knowledge information is the latest or whether it is applicable to the current time point.

[0062] The preset evaluation rules refer to the corresponding specific evaluation criteria formulated for each preset vehicle knowledge evaluation dimension, which can guide the large language model to conduct quality evaluation on vehicle domain knowledge. By quantifying the vehicle knowledge information, it can standardly conduct quality evaluation on vehicle domain knowledge.

[0063] Collect vehicle knowledge information in the vehicle domain from various channels such as news, blogs, and public documents, such as automobile function introductions, usage guides, and maintenance information, etc.

[0064] Then, in accordance with the preset vehicle knowledge evaluation criteria, combined with the collected vehicle knowledge content and the corresponding evaluation rules, evaluate the knowledge quality of the preset vehicle knowledge evaluation dimensions of the vehicle knowledge information, and judge the quality level of the vehicle knowledge information according to the evaluation results.

[0065] In summary, in the vehicle knowledge quality assessment method and server provided by the embodiments of the present application, the server obtains vehicle knowledge information. Then, based on the preset vehicle knowledge assessment dimensions, the server performs a quality assessment on the vehicle knowledge information according to the vehicle knowledge information and the preset assessment rules associated with the preset vehicle knowledge assessment dimensions. In this way, through automated quality assessment, high-quality vehicle knowledge information is obtained, a reliable knowledge base is constructed, high-quality data is provided for the training and application of large language models in the in-vehicle field, the reasoning ability of the large language model can be improved, and thus the user experience can be enhanced.

[0066] Please refer to Figure 2 , in some embodiments, the preset vehicle knowledge assessment dimensions include multiple preset sub-dimensions, the preset assessment rules include multiple groups of preset sub-assessment rules, each preset sub-dimension is associated with a group of preset sub-assessment rules, and step 02 (based on the preset vehicle knowledge assessment dimensions, performing a quality assessment on the vehicle knowledge information according to the vehicle knowledge information and the preset assessment rules associated with the preset vehicle knowledge assessment dimensions) includes:

[0067] 021: Determine the sub-assessment results of the vehicle knowledge information in each preset sub-dimension according to the vehicle knowledge information and the preset sub-assessment rules;

[0068] 022: Based on the preset quality assessment rules, perform a quality assessment on the vehicle knowledge information according to multiple sub-assessment results.

[0069] In some embodiments, the quality assessment module is further configured to determine the sub-assessment results of the vehicle knowledge information in each preset sub-dimension according to the vehicle knowledge information and the preset sub-assessment rules. And based on the preset quality assessment rules, perform a quality assessment on the vehicle knowledge information according to multiple sub-assessment results.

[0070] In some embodiments, the processor is further configured to determine the sub-assessment results of the vehicle knowledge information in each preset sub-dimension according to the vehicle knowledge information and the preset sub-assessment rules. And based on the preset quality assessment rules, perform a quality assessment on the vehicle knowledge information according to multiple sub-assessment results.

[0071] Specifically, the multiple preset sub-dimensions refer to dimensions such as authenticity, objectivity, comprehensiveness, clarity, conciseness, and timeliness. For each sub-dimension, corresponding preset sub-evaluation rules are formulated. For example, for authenticity, the corresponding preset sub-evaluation rule is: "If there are 4 or more factual errors violating common sense in the vehicle knowledge information, 1 point is given; if there are 3 factual errors violating common sense in the vehicle knowledge information, 2 points are given; if there are 2 factual errors violating common sense in the vehicle knowledge information, 3 points are given; if there is 1 factual error violating common sense in the vehicle knowledge information, 4 points are given; if there are no factual errors violating common sense in the vehicle knowledge information, 5 points are given." It should be noted that the quality of the vehicle knowledge information is evaluated from the five preset sub-dimensions of authenticity, objectivity, comprehensiveness, clarity, and conciseness below, and the sub-evaluation results of the above five preset sub-dimensions are determined.

[0072] The preset quality evaluation rule refers to the standard for determining the overall evaluation result of the vehicle knowledge information based on the sub-evaluation results of the vehicle knowledge information in each preset sub-dimension. In some embodiments, the preset quality evaluation rule for evaluating the quality of the vehicle knowledge information according to the sub-evaluation results of the five preset sub-dimensions of authenticity, objectivity, comprehensiveness, clarity, and conciseness is: "According to the scores of the vehicle knowledge information in each dimension, the vehicle knowledge information can be divided into three categories: qualified vehicle knowledge information, vehicle knowledge information to be optimized, and discarded vehicle knowledge information. Among them, the vehicle knowledge information with authenticity of 5 points, objectivity of 4 - 5 points, comprehensiveness of 5 points, clarity of 4 - 5 points, and conciseness of 4 - 5 points is determined as qualified vehicle knowledge information. The vehicle knowledge information that meets one of the following conditions is determined as discarded vehicle knowledge information: authenticity of 1 - 3 points, objectivity of 1 - 2 points, comprehensiveness of 1 point, clarity of 1 point, and conciseness of 1 point. The vehicle knowledge information that is neither qualified vehicle knowledge information nor discarded vehicle knowledge information is determined as vehicle knowledge information to be optimized."

[0073] According to the preset sub-evaluation rules corresponding to the five preset sub-dimensions of authenticity, objectivity, comprehensiveness, clarity, and conciseness, the quality of the vehicle knowledge information is evaluated to determine the five sub-evaluation results corresponding to the vehicle knowledge information and the above five preset sub-dimensions.

[0074] Next, based on the preset quality evaluation rule, the multiple sub-evaluation results are comprehensively analyzed to obtain the overall evaluation result of the vehicle knowledge information.

[0075] In this way, the server determines the sub - evaluation results of the vehicle knowledge information in each preset sub - dimension according to the vehicle knowledge information and the preset sub - evaluation rules. Then, based on the preset quality evaluation rules, the server conducts a quality evaluation of the vehicle knowledge information according to multiple sub - evaluation results. In this way, through different sub - dimensions and sub - evaluation rules, the quality of all aspects of the vehicle knowledge information can be evaluated in detail, improving the accuracy and comprehensiveness of the evaluation. In addition, the detailed evaluation results can provide specific guidance for the optimization and update of the vehicle knowledge information, enabling targeted optimization and contributing to the continuous improvement of the quality of the vehicle knowledge information.

[0076] Please refer to Figure 3 , in some embodiments, the preset vehicle knowledge evaluation dimension includes a first preset sub - dimension, and the preset evaluation rule includes a first preset sub - evaluation rule corresponding to the first preset sub - dimension. Step 021 (determining the sub - evaluation results of the vehicle knowledge information in each preset sub - dimension according to the vehicle knowledge information and the preset sub - evaluation rules) includes:

[0077] 0211: Construct a first prompt word according to the first preset evaluation rule;

[0078] 0212: Based on the first prompt word, evaluate the vehicle knowledge information through the first preset large - language model to determine the first sub - evaluation result of the vehicle knowledge information in the first preset sub - dimension.

[0079] In some embodiments, the quality evaluation device further includes a construction module, and the construction module is further used to construct a first prompt word according to the first preset evaluation rule. The quality evaluation module is further used to evaluate the vehicle knowledge information through the first preset large - language model based on the first prompt word to determine the first sub - evaluation result of the vehicle knowledge information in the first preset sub - dimension.

[0080] In some embodiments, the processor is further used to construct a first prompt word according to the first preset evaluation rule. And based on the first prompt word, evaluate the vehicle knowledge information through the first preset large - language model to determine the first sub - evaluation result of the vehicle knowledge information in the first preset sub - dimension.

[0081] Specifically, the first preset sub - dimension refers to the authenticity of the vehicle knowledge information, that is, whether the vehicle knowledge information accurately reflects objective facts and whether there are errors that violate common sense.

[0082] The first preset sub - evaluation rule refers to analyzing vehicle knowledge information through a preset large - language model to identify whether there are factual errors that violate common sense, and scoring according to the number of errors. In some embodiments, the first preset sub - evaluation rule is: "If there are 4 or more factual errors that violate common sense in the vehicle knowledge information, give 1 point; if there are 3 factual errors that violate common sense in the vehicle knowledge information, give 2 points; if there are 2 factual errors that violate common sense in the vehicle knowledge information, give 3 points; if there is 1 factual error that violates common sense in the vehicle knowledge information, give 4 points; if there are no factual errors that violate common sense in the vehicle knowledge information, give 5 points."

[0083] A factual error refers to a description or statement that does not conform to objective facts, including errors that violate common sense and errors that do not conform to facts, etc.

[0084] The first prompt refers to the text that guides the first preset large - language model to evaluate the authenticity of vehicle knowledge information, that is, analyzing vehicle knowledge information through the first preset large - language model to identify whether there are errors that violate common sense, scoring according to the number of errors, and determining the authenticity of vehicle knowledge information. The factual error existence situation refers to the number of factual errors in the vehicle knowledge information.

[0085] The first preset large - language model refers to a pre - trained large - language model that can be used to evaluate the authenticity of vehicle knowledge information.

[0086] The server constructs the first prompt according to the first preset evaluation rule. The first prompt is configured to guide the first preset large - language model to judge the factual error existence situation of vehicle knowledge information. For example, the first prompt can be: You are a helpful assistant and you will receive a piece of vehicle knowledge information. Please provide a "score" for the authenticity of this piece of vehicle knowledge information. Please give your scoring answer in the range of 1 to 5, where 1 means there are more than 4 errors in the description, and 5 means there are no errors in the description.

[0087] The following is the scale you should use to construct your answer:

[0088] If there are 4 or more factual errors that violate common sense in the vehicle knowledge information, give 1 point; if there are 3 factual errors that violate common sense in the vehicle knowledge information, give 2 points; if there are 2 factual errors that violate common sense in the vehicle knowledge information, give 3 points; if there is 1 factual error that violates common sense in the vehicle knowledge information, give 4 points; if there are no factual errors that violate common sense in the vehicle knowledge information, give 5 points

[0089] Please provide your feedback in the following way:

[0090] First scoring description: (Your reason for rating, as text)

[0091] First - dimension quality score: (Your score, a number between 1 and 5).

[0092] Next, based on the first prompt, the server evaluates the vehicle knowledge information through the first pre - set large - language model to determine the first sub - evaluation result of the vehicle knowledge information in the first pre - set sub - dimension. The first sub - evaluation result includes the first - dimension quality score and the first score description associated with the first - dimension quality score.

[0093] For example, the vehicle knowledge information is "Steering wheel heating: The steering wheel heating function refers to heating the steering wheel to increase its temperature so that the driver's hands won't get cold while driving. When it is turned off, no heating is performed. When it is turned on, the previous gear is used, and the higher the adjusted gear, the higher the temperature of the steering wheel." After the first pre - set large - language model evaluates and processes this vehicle knowledge information, it determines that the first - dimension quality score is "5 points", and the first score description is: There are no errors against common sense mentioned in the vehicle knowledge information. The steering wheel heating function is indeed used to increase the temperature of the steering wheel to prevent the driver from feeling cold hands in cold weather. When it is turned off, no heating is performed. When it is turned on, the previous gear is used, and the higher the adjusted gear, the higher the temperature. These are all common designs. Therefore, this piece of vehicle knowledge information conforms to common sense.

[0094] In this way, the server constructs the first prompt according to the first pre - set evaluation rule. The first prompt is configured to guide the first pre - set large - language model to determine the existence of factual errors in the vehicle knowledge information. Next, based on the first prompt, the server evaluates the vehicle knowledge information through the first pre - set large - language model to determine the first sub - evaluation result of the vehicle knowledge information in the first pre - set sub - dimension. The first sub - evaluation result includes the first - dimension quality score and the first score description associated with the first - dimension quality score. In this way, by constructing a specific first prompt, it is possible to guide the large - language model to utilize its powerful semantic understanding and reasoning capabilities to accurately identify factual errors in the vehicle knowledge information, give a quality score, and provide a detailed score description to help users understand the evaluation result.

[0095] Please refer to Figure 4 , in some embodiments, the pre - set vehicle knowledge evaluation dimension further includes a second pre - set sub - dimension, the pre - set evaluation rule further includes a second pre - set sub - evaluation rule corresponding to the second pre - set sub - dimension, and step 021 (determining the sub - evaluation result of the vehicle knowledge information in each pre - set sub - dimension according to the vehicle knowledge information and the pre - set sub - evaluation rule) further includes:

[0096] 0213: Construct a second prompt according to the second pre - set evaluation rule;

[0097] 0214: Based on the second prompt, evaluate the vehicle knowledge information through the first preset large language model to determine the second sub-evaluation result of the vehicle knowledge information in the second preset sub-dimension.

[0098] In some embodiments, the construction module is further configured to construct the second prompt according to the second preset evaluation rule. The quality evaluation module is further configured to evaluate the vehicle knowledge information through the first preset large language model based on the second prompt to determine the second sub-evaluation result of the vehicle knowledge information in the second preset sub-dimension.

[0099] In some embodiments, the processor is further configured to construct the second prompt according to the second preset evaluation rule. And evaluate the vehicle knowledge information through the first preset large language model based on the second prompt to determine the second sub-evaluation result of the vehicle knowledge information in the second preset sub-dimension.

[0100] Specifically, the second preset sub-dimension refers to the objectivity of the vehicle knowledge information, that is, whether the vehicle knowledge information is based on objective facts and evidence, without including personal subjective judgments, emotions or biases. For example, whether the vehicle knowledge information uses words with subjective judgments, whether it uses rhetorical devices such as metaphors and exaggerations, and whether it is mixed with the subjective emotions of the writer.

[0101] The second preset sub-evaluation rule refers to analyzing the vehicle knowledge information through the preset large language model to identify whether there is user subjective feeling information, and scoring according to the presence of the user subjective feeling information. In some embodiments, the first preset sub-evaluation rule is: there is a strong subjective color in the vehicle knowledge information, and the writer is just venting emotions, and there is no objective content, giving 1 point; there is a serious subjective color in the vehicle knowledge information, and the writer is belittling or praising the corresponding function, and it is difficult for the reader to completely exclude it, affecting the objective understanding, giving 2 points; there are subjective evaluations and rhetorical devices in the vehicle knowledge information, with a strong subjective color, but a small amount of objective information is provided, and the reader needs to carefully distinguish, giving 3 points; there are a small number of words with subjective evaluations in the vehicle knowledge information, but the proportion is very small and does not affect the objective understanding of the reader, giving 4 points; the vehicle knowledge information is completely objective, and there is no subjective clarity in it, which can help the reader understand objectively, giving 5 points.

[0102] The second prompt refers to the text that guides the first preset large language model to evaluate the objectivity of the vehicle knowledge information, that is, analyzing the vehicle knowledge information through the first preset large language model to identify whether there is user subjective feeling information, and scoring according to the presence of the user subjective feeling information to determine the objectivity of the vehicle knowledge information. The presence of user subjective feeling information refers to whether the vehicle knowledge information uses words with subjective judgments, whether it uses rhetorical devices such as metaphors and exaggerations, and whether it is mixed with the subjective emotions of the writer.

[0103] The server constructs a second prompt word according to the second preset evaluation rule, and the second prompt word is configured to guide the first preset large language model to determine the existence of user subjective feeling information of vehicle knowledge information. For example, the second prompt word can be:

[0104] You are a helpful assistant and you will receive a piece of vehicle knowledge information. Please give a "score" for the objectivity of this piece of vehicle knowledge information. The objectivity needs to consider the following aspects:

[0105] Subjective evaluation: Whether the writer uses rhetorical devices such as subjective judgment words like "good", "bad", "comfortable", etc.: Whether the writer uses rhetorical devices such as metaphor and hyperbole, making the description show a tendency of subjective emotion: Whether the writer mixes in subjective emotions of admiration or disgust

[0106] Please give your scoring answer in the range of 1 to 5, where 1 means the description has a strong subjective color of the writer, and 5 means the description content is objective and not affected by subjective emotions.

[0107] The following is the scale you should use to construct the answer:

[0108] There is a strong subjective color in the vehicle knowledge information, and the writer is just venting emotions, and there is no objective content in it, give 1 point; There is a serious subjective color in the vehicle knowledge information, and the writer is belittling or praising the corresponding function, and it is difficult for readers to completely exclude it, affecting objective understanding, give 2 points; There are subjective evaluations and rhetorical devices in the vehicle knowledge information, with a strong subjective color, but a small amount of objective information is provided, and readers need to distinguish carefully, give 3 points; There are a small number of words with subjective evaluations in the vehicle knowledge information, but the proportion is very small and does not affect the objective understanding of readers, give 4 points; The vehicle knowledge information is completely objective, and there is no subjective clarity in it, which can help readers understand objectively, give 5 points

[0109] Please provide your feedback in the following way:

[0110] Second scoring description: (Your rating reason, as text)

[0111] Second dimension quality score: (Your score, a number between 1 and 5).

[0112] Next, based on the second prompt word, the server evaluates the vehicle knowledge information through the first preset large language model to determine the second sub-evaluation result of the vehicle knowledge information in the second preset sub-dimension. The second sub-evaluation result includes the second dimension quality score and the second scoring description associated with the second dimension quality score.

[0113] For example, the vehicle knowledge information is "Steering wheel heating: The steering wheel heating function refers to a great function that can heat the steering wheel, making me feel comfortable when driving, but when the gear is too high, it will be hot." After the first preset large language model evaluates and processes the vehicle knowledge information, it determines that the second dimension quality score is "2 points", and the second score description is "In the vehicle knowledge information, the author uses words such as 'great' and 'feeling very comfortable', which are obviously words with subjective judgments and reflect the author's personal preferences. In addition, the sentence 'will be hot' also contains the author's personal experience and feelings, which can be regarded as an expression of subjective emotions, because not all users will have the same feelings in the same situation. Although the description provides some actual information about the steering wheel heating function, it tends to express personal opinions rather than objective facts as a whole. No obvious rhetorical devices are used, such as metaphors or exaggerations, but due to the presence of more subjective evaluations and emotions, this may affect the reader's objective understanding of the function."

[0114] In this way, the server constructs a second prompt word according to the second preset evaluation rule, and the second prompt word is configured to guide the first preset large language model to determine the existence of user subjective feeling information in the vehicle knowledge information. Then, based on the second prompt word, the server evaluates the vehicle knowledge information through the first preset large language model, and determines the second sub-evaluation result of the vehicle knowledge information in the second preset sub-dimension. The second sub-evaluation result includes the second dimension quality score and the second score description associated with the second dimension quality score. In this way, by constructing a specific second prompt word, it is possible to guide the large language model to use its own powerful semantic understanding and reasoning capabilities to accurately identify the user's subjective feeling information in the vehicle knowledge information, give a quality score, and provide a detailed score description to help users understand the evaluation results.

[0115] See also Figure 5 In some embodiments, the preset vehicle knowledge evaluation dimension further includes a third preset sub-dimension, and the preset evaluation rule further includes a third preset sub-evaluation rule corresponding to the third preset sub-dimension. Step 021 (determining a sub-evaluation result of the vehicle knowledge information in each preset sub-dimension according to the vehicle knowledge information and the preset sub-evaluation rule) further includes:

[0116] 0215: constructing a third prompt word according to a third preset evaluation rule;

[0117] 0216: Based on the third prompt word, the vehicle knowledge information is evaluated by using the first preset large language model to determine a third sub-evaluation result of the vehicle knowledge information in a third preset sub-dimension.

[0118] In some embodiments, the building module is further configured to construct a third prompt word according to a third preset evaluation rule. The quality evaluation module is further configured to evaluate the vehicle knowledge information through a first preset large language model based on the third prompt word, and determine a third sub-evaluation result of the vehicle knowledge information in a third preset sub-dimension.

[0119] In some embodiments, the processor is further configured to construct a third prompt word according to a third preset evaluation rule. And evaluate the vehicle knowledge information through a first preset large language model based on the third prompt word, and determine a third sub-evaluation result of the vehicle knowledge information in a third preset sub-dimension.

[0120] Specifically, the third preset sub-dimension refers to the comprehensiveness of the vehicle knowledge information, that is, whether the vehicle knowledge information covers all functional description elements related to the function, including associated devices, usage effects, option differences, application scenarios, and taboo scenarios, etc. The associated device refers to whether it is clearly stated in the vehicle knowledge information what device it is related to. If there is no corresponding device for the function itself, it does not need to be described. The usage effect refers to whether it is clearly stated in the vehicle knowledge information what effect can be achieved by executing this function. The option difference refers to whether there are different options for this function. If so, whether the differences between each option are clear. If not, it does not need to be described. The application scenario refers to whether it is clearly stated in the vehicle knowledge information under what scenario this function is usually used. The taboo scenario refers to whether it is clearly stated in the vehicle knowledge information the negative impact of executing this function and under what scenario it is not recommended to use.

[0121] The third preset sub-evaluation rule refers to analyzing the vehicle knowledge information through a preset large language model, identifying the presence of functional description elements therein, and scoring according to the presence of functional description elements. In some embodiments, the first preset sub-evaluation rule is "if there are 4 or more missing functional description elements in the vehicle knowledge information, give 1 point; if there are 3 missing functional description elements in the description of the vehicle knowledge information, give 2 points; if there are 2 missing functional description elements in the vehicle knowledge information, give 3 points; if there is 1 missing functional description element in the vehicle knowledge information, give 4 points; if the vehicle knowledge information is completely correct, including all functional description elements, give 5 points".

[0122] The third prompt word refers to the text that guides the first preset large language model to evaluate the comprehensiveness of the vehicle knowledge information, that is, analyzing the vehicle knowledge information through the first preset large language model, identifying the presence of functional description elements therein, and scoring according to the presence of functional description elements to determine the comprehensiveness of the vehicle knowledge information. The presence of functional description elements refers to whether the vehicle knowledge information includes all functional description elements related to the function.

[0123] The server constructs a third prompt word according to the third preset evaluation rule. The third prompt word is configured to guide the first preset large language model to determine the existence of functional description elements of vehicle knowledge information. For example, the third prompt word can be: You are a helpful assistant and you will receive a piece of vehicle knowledge information. Please provide a "score" for the comprehensiveness of this piece of vehicle knowledge information. The comprehensiveness needs to consider the following aspects:

[0124] Associated device: Whether it is clearly stated in the description what device it is related to. If there is no corresponding device for the function itself, there is no need to describe it.

[0125] Usage effect: Whether it is clearly stated in the description what effect can be achieved by executing this function.

[0126] Option difference: Whether there are different options for this function. If so, whether the differences between each option are clear. If not, there is no need to describe it.

[0127] Usage scenario: Whether it is clearly stated in the description under what scenario this function is usually used.

[0128] Taboo scenario: Whether it is clearly stated in the description the negative impact of executing this function and under what scenario it is not recommended to use.

[0129] Please give your scoring answer in the range of 1 to 5, where 1 means that there are more than 4 unclear statements in the description, and 5 means that the description is completely clear.

[0130] The following is the scale you should use to construct the answer:

[0131] If there are 4 or more missing functional description elements in the vehicle knowledge information, give 1 point; if there are 3 missing functional description elements in the description of the vehicle knowledge information, give 2 points; if there are 2 missing functional description elements in the vehicle knowledge information, give 3 points; if there is 1 missing functional description element in the vehicle knowledge information, give 4 points; if the vehicle knowledge information is completely correct, including all functional description elements, give 5 points.

[0132] Please provide your feedback in the following way:

[0133] Third scoring description: (Your rating reason, as text)

[0134] Third dimension quality score: (Your score, a number between 1 and 5).

[0135] Next, based on the third prompt word, the server evaluates the vehicle knowledge information through the first preset large language model to determine the third sub-evaluation result of the vehicle knowledge information in the third preset sub-dimension. The third sub-evaluation result includes the third dimension quality score and the third scoring description associated with the third dimension quality score.

[0136] For example, the vehicle knowledge information is "Air conditioner rapid cooling: Air conditioner rapid cooling is a function of the air conditioning equipment, which has two states: on or off. When it is on, it can adjust the temperature inside the vehicle to a suitable temperature in a very short time. It is not suitable to use when the vehicle battery power is low." After the first preset large language model evaluates and processes this vehicle knowledge information, it determines that the quality score for the third dimension is "3 points", and the third scoring description is "Based on the provided vehicle knowledge information, we can analyze as follows:

[0137] Associated equipment: The vehicle knowledge information mentions "air conditioning equipment", so this point is clear.

[0138] Usage effect: The vehicle knowledge information mentions that this function "can adjust the temperature inside the vehicle to a suitable temperature in a very short time", so this point is also clear.

[0139] Option differences: The vehicle knowledge information mentions "has two states: on or off", but does not further explain other possible options or differences between these two states, such as whether the cooling speed can be adjusted or specific temperatures can be set. This makes the vehicle knowledge information about option differences incomplete.

[0140] Usage scenarios: The vehicle knowledge information does not specifically point out in what scenarios this function is usually used, such as in high-temperature weather, after long-term parking, etc.

[0141] Taboo scenarios: The vehicle knowledge information mentions "it is not suitable to use when the vehicle battery power is low", and this part of the content provides a scenario where it is not recommended to use.

[0142] In summary, there are two key omissions in the vehicle knowledge information: option differences and usage scenarios are not fully elaborated."

[0143] In this way, the server constructs a third prompt word according to the third preset evaluation rule. The third prompt word is configured to guide the first preset large language model to distinguish the presence of functional description elements in the vehicle knowledge information. Then, based on the third prompt word, the server evaluates the vehicle knowledge information through the first preset large language model to determine the third sub-evaluation result of the vehicle knowledge information in the third preset sub-dimension. The third sub-evaluation result includes the quality score for the third dimension and the third scoring description associated with the quality score for the third dimension. In this way, by constructing a specific third prompt word, it is possible to guide the large language model to utilize its own powerful semantic understanding and reasoning capabilities to accurately identify the functional description element information in the vehicle knowledge information, give a quality score, and provide a detailed scoring description to help users understand the evaluation result.

[0144] Please refer to Figure 6, in some embodiments, the preset vehicle knowledge evaluation dimension further includes a fourth preset sub-dimension, the preset evaluation rule further includes a fourth preset sub-evaluation rule corresponding to the fourth preset sub-dimension, and step 021 (determining the sub-evaluation results of the vehicle knowledge information in each preset sub-dimension according to the vehicle knowledge information and the preset sub-evaluation rules) further includes:

[0145] 0217: Construct a fourth prompt word according to the fourth preset evaluation rule;

[0146] 0218: Based on the fourth prompt word, evaluate the vehicle knowledge information through the first preset large language model to determine the fourth sub-evaluation result of the vehicle knowledge information in the fourth preset sub-dimension.

[0147] In some embodiments, the construction module is further configured to construct a fourth prompt word according to the fourth preset evaluation rule. The quality evaluation module is further configured to evaluate the vehicle knowledge information through the first preset large language model based on the fourth prompt word to determine the fourth sub-evaluation result of the vehicle knowledge information in the fourth preset sub-dimension.

[0148] In some embodiments, the processor is further configured to construct a fourth prompt word according to the fourth preset evaluation rule. And based on the fourth prompt word, evaluate the vehicle knowledge information through the first preset large language model to determine the fourth sub-evaluation result of the vehicle knowledge information in the fourth preset sub-dimension.

[0149] Specifically, the fourth preset sub-dimension refers to the clarity of the vehicle knowledge information, that is, whether the vehicle knowledge information includes semantically ambiguous information, that is, whether it includes qualifying ambiguous information such as approximately or possibly, or key information expressions that are unclear, such as the specific application method of the function is unclear.

[0150] The fourth preset sub-evaluation rule refers to analyzing the vehicle knowledge information through a preset large language model, identifying the presence of semantically ambiguous information therein, and scoring according to the presence of semantically ambiguous information. In some embodiments, the first preset sub-evaluation rule is: "The function description in the vehicle knowledge information is very poor: completely irrelevant to the function itself, and no useful knowledge can be obtained, score 1 point; most of the content of the function description in the vehicle knowledge information is unhelpful: a large number of ambiguous statements are used in the description, and only a small amount of exact information can be obtained, score 2 points; there are some ambiguities in the function description in the vehicle knowledge information: important information can be obtained from the description, but there are multiple unclear specific values or contents, score 3 points; the function description of the vehicle knowledge information is relatively clear: a large amount of useful information can be provided, but there is still room for improvement, score 4 points; the function description in the vehicle knowledge information is very good: relevant, direct, detailed, and can help users quickly and correctly understand how to use the function, score 5 points".

[0151] The fourth prompt refers to the text that guides the first pre-set large language model to evaluate the clarity of vehicle knowledge information. That is, by analyzing the vehicle knowledge information through the first pre-set large language model, identifying the existence of semantically ambiguous information therein, and scoring according to the existence of semantically ambiguous information to determine the clarity of the vehicle knowledge information. The existence of semantically ambiguous information refers to whether there is some content in the vehicle knowledge information that is not clearly expressed and is likely to cause ambiguity or misunderstanding.

[0152] The server constructs the fourth prompt according to the fourth pre-set evaluation rule, and the fourth prompt is configured to guide the first pre-set large language model to judge the existence of semantically ambiguous information in the vehicle knowledge information. For example, the fourth prompt can be: You are a helpful assistant and you will receive a piece of vehicle knowledge information. Please provide a "score" for the clarity of this piece of vehicle knowledge information. The clarity needs to consider the following aspects:

[0153] Limited ambiguity: If it contains vague qualifiers such as "about", "possibly", "seemingly", etc.

[0154] Information ambiguity: There is ambiguity in the expression of key information, which affects understanding. For example, the specific application method of the function is not clear.

[0155] Please give your scoring answer in the range of 1 to 5, where 1 means there are more than 4 unclear expressions in the description, and 5 means the description is completely clear.

[0156] The following is the scale you should use to construct the answer:

[0157] The function description in the vehicle knowledge information is very poor: completely irrelevant to the function itself, and no useful knowledge can be obtained, score 1; most of the content of the function description in the vehicle knowledge information is unhelpful: a large number of vague statements are used in the description, and only a small amount of exact information can be obtained, score 2; there is some ambiguity in the function description in the vehicle knowledge information: important information can be obtained from the description, but there are multiple unclear specific values or contents, score 3; the function description of the vehicle knowledge information is relatively clear: a large amount of useful information can be provided, but there is still room for improvement, score 4; the function description in the vehicle knowledge information is very good: relevant, direct, detailed, and can help users quickly and correctly understand how to use the function, score 5.

[0158] Please provide your feedback in the following way:

[0159] Fourth scoring description: (Your rating reason, as text)

[0160] Fourth dimension quality score: (Your score, a number between 1 and 5).

[0161] Next, based on the fourth prompt, the server evaluates the vehicle knowledge information through the first pre-set large language model to determine the fourth sub-evaluation result of the vehicle knowledge information in the fourth pre-set sub-dimension. The fourth sub-evaluation result includes the fourth dimension quality score and the fourth score description associated with the fourth dimension quality score.

[0162] For example, the vehicle knowledge information is "Air conditioner rapid cooling: The rapid cooling switch is used to turn on or off the rapid cooling function of the in-vehicle air conditioner, which can adjust the in-vehicle temperature to a suitable temperature in a very short time." After the first pre-set large language model evaluates and processes this vehicle knowledge information, it determines that the fourth dimension quality score is "4 points", and the fourth score description is "The vehicle knowledge information does not use fuzzy qualifiers and explains that the function of the rapid cooling switch is to quickly adjust the in-vehicle temperature. However, the vehicle knowledge information does not mention how long 'a very short time' is specifically, nor does it explain what'suitable temperature' is specifically. The lack of these two key pieces of information affects the clarity of the vehicle knowledge information."

[0163] In this way, the server constructs the fourth prompt according to the fourth pre-set evaluation rule. The fourth prompt is configured to guide the first pre-set large language model to determine the existence of semantic fuzzy information in the vehicle knowledge information. Next, based on the fourth prompt, the server evaluates the vehicle knowledge information through the first pre-set large language model to determine the fourth sub-evaluation result of the vehicle knowledge information in the fourth pre-set sub-dimension. The fourth sub-evaluation result includes the fourth dimension quality score and the fourth score description associated with the fourth dimension quality score. In this way, by constructing a specific fourth prompt, it is possible to guide the large language model to utilize its own powerful semantic understanding and reasoning capabilities to accurately identify the semantic fuzzy information in the vehicle knowledge information, give a quality score, and provide a detailed score description to help users understand the evaluation result.

[0164] Please refer to Figure 7 , in some embodiments, the pre-set vehicle knowledge evaluation dimension further includes a fifth pre-set sub-dimension, the pre-set evaluation rule further includes a fifth pre-set sub-evaluation rule corresponding to the fifth pre-set sub-dimension, and step 021 (determining the sub-evaluation result of the vehicle knowledge information in each pre-set sub-dimension according to the vehicle knowledge information and the pre-set sub-evaluation rule) further includes:

[0165] 0219: Construct a fifth prompt according to the fifth pre-set evaluation rule;

[0166] 0250: Based on the fifth prompt, evaluate the vehicle knowledge information through the first pre-set large language model to determine the fifth sub-evaluation result of the vehicle knowledge information in the fifth pre-set sub-dimension.

[0167] In some embodiments, the building module is further configured to construct a fifth prompt word according to a fifth preset evaluation rule. The quality evaluation module is further configured to evaluate the vehicle knowledge information through a first preset large language model based on the fifth prompt word, and determine a fifth sub-evaluation result of the vehicle knowledge information in a fifth preset sub-dimension.

[0168] In some embodiments, the processor is further configured to construct a fifth prompt word according to a fifth preset evaluation rule. And evaluate the vehicle knowledge information through a first preset large language model based on the fifth prompt word, and determine a fifth sub-evaluation result of the vehicle knowledge information in a fifth preset sub-dimension.

[0169] Specifically, the fifth preset sub-dimension refers to the conciseness of the vehicle knowledge information, that is, whether the vehicle knowledge information includes redundant information, that is, whether the vehicle knowledge information is centered around the target content, whether there is irrelevant content, or there is redundant nonsense.

[0170] The fifth preset sub-evaluation rule refers to analyzing the vehicle knowledge information through a preset large language model, identifying the presence of redundant information therein, and scoring according to the presence of redundant information. In some embodiments, the first preset sub-evaluation rule is: "If all the function descriptions in the vehicle knowledge information are nonsense and there is no useful content, give 1 point; if most of the function descriptions in the vehicle knowledge information are nonsense and redundant content, and there is only a small amount of information, give 2 points; if there is serious redundancy in the function descriptions in the vehicle knowledge information: there is irrelevant content and redundant statements in the description, but the proportion is not high, give 3 points; the function description of the vehicle knowledge information is relatively concise: there is no irrelevant content in the description, but there is a small amount of description with redundancy that can be expressed in a more concise way, give 4 points; the function description in the vehicle knowledge information is very concise: each sentence is extremely concise, with clear key points and prominent content, give 5 points".

[0171] The fifth prompt word refers to the text that guides the first preset large language model to evaluate the conciseness of the vehicle knowledge information, that is, analyzing the vehicle knowledge information through the first preset large language model, identifying the presence of redundant information therein, and scoring according to the presence of redundant information to determine the conciseness of the vehicle knowledge information. The presence of redundant information refers to the content in the vehicle knowledge information that is irrelevant to the function and theme. For example, when describing the function of the air conditioner to cool down rapidly, adding memories of the past air conditioner effect and personal feelings, these contents are irrelevant to the current function to be described and belong to irrelevant content. Or the content that could have been described clearly in one sentence in the vehicle knowledge information is forcibly added with redundant information. For example, using too many adjectives or adverbs in the description, or repeating the same meaning, all belong to redundant nonsense.

[0172] The server constructs a fifth prompt word according to the fifth preset evaluation rule, and the fifth prompt word is configured to guide the first preset large language model to determine the existence of redundant information in the vehicle knowledge information. For example, the fifth prompt word can be: You are a helpful assistant, and you will receive a piece of vehicle knowledge information. Please provide a "score" for the conciseness of this piece of vehicle knowledge information. The conciseness needs to consider the following aspects:

[0173] Irrelevant content: If there is content in the description that is irrelevant to the function or theme, it affects conciseness

[0174] Redundant nonsense: If the content that could have been stated in one sentence in the description is forcibly added with redundant information, it is considered to affect conciseness

[0175] Please give your scoring answer in the range of 1 to 5, where 1 means the description content is concise and 5 means the description is full of nonsense

[0176] The following is the scale you should use to construct the answer:

[0177] If all the functional descriptions in the vehicle knowledge information are nonsense and there is no useful content, give 1 point; if most of the functional descriptions in the vehicle knowledge information are nonsense and redundant content, and there is only a small amount of information, give 2 points; if there is serious redundancy in the functional descriptions in the vehicle knowledge information: there is irrelevant content and redundant statements in the description, but the proportion is not high, give 3 points; if the functional description of the vehicle knowledge information is relatively concise: there is no irrelevant content in the description, but there is a small amount of description with redundancy that can be expressed in a more concise way, give 4 points; if the functional description in the vehicle knowledge information is very concise: each sentence is extremely concise content, with clear key points and prominent content, give 5 points

[0178] Please provide your feedback in the following way:

[0179] Fifth scoring description: (Your rating reason, as text)

[0180] Fifth dimension quality score: (Your score, a number between 1 and 5).

[0181] Next, based on the fifth prompt word, the server evaluates the vehicle knowledge information through the first preset large language model to determine the fifth sub-evaluation result of the vehicle knowledge information in the fifth preset sub-dimension. The fifth sub-evaluation result includes the fifth dimension quality score and the fifth scoring description associated with the fifth dimension quality score

[0182] For example, the vehicle knowledge information is "Rapid cooling of the air conditioner: This function of rapid cooling of the air conditioner can adjust the temperature inside the vehicle to a suitable temperature in an extremely short time. Modern vehicles are much more convenient than before. I remember that in vehicles ten years ago, if the air conditioner didn't work well, the seat cushion would still be scalding hot for a long time after you got in the car, not to mention the strange smell in the car caused by the sun shining in, which was really suffocating." After the first preset large language model evaluates and processes this vehicle knowledge information, it determines that the quality score in the fifth dimension is "3 points", and the fifth scoring description is "In the vehicle knowledge information, the core information about the rapid cooling function of the air conditioner is 'can adjust the temperature inside the vehicle to a suitable temperature in an extremely short time'. This part of the content is directly relevant and concise. However, the subsequent narrative turns to the recollection of the air conditioner effect of past vehicles and personal feelings, which are irrelevant to the function of the current vehicle knowledge information, and adds unnecessary details such as scalding hot seat cushions and strange smells in the car, etc. These all affect the conciseness of the vehicle knowledge information. Therefore, there are obvious redundant and irrelevant contents."

[0183] In this way, the server constructs a fifth prompt word according to the fifth preset evaluation rule. The fifth prompt word is configured to guide the first preset large language model to determine the existence of redundant information in the vehicle knowledge information. Then, based on the fifth prompt word, the server evaluates the vehicle knowledge information through the first preset large language model, and determines the fifth sub-evaluation result in the fifth preset sub-dimension of the vehicle knowledge information. The fifth sub-evaluation result includes the quality score in the fifth dimension and the fifth scoring description associated with the quality score in the fifth dimension. In this way, by constructing a specific fifth prompt word, it is possible to guide the large language model to utilize its own powerful semantic understanding and reasoning ability to accurately identify the redundant information in the vehicle knowledge information, give a quality score, and provide a detailed scoring description to help users understand the evaluation result.

[0184] Please refer to Figure 8 , in some embodiments, the method further includes:

[0185] 023: When the quality evaluation result of the vehicle knowledge information is to be optimized, based on the second preset large language model, perform optimization processing on the vehicle knowledge information to determine the optimized vehicle knowledge information;

[0186] 024: Based on the preset vehicle knowledge evaluation dimension, according to the optimized vehicle knowledge information and the preset evaluation rule, perform quality evaluation on the optimized vehicle knowledge information.

[0187] In some embodiments, the quality assessment device further includes an optimization module. The optimization module is configured to, when the quality assessment result of the vehicle knowledge information is to be optimized, optimize the vehicle knowledge information based on a second pre-set large language model to determine the optimized vehicle knowledge information. The quality assessment module is further configured to, based on a pre-set vehicle knowledge assessment dimension, perform a quality assessment on the optimized vehicle knowledge information according to the optimized vehicle knowledge information and the pre-set assessment rules.

[0188] In some embodiments, the processor is further configured to, when the quality assessment result of the vehicle knowledge information is to be optimized, optimize the vehicle knowledge information based on a second pre-set large language model to determine the optimized vehicle knowledge information. And based on a pre-set vehicle knowledge assessment dimension, perform a quality assessment on the optimized vehicle knowledge information according to the optimized vehicle knowledge information and the pre-set assessment rules.

[0189] Specifically, the quality assessment result of the vehicle knowledge information includes qualified vehicle knowledge information, vehicle knowledge information to be optimized, and discarded vehicle knowledge information, which is determined after performing a quality assessment process on the vehicle knowledge information through a first pre-set large language model.

[0190] The second pre-set large language model refers to the large language model used to optimize the vehicle knowledge information and is used when the assessment result is to be optimized. In some embodiments, the first pre-set large language model and the second pre-set large language model may be the same large language model that has been trained, or different large language models obtained according to different training methods, which is not limited herein.

[0191] After determining the quality assessment result of the vehicle knowledge information according to the first pre-set large language model, use the second pre-set large language model to optimize the vehicle knowledge information with a quality assessment result of vehicle knowledge information to be optimized to determine the optimized vehicle knowledge information.

[0192] Subsequently, again based on the pre-set vehicle knowledge assessment dimension, the server uses the pre-set assessment rules and the first pre-set large language model to perform a quality assessment on the optimized vehicle knowledge information to obtain a new assessment result.

[0193] In this way, when the quality assessment result of the vehicle knowledge information is to be optimized, based on the second pre-set large language model, the server optimizes the vehicle knowledge information to determine the optimized vehicle knowledge information. Then, based on the pre-set vehicle knowledge assessment dimension, the server performs a quality assessment on the optimized vehicle knowledge information according to the optimized vehicle knowledge information and the pre-set assessment rules. In this way, by optimizing the vehicle knowledge information to be optimized and re-performing the quality assessment process, the number of vehicle knowledge information that meets the quality requirements can be increased.

[0194] Please refer to Figure 9, in some embodiments, step 023 (optimizing the vehicle knowledge information based on the second pre-set large language model to determine the optimized vehicle knowledge information) includes:

[0195] 0231: Perform fact error removal processing, rewriting processing, and / or information combination processing on the vehicle knowledge information to determine the optimized vehicle knowledge information.

[0196] In some embodiments, the processing module is further configured to perform fact error removal processing, rewriting processing, and / or information combination processing on the vehicle knowledge information to determine the optimized vehicle knowledge information.

[0197] In some embodiments, the processor is further configured to perform fact error removal processing, rewriting processing, and / or information combination processing on the vehicle knowledge information to determine the optimized vehicle knowledge information.

[0198] Specifically, fact error removal processing refers to discovering and removing the incorrect parts in the vehicle knowledge information. By using the second pre-set large language model to analyze the knowledge, the content that does not conform to the facts is identified and deleted or corrected. For example, if the knowledge information mentions that a certain vehicle model "only takes 1 second for 0-100 km / h acceleration", while in fact the 0-100 km / h acceleration time of this vehicle model is much longer than 1 second, the large language model will identify this incorrect information and correct or delete it.

[0199] Rewriting processing refers to, for the deficiencies in the three aspects of the second pre-set sub-dimension, the fourth pre-set sub-dimension, and the fifth pre-set sub-dimension of the vehicle knowledge information, using the second pre-set large language model to rewrite the relevant parts of the vehicle knowledge information to improve the objectivity, clarity, and conciseness of the vehicle knowledge information. For example, if the knowledge information uses a subjective evaluation word such as "this function is very good", the large language model can be used to rewrite it as "this function can effectively improve the handling performance of the vehicle".

[0200] Information combination processing refers to randomly combining the low-comprehensiveness knowledge data related to the same function to improve the comprehensiveness of the vehicle knowledge information. For example, if two pieces of knowledge information respectively describe the activation method and usage scenario of a certain function, but both lack the description of the function effect, these two pieces of information can be combined to make it more comprehensive.

[0201] Use the first pre-set large language model to identify the content to be optimized in the vehicle knowledge information, such as fact errors, subjective evaluations, fuzzy information, redundant information, etc.

[0202] For the content to be optimized in the first pre-set sub-dimension of the vehicle knowledge information, use the second pre-set large language model to remove the identified fact errors to ensure the authenticity of the knowledge.

[0203] For the content to be optimized in the second preset sub-dimension, fourth preset sub-dimension, and fifth preset sub-dimension of vehicle knowledge information, use the second preset large language model to rewrite the identified subjective evaluations, fuzzy information, redundant information, etc., to make it more objective, clear, and concise.

[0204] For the content to be optimized in the third preset sub-dimension of vehicle knowledge information, use the second preset large language model to randomly combine the low-comprehensiveness knowledge data related to the same function.

[0205] In this way, the server performs fact error removal processing, rewriting processing, and / or information combination processing on the vehicle knowledge information to determine the optimized vehicle knowledge information. In this way, by using the second preset large language model to identify the content to be optimized in the vehicle knowledge information and performing optimization processing on the content to be optimized, the quality of the vehicle knowledge information can be improved.

[0206] This application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the vehicle knowledge quality assessment method as described above are implemented.

[0207] It can be understood that the computer program includes computer program code. The computer program code can be in the form of source code, object code, executable file, or some intermediate form, etc. The computer-readable storage medium can include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disc, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), and software distribution media, etc.

[0208] In the description of this specification, the descriptions referring to terms such as "specifically", "furthermore", "specially", "understandably", etc. mean that the specific features, structures, materials, or characteristics described in combination with the embodiments or examples are included in at least one embodiment or example of this application. In this specification, the schematic expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in a suitable manner in any one or more embodiments or examples. In addition, without contradiction, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0209] Any process or method description, whether in a flowchart or otherwise described herein, can be understood to represent a module, segment, or portion of executable code that includes one or more steps for implementing a specific logical function or process, and the scope of the preferred embodiments of the present application includes additional implementations, where the functions may be performed in a substantially simultaneous manner or in the reverse order according to the functions involved, rather than in the order shown or discussed, which should be understood by those skilled in the technical field to which the embodiments of the present application pertain.

[0210] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and should not be construed as limiting the present application. Those of ordinary skill in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present application.

Claims

1. A vehicle knowledge quality assessment method, characterized in that: The method comprises: Obtain vehicle knowledge information; Based on a preset vehicle knowledge evaluation dimension, the vehicle knowledge information is quality evaluated according to the vehicle knowledge information and a preset evaluation rule associated with the preset vehicle knowledge evaluation dimension.

2. The method according to claim 1, characterized in that The preset vehicle knowledge evaluation dimension includes a plurality of preset sub-dimensions, the preset evaluation rule includes a plurality of groups of preset sub-evaluation rules, each of the preset sub-dimensions is associated with a group of the preset sub-evaluation rules, and the quality evaluation of the vehicle knowledge information based on the preset vehicle knowledge evaluation dimension according to the vehicle knowledge information and the preset evaluation rules associated with the preset vehicle knowledge evaluation dimension includes: Determining a sub-evaluation result of the vehicle knowledge information in each of the preset sub-dimensions according to the vehicle knowledge information and the preset sub-evaluation rules; Based on preset quality assessment rules, quality assessment is performed on the vehicle knowledge information according to the plurality of sub-assessment results.

3. The method according to claim 2, characterized in that The preset vehicle knowledge evaluation dimension includes a first preset sub-dimension, the preset evaluation rule includes a first preset sub-evaluation rule corresponding to the first preset sub-dimension, and determining a sub-evaluation result of the vehicle knowledge information in each of the preset sub-dimensions according to the vehicle knowledge information and the preset sub-evaluation rule includes: Constructing a first prompt word according to the first preset evaluation rule, wherein the first prompt word is configured to guide a first preset large language model to determine whether there is a factual error in the vehicle knowledge information; Based on the first prompt word, the vehicle knowledge information is evaluated by the first preset large language model to determine a first sub-evaluation result of the vehicle knowledge information in the first preset sub-dimension, wherein the first sub-evaluation result includes a first dimension quality score and a first score description associated with the first dimension quality score.

4. The method according to claim 3, characterized in that The preset vehicle knowledge evaluation dimension further includes a second preset sub-dimension, the preset evaluation rule further includes a second preset sub-evaluation rule corresponding to the second preset sub-dimension, and determining the sub-evaluation result of the vehicle knowledge information in each of the preset sub-dimensions according to the vehicle knowledge information and the preset sub-evaluation rule further includes: constructing a second prompt word according to the second preset evaluation rule, wherein the second prompt word is configured to guide the first preset large language model to determine the existence of user subjective feeling information of the vehicle knowledge information; Based on the second prompt word, the vehicle knowledge information is evaluated by the first preset large language model to determine a second sub-evaluation result of the vehicle knowledge information in the second preset sub-dimension, wherein the second sub-evaluation result includes a second dimension quality score and a second score description associated with the second dimension quality score.

5. The method according to claim 4, characterized in that The preset vehicle knowledge evaluation dimension further includes a third preset sub-dimension, the preset evaluation rule further includes a third preset sub-evaluation rule corresponding to the third preset sub-dimension, and determining the sub-evaluation result of the vehicle knowledge information in each of the preset sub-dimensions according to the vehicle knowledge information and the preset sub-evaluation rule further includes: constructing a third prompt word according to the third preset evaluation rule, wherein the third prompt word is configured to guide the first preset large language model to determine the existence of a functional description element of the vehicle knowledge information; Based on the third prompt word, the vehicle knowledge information is evaluated by the first preset large language model to determine a third sub-evaluation result of the vehicle knowledge information in the third preset sub-dimension, wherein the third sub-evaluation result includes a third dimension quality score and a third score description associated with the third dimension quality score.

6. The method according to claim 5, characterized in that The preset vehicle knowledge evaluation dimension further includes a fourth preset sub-dimension, the preset evaluation rule further includes a fourth preset sub-evaluation rule corresponding to the fourth preset sub-dimension, and determining the sub-evaluation result of the vehicle knowledge information in each of the preset sub-dimensions according to the vehicle knowledge information and the preset sub-evaluation rule further includes: Constructing a fourth prompt word according to the fourth preset evaluation rule, wherein the fourth prompt word is configured to guide the first preset large language model to determine the existence of semantically ambiguous information in the vehicle knowledge information; Based on the fourth prompt word, the vehicle knowledge information is evaluated by the first preset large language model to determine a fourth sub-evaluation result of the vehicle knowledge information in the fourth preset sub-dimension, wherein the fourth sub-evaluation result includes a fourth dimension quality score and a fourth score description associated with the fourth dimension quality score.

7. The method according to claim 6, characterized in that The preset vehicle knowledge evaluation dimension further includes a fifth preset sub-dimension, the preset evaluation rule further includes a fifth preset sub-evaluation rule corresponding to the fifth preset sub-dimension, and determining the sub-evaluation result of the vehicle knowledge information in each of the preset sub-dimensions according to the vehicle knowledge information and the preset sub-evaluation rule further includes: constructing a fifth prompt word according to the fifth preset evaluation rule, wherein the fifth prompt word is configured to guide the first preset large language model to determine the existence of redundant information in the vehicle knowledge information; Based on the fifth prompt word, the vehicle knowledge information is evaluated by the first preset large language model to determine a fifth sub-evaluation result of the vehicle knowledge information in the fifth preset sub-dimension, wherein the fifth sub-evaluation result includes a fifth dimension quality score and a fifth score description associated with the fifth dimension quality score.

8. The method according to claim 2, characterized in that: The method further comprises: When the quality assessment result of the vehicle knowledge information is to be optimized, optimizing the vehicle knowledge information based on a second preset large language model to determine optimized vehicle knowledge information; Based on the preset vehicle knowledge evaluation dimension, the quality of the optimized vehicle knowledge information is evaluated according to the optimized vehicle knowledge information and the preset evaluation rules.

9. The method according to claim 8, characterized in that The optimizing the vehicle knowledge information based on the second preset large language model to determine the optimized vehicle knowledge information includes: The vehicle knowledge information is subjected to factual error removal processing, rewriting processing and / or information combination processing to determine optimized vehicle knowledge information.

10. A server, characterized in that: The server includes a processor and a memory, wherein a computer program is stored in the memory. When the computer program is executed by the processor, the method according to any one of claims 1 to 9 is implemented.

11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

Images