A data processing method, device, computer, storage medium and program product

Abstract

Embodiments of the present application disclose a data processing method and device, a computer, a storage medium and a program product. The method comprises: obtaining a first text result output by a business model for first business data, and a first sequence of logarithmic probability values corresponding to the first text result; performing key-value pair alignment processing on the first text result to obtain a first structured result for the first business data; obtaining, from the first sequence of logarithmic probability values, a first logarithmic probability value segment corresponding to each of N attribute values, performing data integration on the first logarithmic probability value segment corresponding to each attribute value to obtain a first local confidence value corresponding to each of the N attribute values; and performing data statistics on the first local confidence values corresponding to the N attribute values to obtain a first global confidence value for the first text result. According to the present application, whether the first text result output by the business model is reliable can be determined more accurately.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a data processing method, apparatus, computer, storage medium and program product. Background Technology

[0002] With the rapid development of artificial intelligence technology, large models have been widely used in various fields such as natural language processing (NLP), computer vision, and speech recognition. Model confidence assessment, which determines the model's degree of confidence in its predictions, is a crucial step in ensuring model reliability and accuracy, and also a quantitative evaluation method for the reliability of results generated by large models. When large models directly generate text, they often encounter the illusion problem (i.e., generating irrelevant results) or inaccurate results for questions the model itself cannot answer. Therefore, assessing the confidence of text generated by large models is essential. Assessing confidence helps users judge the credibility of the generated text and decide whether to use relevant content. It also helps developers understand model performance, identify tasks or domains where the model performs poorly, and thus improve the model, enhancing its overall quality and usability. Furthermore, providing a separate confidence score for some key fields in the results is necessary, as this allows downstream users to better manage request risks.

[0003] Currently, there are three main types of confidence assessment methods. The first is to average the log probabilities (logprobs) output by the business model to obtain the confidence of the generated text. The second is based on the P(True) method [where P(True), or Probability (True), refers to the probability that an event is predicted to be true]. This method inputs the question from the original model and the generated text into an additional model, and queries the additional model about the reliability of the results given by the original model. Only the additional model is allowed to output True; the log probabilities (logprobs) of the additional model's True output are then calculated to determine the confidence of the generated text. The third is based on consistency assessment. This method allows the model to generate multiple results for the same question simultaneously, and then calculates the similarity between these results. Higher similarity indicates that the model maintains high consistency across different generation processes, resulting in higher confidence for the generated text. However, all three methods directly perform overall statistical analysis on the model's output, which may result in the loss of some information, especially when the model itself has result biases, leading to lower accuracy in determining the confidence of the model's results. Summary of the Invention

[0004] This application provides a data processing method, apparatus, computer, storage medium, and program product that can obtain a global confidence score for structured results and a local confidence score for some key fields (attribute values) in the structured results, thereby enabling a more accurate determination of the credibility of the first text result generated by the business model.

[0005] One embodiment of this application provides a data processing method, the method comprising:

[0006] Obtain the first text result output by the business model for the first business data, and the first log probability value sequence corresponding to the first text result; the first log probability value sequence includes the log probability value corresponding to each character in the first text result;

[0007] The first text result is subjected to key-value pair alignment to obtain the first structured result for the first business data; the first structured result includes N business attributes and the attribute values ​​corresponding to the N business attributes; N is a positive integer;

[0008] From the first log probability value sequence, obtain the first log probability value slices corresponding to N attribute values ​​respectively, integrate the data of the first log probability value slices corresponding to each attribute value, and obtain the first local confidence values ​​corresponding to N attribute values ​​respectively;

[0009] The first local confidence scores corresponding to the N attribute values ​​are statistically analyzed to obtain the first global confidence score for the first text result. The first local confidence score and the first global confidence score are used to represent the credibility of the first text result when data processing is performed based on the first text result.

[0010] One embodiment of this application provides a data processing apparatus, the apparatus comprising:

[0011] The text result acquisition module is used to acquire the first text result output by the business model for the first business data, and the first log probability value sequence corresponding to the first text result; the first log probability value sequence includes the log probability value corresponding to each character in the first text result;

[0012] The alignment module is used to perform key-value pair alignment on the first text result to obtain a first structured result for the first business data; the first structured result includes N business attributes and the attribute values ​​corresponding to the N business attributes; N is a positive integer;

[0013] The local confidence determination module is used to obtain the first log probability value slices corresponding to N attribute values ​​from the first log probability value sequence, integrate the data of the first log probability value slices corresponding to each attribute value, and obtain the first local confidence value corresponding to the N attribute values ​​respectively.

[0014] The global confidence determination module is used to perform data statistics on the first local confidence values ​​corresponding to N attribute values ​​respectively, and obtain the first global confidence value for the first text result; the first local confidence value and the first global confidence value are used to represent the credibility of the first text result when data processing is performed based on the first text result.

[0015] In one possible implementation, the alignment processing module is used to perform key-value pair alignment on the first text result to obtain the first structured result for the first business data. Specifically, the alignment processing module is used to perform the following operations:

[0016] The first text result is parsed character by character to obtain the character type of each character in the first text result;

[0017] Based on the characters of type 1 symbol in the first text result, the first text result is split into N character sequences; the N character sequences include character sequence C. k k is a positive integer less than or equal to N; the first symbol type character is used to distinguish characters associated with different business attributes;

[0018] Character sequence C k The first character preceding the character of type 2 symbol is determined as character sequence C. k The corresponding business attribute will be the character sequence C. k The second character following the character of type 2 symbol is determined as character sequence C. k The attribute value of the corresponding business attribute;

[0019] When the business attributes and attribute values ​​corresponding to N character sequences are obtained, the business attributes and attribute values ​​corresponding to each character sequence are combined to obtain the first structured result for the first business data.

[0020] In one possible implementation, the N attribute values ​​include attribute value A. i , where i is a positive integer less than or equal to N; the local confidence determination module is used to obtain the first logarithmic probability value slices corresponding to N attribute values ​​from the first logarithmic probability value sequence, and to integrate the first logarithmic probability value slices corresponding to each attribute value to obtain the first local confidence values ​​corresponding to N attribute values. Specifically, the local confidence determination module is used to perform the following operations:

[0021] Get attribute value A i The included M characters are used to form attribute value A by combining the logarithmic probability values ​​corresponding to the M characters in the first logarithmic probability value sequence. i The corresponding first logarithmic probability value slice; M is a positive integer;

[0022] For attribute value A i The attribute value A is obtained by averaging the M log probability values ​​in the first log probability value slice. i The corresponding first local confidence value.

[0023] In one possible implementation, the global confidence determination module is used to perform data statistics on the first local confidence values ​​corresponding to the N attribute values ​​respectively. When obtaining the first global confidence value for the first text result, the global confidence determination module is specifically used to perform the following operations:

[0024] Obtain the attribute weights corresponding to N business attributes;

[0025] By using the attribute weights corresponding to N business attributes, the first local confidence scores corresponding to the N attribute values ​​are weighted and summed to obtain the first global confidence score for the first text result.

[0026] In one possible implementation, when the global confidence determination module obtains the attribute weights corresponding to N business attributes, it specifically performs the following operations:

[0027] The N business attributes and the first business data are input into the importance detection model. The importance detection model is used to detect the correlation between the N business attributes and the first business data, and the attribute weights corresponding to the N business attributes are obtained.

[0028] Alternatively, in the importance detection model, the importance of N business attributes is detected based on the first business data to obtain the importance of each business attribute, and an attribute weight is assigned to each business attribute based on the importance of each business attribute.

[0029] In one possible implementation, when the global confidence determination module obtains the attribute weights corresponding to N business attributes, it specifically performs the following operations:

[0030] Obtain the first attribute category corresponding to each of the N business attributes, as well as the preset category weight information; the preset category weight information includes the attribute category and the category weight of the attribute category; the attribute category includes the first attribute category;

[0031] Obtain the category weights of the first attribute category corresponding to each of the N business attributes from the preset category weight information;

[0032] Normalize the category weights corresponding to the N business attributes to obtain the attribute weights corresponding to the N business attributes.

[0033] In one possible implementation, the N business attributes include business attribute B. i N attribute values, including attribute value A i Attribute value A i For business attribute B i The corresponding attribute value, i, is a positive integer less than or equal to N; the global confidence determination module is used to perform a weighted sum of the first local confidence values ​​corresponding to the N attribute values ​​using the attribute weights corresponding to the N business attributes, in order to obtain the first global confidence value for the first text result. Specifically, the global confidence determination module performs the following operations:

[0034] Obtain business attribute B from the first logarithmic probability value sequence. i The corresponding second logarithmic probability value fragment is then integrated to obtain business attribute B. i The corresponding attribute confidence value;

[0035] Obtain the first weighted weight and the second weighted weight based on business attribute B. i The corresponding attribute weights and the first weighted weight determine the business attribute B. i The attribute weighting is based on business attribute B. i The corresponding attribute weight and second weight are used to determine the attribute value A. i The values ​​are weighted; the first weight is used to represent the weight percentage of the business attribute in the same group of business attributes and attribute values; the second weight is used to represent the weight percentage of the attribute value in the same group of business attributes and attribute values.

[0036] Based on business attribute B i The attribute weights and attribute values ​​A i The value of the weighted weight is applied to business attribute B. i The corresponding attribute confidence value and attribute value A i The corresponding first local confidence scores are weighted and averaged to obtain the result for attribute value A. i The weighted confidence score;

[0037] When the weighted confidence scores corresponding to the N attribute values ​​are obtained, the sum of the N weighted confidence scores is determined as the first global confidence score for the first text result.

[0038] In one possible implementation, the data processing device further includes a business processing result acquisition module, which is specifically used to perform the following operations:

[0039] If the first local confidence value corresponding to the key attribute is less than the local confidence threshold, or the first global confidence value is less than the global confidence threshold, then an abnormal confidence result is determined for the first text result. Based on the abnormal confidence result, a data prompt text is generated. The key attribute refers to the business attribute among N business attributes whose attribute weight is greater than or equal to the key weight threshold. The data prompt text is used to prompt the business object associated with the first business data to input updated business data in the business model.

[0040] If the first local confidence value corresponding to the key attribute is greater than or equal to the local confidence threshold, and the first global confidence value is greater than or equal to the global confidence threshold, then the business processing result for the first business data is obtained based on the first structured result, and the business processing result is output.

[0041] In one possible implementation, the first text result is the request result for the first business data; when the business processing result acquisition module is used to acquire the business processing result for the first business data based on the first structured result, the business processing result acquisition module is specifically used to perform the following operations:

[0042] From the N business attributes in the first structured result, obtain the attribute value corresponding to the first business attribute; the first business attribute refers to the business attribute whose attribute category is result attribute among the N business attributes;

[0043] The attribute value corresponding to the first business attribute is determined as the business processing result for the first business data.

[0044] In one possible implementation, the first text result is the result of intent parsing for the first business data; when the business processing result acquisition module is used to acquire the business processing result for the first business data based on the first structured result, the business processing result acquisition module is specifically used to perform the following operations:

[0045] Obtain H media data associated with the first structured result, and obtain the data correlation degree between each of the H media data and the first structured result; H is a positive integer;

[0046] Based on the data correlation between H media data and the first structured result, the associated media data corresponding to the first structured result is obtained from the H media data;

[0047] Based on the first business data, the associated media data is integrated to obtain the business processing results for the first business data.

[0048] In one possible implementation, the data processing apparatus further includes a confidence adjustment module, which is specifically used to perform the following operations:

[0049] When second business data associated with the first business data is obtained through the business model, the second business data is parsed to generate a second text result for the second business data, as well as a second log probability value sequence corresponding to the second text result;

[0050] Based on the second text result and the second log probability value sequence, a second global confidence value is generated for the second text result;

[0051] The association attribute values ​​associated with the second text result are obtained from the first structured result. Based on the first local confidence value corresponding to the association attribute value, the second global confidence value is adjusted to obtain the updated global confidence value for the second text result. The N attribute values ​​include the association attribute values.

[0052] In one possible implementation, when the text result acquisition module is used to acquire the first text result output by the business model for the first business data, and the first logarithmic probability value sequence corresponding to the first text result, the text result acquisition module is specifically used to perform the following operations:

[0053] The first business data and the format prompt instructions for the first business data are input into the business model. The business model performs text splitting on the first business data to obtain the word sequence corresponding to the first business data. The format prompt instructions are used to instruct the business model to output data with a structured format.

[0054] Each word in the word sequence is encoded to obtain the text encoding features of the first business data;

[0055] The format prompt instructions guide the business model to perform text prediction based on text encoding features, generating multiple characters and the corresponding log probability value for each character;

[0056] The first text result is composed of multiple characters, and the log probability value corresponding to each character is used to form the first log probability value sequence corresponding to the first text result.

[0057] One embodiment of this application provides a computer device, including a processor, a memory, and an input / output interface;

[0058] The processor is connected to a memory and an input / output interface, respectively. The input / output interface is used to receive and output data, the memory is used to store computer programs, and the processor is used to call the computer programs so that the computer device containing the processor executes the method in one aspect of the embodiments of this application.

[0059] One aspect of this application provides a computer-readable storage medium storing a computer program adapted to be loaded and executed by a processor, so that a computer device having the processor performs the method of one aspect of this application.

[0060] One aspect of this application provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in various optional embodiments of this application. In other words, when the computer instructions are executed by the processor, they implement the methods provided in various optional embodiments of this application.

[0061] Implementing the embodiments of this application will have the following beneficial effects:

[0062] In this embodiment, a first text result output by the business model for the first business data and a first logarithmic probability value sequence corresponding to the first text result are obtained. The first logarithmic probability value sequence includes the logarithmic probability value corresponding to each character in the first text result. The first text result is subjected to key-value pair alignment processing to obtain a first structured result for the first business data. The first structured result includes N business attributes and attribute values ​​corresponding to the N business attributes respectively. N is a positive integer. From the first logarithmic probability value sequence, the first logarithmic probability value slices corresponding to the N attribute values ​​are obtained. The first logarithmic probability value slices corresponding to each attribute value are integrated to obtain the first local confidence values ​​corresponding to the N attribute values ​​respectively. Data statistics are performed on the first local confidence values ​​corresponding to the N attribute values ​​to obtain the first global confidence value for the first text result. The first local confidence value and the first global confidence value are used to represent the credibility of the first text result when data processing is performed based on the first text result. Through the above process, the first text output of the model is aligned to obtain the first structured result. Based on the first structured result, local and global confidence scores are determined for the business model's output, achieving a structured parsing of the model's output. This means that the confidence score determination not only considers the model's own output but also its structured information, thereby improving the comprehensiveness of the information referenced in the confidence score determination and increasing its accuracy. Furthermore, generating a global confidence score based on the local confidence score ensures that the global confidence score fully considers the structured data composition of the first text output of the business model, achieving attribute-based parsing of the first text output. Therefore, the accuracy and usability of the first text output of the business model can be more accurately evaluated through two dimensions of confidence scores (first local confidence score and first global confidence score). Attached Figure Description

[0063] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0064] Figure 1 This is a network interaction architecture diagram provided in an embodiment of this application;

[0065] Figure 2 This is a schematic diagram of a data processing method provided in an embodiment of this application;

[0066] Figure 3This is a flowchart illustrating a data processing method provided in an embodiment of this application. Figure 1 ;

[0067] Figure 4 This is a schematic diagram of a key-value pair alignment processing scenario provided in an embodiment of this application. Figure 1 ;

[0068] Figure 5 This is a flowchart illustrating a data processing method provided in an embodiment of this application. Figure 2 ;

[0069] Figure 6 This is a schematic diagram of a key-value pair alignment processing scenario provided in an embodiment of this application. Figure 2 ;

[0070] Figure 7 This is a schematic diagram of a weighted processing scenario provided in an embodiment of this application;

[0071] Figure 8 This is a schematic diagram of a data processing device provided in an embodiment of this application;

[0072] Figure 9 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0073] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0074] If this application requires the collection of object data (such as user data), a prompt interface or pop-up window will be displayed before and during the collection process. This prompt interface or pop-up window is used to inform the user that certain data is being collected. The data acquisition steps will only begin after the user confirms the prompt interface or pop-up window; otherwise, the process will end. Furthermore, the acquired user data will be used in reasonable and legal scenarios or for legitimate purposes. Optionally, in scenarios where user data needs to be used but user authorization has not been obtained, authorization can be requested from the user, and the user data can only be used after authorization is granted.

[0075] It is understood that, in the specific embodiments of this application, the user data involved requires user permission or consent when the following embodiments of this application are applied to specific products or technologies, and the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant regions.

[0076] In the embodiments of the present application, please refer to Figure 1 , Figure 1 is a network interaction architecture diagram provided in the embodiments of the present application, as shown in Figure 1 , as shown in Figure 1 , the network interaction architecture diagram may include a computer device 101 and a service device cluster. The service device cluster may include service device 102a, service device 102b, service device 102c, ..., service device 102n, where communication connections may exist between the service devices in the service device cluster, for example, a communication connection exists between service device 102a and service device 102b, and a communication connection exists between service device 102a and service device 102c. Meanwhile, any service device in the service device cluster may have a communication connection with the computer device 101, for example, a communication connection exists between service device 102a and the computer device 101, where the above communication connection is not limited to a specific connection manner, and may be directly or indirectly connected through a wired communication manner, directly or indirectly connected through a wireless communication manner, or connected through other manners, which is not limited in the present application.

[0077] It should be understood that the present data processing method may be executed in any service device in the service device cluster, or may be executed in the computer device 101. Taking Figure 1 the service device 102b shown in as an example, the present data processing method may be executed in the service device 102b. The following description takes the execution of the present data processing method in the computer device 101 as an example. In the embodiments of the present application, a service object may input first service data into a service model, and the computer device 101 may obtain a first text result output by the service model for the first service data, and a first log probability value sequence corresponding to the first text result. The first log probability value sequence includes a log probability value corresponding to each character in the first text result. For example, if the first character in the first text result is "meaning", the first log probability value in the first log probability value sequence is the log probability value corresponding to the character "meaning".

[0078] Furthermore, the computer device 101 can perform key-value pair alignment processing on the first text result to obtain a first structured result for the first business data. The first structured result includes N business attributes and corresponding attribute values ​​for each of the N business attributes, where N is a positive integer. The business attributes refer to the descriptive properties or characteristics corresponding to the first text result; the attribute values ​​are the specific information of the corresponding business attributes. It can be understood that key-value pair alignment processing can involve forming a key-value pair with a business attribute and its corresponding attribute value. That is, in a set of key-value pairs, the business attribute is the data key, and the attribute value is the data value. For example, the first structured result is represented as "{"A1:B1", "A2:B2"}", where A1 and A2 are business attributes, and B1 and B2 are attribute values. "A1:B1" forms key-value pair 1, and "A2:B2" forms key-value pair 2. It can be understood that the first structured result makes the result clearer and more explicit, and because of its standardized format and clear organization, it is easier for the program to understand and process. Computer device 101 can obtain the first log probability value slices corresponding to N attribute values ​​from the first log probability value sequence.

[0079] Furthermore, the computer device 101 can perform data statistics on the first local confidence scores corresponding to the N attribute values ​​to obtain a first global confidence score for the first text result. The first local confidence score and the first global confidence score represent the reliability of the first text result when data processing is performed based on it. They can also be understood as representing the accuracy of the first text result output by the business model.

[0080] Through the above process, the first text output of the model is aligned to obtain the first structured result. Based on the first structured result, local and global confidence scores are determined for the business model's output, achieving a structured parsing of the model's output. This means that the confidence score determination not only considers the model's own output but also its structured information, thereby improving the comprehensiveness of the information referenced in the confidence score determination and increasing its accuracy. Furthermore, generating a global confidence score based on the local confidence score ensures that the global confidence score fully considers the structured data composition of the first text output of the business model, achieving attribute-based parsing of the first text output. Therefore, the accuracy and usability of the first text output of the business model can be more accurately evaluated through two dimensions of confidence scores (first local confidence score and first global confidence score).

[0081] For details, please see Figure 2 , Figure 2 This is a schematic diagram illustrating a data processing method provided in an embodiment of this application. For example... Figure 2 As shown, the computer device may include a business model 202. A business object 201 can input first business data and a prompt into the business model 202. The computer device can use the business model 202 to parse the prompt and the first business data, and obtain a first text result 203 output by the business model 202 for the first business data, as well as a first logarithmic probability value sequence corresponding to the first text result 203. For example, the first business data could be "Besides restaurant X, what other good Hunan restaurants are nearby?" The business model 202 can parse the intent of the business object 201 and its intent parameters based on the prompt and the first business data, and output a first text result 203 containing structured content, as well as a first logarithmic probability value sequence 204 corresponding to the first text result.

[0082] It is important to note that the first text result 203 differs from ordinary text. It can be text with structured content but not yet with a structured form. Ordinary text typically refers to the expression of natural language, which lacks a specific structure or format to organize information. The first text result 203 contains data fields that can be directly identified and extracted. These data fields can be transformed into structured text through certain processing. For example, the first text result 203 can be represented as {"Intent": "Nearby Search", "Center Point": "Restaurant X", "Keyword": "Hunan Cuisine Restaurant"}, which simulates the content of structured text for output. Among them, fields such as "Intent" and "Nearby Search" are data fields in the first text result that can be directly identified and extracted. The first logarithmic probability value sequence 204 includes the logarithmic probability value corresponding to each character in the first text result 203. For example, if the first text result 203 includes 5 characters, the first logarithmic probability value sequence 204 can be represented as: logprobs = {0.82, 0.67, 0.92, 0.66, 0.77}.

[0083] Furthermore, the computer device can perform key-value pair alignment processing on the first text result 203 to obtain a first structured result 205. It can then obtain the first logarithmic probability value slices corresponding to the N attribute values ​​in the first structured result 205, resulting in N first logarithmic probability value slices 206. Local confidence assessments are then performed on each of the N first logarithmic probability value slices 206 to obtain the first local confidence values ​​corresponding to the N attribute values ​​in the first structured result 205, i.e., N first local confidence values ​​207. Finally, a global confidence assessment is performed on the N first local confidence values ​​207 to obtain the first global confidence value 208 for the first text result 203. Specifically, the computer device can perform key-value pair alignment processing on the first text result 203 to obtain a first structured result 205 for the first business data; wherein the first structured result 205 includes N business attributes and the attribute values ​​corresponding to the N business attributes; N is a positive integer. Taking the first text result 203 {"Intent": "Nearby Search", "Center Point": "Restaurant X", "Keyword": "Hunan Cuisine Restaurant"} in the above example as an example, the first structured result 205 can include three business attributes, namely "Intent", "Center Point" and "Keyword". The three attribute values ​​are "Nearby Search" corresponding to "Intent", "Restaurant X" corresponding to "Center Point" and "Hunan Cuisine Restaurant" corresponding to "Keyword".

[0084] Furthermore, the computer device can obtain N logarithmic probability value slices corresponding to each attribute value from the first logarithmic probability value sequence 204, integrate the data of the first logarithmic probability value slices corresponding to each attribute value, and obtain the first local confidence values ​​corresponding to each of the N attribute values. The computer device can perform data statistics on the first local confidence values ​​(N first logarithmic probability value slices 206) corresponding to each of the N attribute values, that is, perform global confidence evaluation, and obtain the first global confidence value 208 for the first text result 203. When N is 3, continuing with the above example, the final N first local confidence values ​​207 and the first global confidence value 208 can be represented as {"Nearby Search": 0.91, "Restaurant X": 0.37, "Hunan Cuisine Restaurant": 0.92, "First Global Confidence Value": 0.71}. Here, N first local confidence values ​​207 and first global confidence values ​​208 are used to represent the credibility of the first text result 203 when data processing is performed based on the first text result 203. That is, the larger the first local confidence value and the first global confidence value, the more credible the first text result 203 is considered. A higher credibility of the first text result 203 means a more credible processing result is obtained when data processing is performed based on the first text result 203. For example, using the first text result 203 to generate result XX, etc. Understandably, the larger the first local confidence value and the first global confidence value, the more reliable the first text result 203 is considered. This can mean that the first local confidence value and the first global confidence value 208 among the N first local confidence values ​​207 are greater than or equal to a certain threshold. For example, if a first local confidence value is greater than the local confidence threshold, then the attribute value corresponding to that first local confidence value can be considered reliable. When the N first local confidence values ​​207 pass the verification, if the first global confidence value 208 is greater than or equal to the global confidence threshold, then the first text result 203 can be considered reliable.

[0085] Through the above process, the first text output of the model is aligned to obtain the first structured result. Based on the first structured result, local and global confidence scores are determined for the business model's output, achieving a structured parsing of the model's output. This means that the confidence score determination not only considers the model's own output but also its structured information, thereby improving the comprehensiveness of the information referenced in the confidence score determination and increasing its accuracy. Furthermore, generating a global confidence score based on the local confidence score ensures that the global confidence score fully considers the structured data composition of the first text output of the business model, achieving attribute-based parsing of the first text output. Therefore, the accuracy and usability of the first text output of the business model can be more accurately evaluated through two dimensions of confidence scores (first local confidence score and first global confidence score).

[0086] It is understood that the business equipment mentioned in the embodiments of this application can also be a computer device, and the computer device in the embodiments of this application includes, but is not limited to, terminal devices or servers. In other words, the computer device can be a server or a terminal device, or a system composed of a server and a terminal device. The terminal device mentioned above can be an electronic device, including but not limited to mobile phones, tablets, desktop computers, laptops, handheld computers, in-vehicle devices, augmented reality / virtual reality (AR / VR) devices, head-mounted displays, smart TVs, wearable devices, and other mobile internet devices (MIDs) with network access capabilities. Figure 1 As shown, the terminal device can be a mobile phone (as shown in service device 102a), a desktop computer (as shown in service device 102b), a tablet computer (as shown in service device 102c), or a laptop computer (as shown in service device 102n), etc. Figure 1 Only a portion of the equipment is listed. The servers mentioned above can be standalone physical servers, server clusters or distributed systems composed of multiple physical servers, or cloud servers providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, vehicle-to-everything (V2X) communication, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0087] Optionally, the data involved in the embodiments of this application may be stored in a computer device, or may be stored based on cloud storage technology or a blockchain network, without limitation.

[0088] Further, please see Figure 3 , Figure 3 This is a flowchart illustrating a data processing method provided in an embodiment of this application. Figure 1 This data processing method can be executed by a computer device, which can be, for example,... Figure 1 Any one of the service devices in the service device cluster shown can also be a computer device 101. The following description will use the example of this data processing method being executed by a computer device. This data processing method will at least include the following steps S301-S304:

[0089] Step S301: Obtain the first text result output by the business model for the first business data, and the first log probability value sequence corresponding to the first text result; the first log probability value sequence includes the log probability value corresponding to each character in the first text result.

[0090] In this embodiment, a business object can input first business data into a business model via a computer device, and can also input prompts for the first business data. The business model can be a text-to-text model, such as a Large Language Model (LLM). A LLM is an artificial intelligence model specifically designed for natural language processing; it learns complex patterns and structures of language through pre-training on large amounts of text data. Prompts are generally used to guide the business model in generating results and controlling output content when the business object interacts with it. In other words, they provide the business model with processing directions or result generation directions for generating text (i.e., the first text result) based on the first business data, helping the business model understand the needs of the business object. After the business model receives the first business data and the prompts, it can process the first business data according to the prompts to obtain the first text result.

[0091] Specifically, when the prompt word is a formatting instruction for the first business data, the computer device can input the first business data and the formatting instruction for the first business data into the business model. The word segmenter in the business model then performs text splitting on the first business data to obtain the word sequence corresponding to the first business data. The formatting instruction is used to instruct the business model to output data with a structured format. Simultaneously, the word segmenter in the business model can also perform text splitting on the formatting instruction to obtain the formatting instruction sequence corresponding to the formatting instruction.

[0092] Furthermore, the computer device can encode each word in the word sequence using an encoder in the business model to obtain the text encoding features of the first business data. The text encoding features include the word vector for each word in the word sequence, with each word vector representing the semantic information of a word. For example, if the word sequence includes 5 words, the text encoding feature D can be represented as "{d1, d2, d3, d4, d5}", where d1 represents the word vector of the first word. Optionally, the computer device can encode each word in the format instruction sequence to obtain the format instruction features corresponding to the format hint instructions. The computer device can use the format hint instructions to guide the business model to perform text prediction on the text encoding features, generating multiple characters and the log probability value corresponding to each character. For example, the computer device can concatenate the text encoding features and the format instruction features using the business model to obtain concatenated encoding features containing enhanced contextual hints. Enhanced contextual hints refer to incorporating additional background information (format instruction features) into the text encoding features, which can clearly define the text format output by the business model. Further computer equipment can perform sequence-to-sequence text prediction tasks based on the fused concatenated encoding features. It can use formatting prompts to guide the business model's generation direction, outputting multiple characters that semantically match the first business data and the formatting prompts, along with the logarithmic probability value corresponding to each character. For example, assuming the business model includes multiple decoding layers and normalization layers, the computer equipment can process the concatenated encoding features layer by layer through the multiple decoding layers (Transformers) of the business model to obtain multiple generated character feature vectors. Each decoding layer includes self-attention processing and feedforward network processing (performing a non-linear transformation on the output of the self-attention processing to enhance the expressive power of the business model). Each character feature vector is processed through a normalization layer (Softmax layer), outputting a vocabulary probability distribution corresponding to each character feature vector. The word (token) corresponding to the highest probability in each vocabulary probability distribution is determined as the character in the corresponding character feature vector, resulting in multiple characters. The highest probability in each vocabulary probability distribution is determined as the output probability of the character in the corresponding character feature vector. Logarithmic processing is then performed on each output probability to obtain the logarithmic probability value corresponding to each character. Computer equipment can combine multiple characters into a first text result, and combine the log probability value corresponding to each character into a first log probability value sequence corresponding to the first text result.

[0093] The computer device can acquire the first text result output by the business model in response to the first business data, and the first logarithmic probability value sequence corresponding to the first text result. The first logarithmic probability value sequence includes the logarithmic probability value corresponding to each character in the first text result. For example, if the first text result includes 10 characters, the first logarithmic probability value sequence will include 10 logarithmic probability values. The logarithmic probability value (logprobs) is the logarithmic representation of probability, that is, the logarithmic form of the probability of the business model outputting a character. In the business model, the first text result is obtained by predicting the first business data, and the output probability of each character in the first text result is obtained. The logarithmic probability value of each character is obtained by performing a logarithmic operation on the output probability of each character. Finally, the business model can output the first text result and the logarithmic probability values ​​corresponding to each character in the first text result (wherein, the logarithmic probability values ​​corresponding to each character form the first logarithmic probability value sequence). For example, if the output probability of character 'a' in the first text result is p, then its logarithmic probability value is log(p). Computers can measure the confidence level, or credibility, of a business model's output for each character by using the log probability value corresponding to each character.

[0094] It is understandable that the business model can be integrated into the computer device, and the business model can also be integrated into other business devices that can interact with the computer device over the network. That is, the business model exists in other business devices, and the computer device can obtain the first text result and the first log probability value sequence output by the business model for the first business data from other business devices. There are no restrictions here, and the following explanation will only take the example of the business model existing in the computer device.

[0095] Step S302: Perform key-value pair alignment processing on the first text result to obtain the first structured result for the first business data; the first structured result includes N business attributes and the attribute values ​​corresponding to the N business attributes; N is a positive integer.

[0096] In the embodiments of the present application, a computer device may perform key-value pair alignment processing on a first text result to obtain a first structured result for first service data. The first structured result includes N service attributes and attribute values respectively corresponding to the N service attributes; N is a positive integer. Specifically, the computer device may perform character-by-character parsing on the first text result to obtain the character type of each character in the first text result. It can be understood that the character types generally include a text type and a symbol type (including a first symbol type and a second symbol type). For example, a piece of text is expressed as "like fruits and vegetables", where the character type of the specific text characters such as "like" and "fruit" in the text is the text type, and the character type of the character "、" is the symbol type. The computer device may split the first text result into N character sequences based on characters of the first symbol type in the first text result. The N character sequences include a character sequence C k , where k is a positive integer less than or equal to N; the characters of the first symbol type are used to distinguish characters associated with different service attributes.

[0097] Further, the computer device may determine the first characters located before the character of the second symbol type in the character sequence C k as the service attribute corresponding to the character sequence C k , and determine the second characters located after the character of the second symbol type in the character sequence C k as the attribute value of the service attribute corresponding to the character sequence C k . When the service attributes and attribute values respectively corresponding to the N character sequences are obtained, the service attribute and the attribute value corresponding to each character sequence are combined to obtain the first structured result for the first service data. It can be understood that the format of the first structured result may be a data structure with a clear hierarchical relationship and organization form such as a table, a tree structure, or a JSON format. The first structured result has the same text content as that presented by the first text result, and there is only a format difference, or it can be understood that the first text result is a text presentation form of the first structured result.

[0098] Please refer to Figure 4 together Figure 4 which is a schematic diagram of a key-value pair alignment processing scenario provided by the embodiment of the present application Figure 1 , as shown in Figure 4As shown, the first text result 401 is represented as {"AAA1":"BBB1", "AAA2":"BBB2", "AAA3":"BBB3"}, where the character type of the comma "," corresponding to characters 401a and 401b is the first symbol type. The computer device can split the first text result 401 into three character sequences based on characters 401a and 401b: {"AAA1":"BBB1"}, {"AAA2":"BBB2"}, and {"AAA3":"BBB3"}. The character sequence {"AAA1":"BBB1"} includes character 402a, which is of the second symbol type (i.e.,...). Figure 4 The character sequence {"AAA2":"BBB2"} includes character 402b, which is of type second symbol; the character sequence {"AAA3":"BBB3"} includes character 402c, which is of type second symbol. Based on character 402a in the character sequence, the computer device can determine the first character "AAA1" preceding character 402a as the business attribute corresponding to the character sequence {"AAA1":"BBB1"}, and the second character "BBB1" following character 402a as the attribute value of the business attribute corresponding to the character sequence {"AAA1":"BBB1"}, and so on, to obtain the business attributes and attribute values ​​corresponding to the three character sequences respectively.

[0099] In this structure, the business attributes and attribute values ​​corresponding to a character sequence exist in key-value pairs (KV). That is, the business attribute corresponding to a character sequence is the data key (K) of the key-value pair corresponding to that character sequence, and the attribute value corresponding to a character sequence is the data value (V) of the key-value pair corresponding to that character sequence. At this point, the computer device can obtain three key-value pairs: key-value pair 1 (including business attribute 403a and attribute value 403b), key-value pair 2 (including business attribute 404a and attribute value 404b), and key-value pair 3 (including business attribute 405a and attribute value 405b). Furthermore, the business attributes and attribute values ​​corresponding to each character sequence can be combined to obtain a first structured result 406 for the first business data. The first structured result 406 has the same text content as the first text result 401, and the first structured result 406 can be data belonging to a structured structure (such as JSON format) corresponding to the first text result 401.

[0100] Step S303: Obtain the first log probability value slices corresponding to N attribute values ​​from the first log probability value sequence, integrate the data of the first log probability value slices corresponding to each attribute value, and obtain the first local confidence values ​​corresponding to the N attribute values.

[0101] In the embodiments of the present application, the computer device may obtain the characters respectively included in the N attribute values from the first log probability value sequence, and determine the log probability value corresponding to the character respectively included in each attribute value in the first log probability value sequence as the first log probability value segment corresponding to the attribute value. For example, the N attribute values include an attribute value b (denoted as "surrounding search"), that is, the attribute value b includes 4 characters, which are "surround", "edge", "search" and "seek" respectively. The first log probability value sequence includes the log probability value corresponding to each character in the first text result, that is, includes the log probability value corresponding to each character in the attribute value b. The computer device may determine the log probability values respectively corresponding to the 4 characters ("surround", "edge", "search" and "seek") as the first log probability value segment corresponding to the attribute value b.

[0102] It should be noted that the data integration mentioned in the embodiments of the present application refers to performing statistical processing on the data involved in the data integration process (for example, each log probability value in the first log probability value segment). The statistical processing may be averaging processing. Specifically, the computer device may respectively perform averaging processing on the log probability values in the first log probability value segment corresponding to each attribute value, to obtain the first local confidence value respectively corresponding to each attribute value.

[0103] In step S304, data statistics is performed on the first local confidence values respectively corresponding to the N attribute values, to obtain a first global confidence value for the first text result; the first local confidence value and the first global confidence value are used to indicate the credibility degree of the first text result when data processing is performed based on the first text result.

[0104] In this embodiment, the computer device can perform data statistics on the first local confidence scores corresponding to N attribute values ​​to obtain a first global confidence score for the first text result. Specifically, an optional implementation process may be: the computer device performs average processing on the N first global confidence scores to obtain a first global confidence score for the first text result. Optionally, the computer device can also obtain the attribute weight of the business attribute corresponding to each attribute value, and perform a weighted summation of the N first global confidence scores based on the N attribute weights to obtain a first global confidence score for the first text result. The attribute weight represents the importance of the business attribute or the attribute value corresponding to the business attribute; the larger the attribute weight, the more important the attribute value of the business attribute corresponding to that attribute weight is considered. The first local confidence score and the first global confidence score represent the credibility of the first text result when data processing is performed based on the first text result. In other words, when processing data based on the first text result, the reliability of the first text result can be determined based on the first local confidence score and the first global confidence score. The reliability can also be understood as the accuracy of the first text result. The process of processing data based on the first text result refers to a data processing procedure that requires the first text result to generate a corresponding execution result. For example, if a business model receives third business data and needs to generate a third text result specific to the third business data using the first text result, then the reliability of the first text result during data processing can be understood as the reliability of the first text result when generating the third text result specific to the third business data.

[0105] For example, if the data processing is to obtain a data answer for the first business data, then when the credibility of the first text result is determined based on the first local confidence value and the first global confidence value, the computer device can obtain the answer format, convert the first text result into a data answer that conforms to the answer format, and output the data answer.

[0106] Optionally, when executing the specific implementation process of steps S302-S304, a confidence evaluation model can be trained separately, and the specific implementation process of steps S302-S304 can be completed through this confidence evaluation model.

[0107] Through the above process, the first text output of the model is aligned to obtain the first structured result. Based on the first structured result, local and global confidence scores are determined for the business model's output, achieving a structured parsing of the model's output. This means that the confidence score determination not only considers the model's own output but also its structured information, thereby improving the comprehensiveness of the information referenced in the confidence score determination and increasing its accuracy. Furthermore, generating a global confidence score based on the local confidence score ensures that the global confidence score fully considers the structured data composition of the first text output of the business model, achieving attribute-based parsing of the first text output. Therefore, the accuracy and usability of the first text output of the business model can be more accurately evaluated through two dimensions of confidence scores (first local confidence score and first global confidence score).

[0108] Further, please see Figure 5 , Figure 5 This is a flowchart illustrating a data processing method provided in an embodiment of this application. Figure 2 This data processing method can be executed by a computer device, which can be, for example,... Figure 1 Any one of the service devices in the service device cluster shown can also be computer device 101. The following description will use the example of this data processing method being executed by a computer device. This data processing method will at least include the following steps S501-S508:

[0109] Step S501: Obtain the first text result output by the business model for the first business data, and the first log probability value sequence corresponding to the first text result; the first log probability value sequence includes the log probability value corresponding to each character in the first text result.

[0110] In this embodiment of the application, the specific implementation process of step S501 can be found as follows: Figure 3 The specific process described in step S301 shown will not be repeated here.

[0111] Step S502: Perform key-value pair alignment processing on the first text result to obtain the first structured result for the first business data; the first structured result includes N business attributes and the attribute values ​​corresponding to the N business attributes; N is a positive integer.

[0112] In this embodiment of the application, the specific implementation process of step S502 can be found as follows: Figure 3 The detailed description of step S302 shown is not repeated here. Please refer to [link to previous text]. Figure 6 , Figure 6is a schematic diagram of a key-value pair alignment processing scenario provided in the embodiments of the present application Figure 2 , which is used to supplement and explain the key-value pair alignment processing procedure. As shown in Figure 6 , a service object may input first service data 601 into a service model 602, and the first service data 601 may be expressed as "Besides Hotel Z, what are other affordable five-star hotels nearby". A computer device parses the first service data through the service model 602, and outputs a first text result 603, and a first log probability value sequence 604 corresponding to the first text result 603. The first text result 603 may be expressed as {"Intent": "Surrounding Search", "Center Point": "Hotel Z", "Keyword": "Five-star Hotel"}, and the first log probability value sequence 604 includes a log probability value corresponding to each character in the first text result. For example, the first log probability value (logprobs) 0.82 in the first log probability value sequence 604 is the log probability value corresponding to the character "意"; the second log probability value 0.88 is the log probability value corresponding to the character "图"; the third log probability value 0.90 is the log probability value corresponding to the first character ":"; the fourth log probability value 0.74 is the log probability value corresponding to the character "周"; and so on, the 8th log probability value 0.92 is the log probability value corresponding to the first character ",", and the last log probability value 0.93 is the log probability value corresponding to the character "店" (the "店" in "five-star hotel").

[0113] Further, the computer device performs key-value pair alignment processing on the first text result 603 to obtain a first structured result 605. For the specific key-value pair alignment procedure, reference may be made to Figure 3 step S302 in and Figure 4 the specific descriptions in, which will not be repeated herein. It can be understood that the content in the first structured result 605 has a corresponding relationship with the content in the first text result 603, if Figure 6As shown by the dashed arrows, the attribute value 1x "surrounding search" in the first structured result 605 corresponds to the four characters "surrounding search" in the first text result 603; the attribute value 1y "hotel Z" in the first structured result 605 corresponds to the three characters "hotel Z" in the first text result 603; and the attribute value 1z "five-star hotel" in the first structured result 605 corresponds to the four characters "five-star hotel" in the first text result 603. Based on these correspondences (mapping relationships or association relationships), in subsequent steps, when determining the first logarithmic probability value slice corresponding to each attribute value, the computer device can directly obtain the characters in the first text result 603 that correspond to each attribute value, and obtain the logarithmic probability value at the position of that character from the first logarithmic probability value sequence, thereby determining the first logarithmic probability value slice corresponding to each attribute value.

[0114] Step S503: Obtain the first log probability value slices corresponding to N attribute values ​​from the first log probability value sequence, integrate the data of the first log probability value slices corresponding to each attribute value, and obtain the first local confidence values ​​corresponding to the N attribute values.

[0115] In this embodiment of the application, the N attribute values ​​include attribute value A. i Where i is a positive integer less than or equal to N. The computer device can obtain the attribute value A. i The included M characters are used to form attribute value A by combining the logarithmic probability values ​​corresponding to the M characters in the first logarithmic probability value sequence. i The corresponding first logarithmic probability value slice; M is a positive integer. Specifically, the first structured result is generated from the first text result, and each business attribute and attribute value in the first structured result has a mapping relationship with the characters in the first text result that generated that business attribute and attribute value. The computer device can obtain attribute value A. i The included M characters can be determined based on the mapping relationship between the first structured result and the first text result. This allows us to identify the corresponding M mapping characters in the first text result. Based on the sorting position of these M mapping characters in the first text result, we obtain the M log probability values ​​located at that sorting position from the first log probability value sequence. These M log probability values ​​are then combined to form the attribute value A. i The corresponding first logarithmic probability value slice.

[0116] For example, attribute value A iThe corresponding M mapped characters are located at positions 8 to 12 in the first text result, meaning the M mapped characters are the 8th to 12th characters in the first text result. Furthermore, the computer device can use the 8th to 12th logarithmic probability values ​​from the first logarithmic probability value sequence to form attribute value A. i The corresponding first log-probability value slice, i.e., attribute value A i The corresponding first logarithmic probability value slice includes the four logarithmic probability values ​​at positions 8 to 12 of the first logarithmic probability value sequence. Optionally, the computer device can obtain attribute value A. i The included M characters can be attribute value A in the first text result. i Given the M characters, the computer device can determine the logarithmic probability value of each of the M characters from the first logarithmic probability value sequence based on their sorting positions in the first text result. Furthermore, the logarithmic probability values ​​of the M characters can be combined to form attribute value A. i The corresponding first logarithmic probability value slice.

[0117] Furthermore, computer equipment can process attribute value A i The corresponding first logarithmic probability value is used to integrate the data to obtain attribute value A. i The corresponding first local confidence score value. Data integration refers to averaging multiple logarithmic probability values ​​within the first logarithmic probability value slice, also known as mean normalization. Specifically, the computer device can process attribute value A... i The attribute value A is obtained by averaging the M log probability values ​​in the first log probability value slice. i The corresponding first local confidence score. The computer device can calculate attribute value A. i The sum of the M log probability values ​​in the corresponding first log probability value slice, and then, based on the quantity M and the sum of the M log probability values, the average of the M log probability values ​​is determined, and the average of the M log probability values ​​is determined as the attribute value A. i The corresponding first local confidence score value. For example, attribute value A. i The corresponding first logarithmic probability value segment is [0.31, 0.53, 0.91, 0.98]. The computer device can calculate the sum of the four logarithmic probability values. For example, 0.31 + 0.53 + 0.91 + 0.98 = 2.73. Dividing 2.73 by 4 equals 0.68, which means the average of these four logarithmic probability values ​​is 0.68. The computer device can then determine 0.68 as the attribute value A. i The corresponding first local confidence score value. Similarly, the computer device can obtain the first local confidence score values ​​corresponding to N attribute values.

[0118] Step S504: Perform data statistics on the first local confidence values ​​corresponding to the N attribute values ​​to obtain the first global confidence value for the first text result; the first local confidence value and the first global confidence value are used to represent the credibility of the first text result when data processing is performed based on the first text result.

[0119] In this embodiment, the computer device can determine the first global confidence value for the first text result by summing the N first local confidence values. Alternatively, it can determine the first global confidence value for the first text result by averaging the N first local confidence values.

[0120] Optionally, the computer device can obtain the attribute weights corresponding to each of the N business attributes, and use these weights to perform a weighted sum of the first local confidence scores corresponding to the N attribute values ​​to obtain the first global confidence score for the first text result. Specifically, the attribute weights corresponding to the N business attributes can be configured by the business object associated with the first business data. It can be understood that the business object typically configures the attribute weights according to the importance of the business attributes; the higher the importance, the greater the attribute weight.

[0121] For example, when there are two business attributes, "process" and "result," if the business object determines that "result" is more important than "process," then "result" can be assigned a higher attribute weight, for example, an attribute weight of 0.8 for "result" and 0.2 for "process." Similarly, if the business object determines that "process" is more important than "result," then "process" can be assigned a higher attribute weight, for example, an attribute weight of 0.8 for "process" and 0.2 for "result." When the attribute weights are configured by the business object, the computer device can directly obtain the attribute weights corresponding to the N business attributes configured by the business object.

[0122] Optionally, the computer device can input N business attributes and the first business data into an importance detection model. The importance detection model then detects the correlation between each of the N business attributes and the first business data, obtaining the attribute weights corresponding to each of the N business attributes. The importance detection model can be any model capable of detecting the correlation between each of the N business attributes and the first business data, or a model that detects the importance of each business attribute. The importance detection model can be trained using business data samples, business attribute samples, and attribute weight samples.

[0123] Alternatively, in the importance detection model, the computer device can perform importance detection on N business attributes based on the first business data to obtain the importance of each business attribute. Furthermore, the computer device can assign attribute weights to each business attribute based on its importance. Optionally, the computer device can obtain an importance-weight mapping table, where one possible importance-weight mapping table is shown in Table 1:

[0124] Table 1

[0125] Attribute Importance Attribute weight range high 0.81～0.99 higher 0.56～0.8 middle 0.31～0.55 lower 0.21～0.3 Low 0.01～0.2

[0126] The importance-weight mapping table can include the attribute importance and the corresponding attribute weight range. When the importance of a business attribute output by the importance detection model is high, the computer device can assign the corresponding attribute importance within the attribute weight range of 0.81 to 0.99, for example, the attribute weight corresponding to this business attribute can be 0.9; when the importance of a business attribute output by the importance detection model is relatively high, the computer device can assign the corresponding attribute weight within the attribute weight range of 0.56 to 0.8, for example, 0.66; when the importance detection model... When the importance of the output business attribute is medium, the computer device can assign a corresponding attribute weight within the attribute weight range of 0.31 to 0.55, for example, 0.5. When the importance of the output business attribute is low, the computer device can assign a corresponding attribute weight within the attribute weight range of 0.21 to 0.3, for example, 0.25. When the importance of the output business attribute is low, the computer device can assign a corresponding attribute weight within the attribute weight range of 0.01 to 0.2, for example, 0.1.

[0127] It is understandable that the importance-weight mapping table can include the range of attribute weights, or it can directly include the attribute weights corresponding to the importance of an attribute. For example, when the importance of a business attribute output by the importance detection model is high, the computer device can directly query the attribute weight corresponding to that importance in the importance-weight mapping table (for example, a high attribute importance corresponds to an attribute weight of 0.9), and the computer device can directly determine that the attribute weight corresponding to that business attribute is 0.9.

[0128] Optionally, the computer device can obtain the first attribute category corresponding to each of the N business attributes, as well as preset category weight information. The preset category weight information includes the attribute category and its category weight; the attribute category includes the first attribute category. The attribute category refers to categories such as process, result, auxiliary information, intent, central point, and keywords. The computer device can obtain the category weights of the first attribute categories corresponding to each of the N business attributes from the preset category weight information, and can normalize the category weights corresponding to the N business attributes to obtain the attribute weights corresponding to each of the N business attributes. The preset category weight information can be pre-set by the business object; for example, an optional set of preset category weight information is shown in Table 2.

[0129] Table 2

[0130] Attribute Category Category weight Category 1 0.3 Category 2 0.3 … … Category 3 0.32 Category 4 0.6 Category 5 0.9

[0131] As shown in Table 2, when the N business attributes (e.g., 3) correspond to the first attribute categories, including category 1, category 2, and category 4, the category weights of the 3 business attributes are 0.3, 0.3, and 0.6, respectively. In this case, the computer device can normalize the 3 category weights to obtain the attribute weights corresponding to the N business attributes. An optional normalization process can be found in formula ①:

[0132]

[0133] As shown in formula ①, “x i "Used to represent category weights, y(x) i This is used to represent the attribute weights obtained after normalizing the category weights. Used to represent the sum of the weights of N categories, after normalization, the attribute weight corresponding to a category weight of 0.3 is 0.25, and the attribute weight corresponding to a category weight of 0.6 is 0.5.

[0134] Furthermore, the computer device can use the attribute weights corresponding to N business attributes to perform a weighted summation of the first local confidence scores corresponding to the N attribute values, thereby obtaining the first global confidence score for the first text result. Specifically, the N attribute values ​​include attribute value A. i Where i is a positive integer less than or equal to N. The computer device can assign attribute value A. i The corresponding business attribute's attribute weight and attribute value A i The product of the corresponding first local confidence scores is determined for attribute value A. iThe weighted confidence scores are calculated. When the weighted confidence scores for each of the N attribute values ​​are obtained, the sum of these N weighted confidence scores is determined as the first global confidence score for the first text result. The specific implementation method for determining the first global confidence score for the first text result can be found in formula ②:

[0135]

[0136] As shown in formula ②, C represents the first global confidence score. i w is used to represent the first global confidence score corresponding to the i-th attribute value out of N attribute values. i c is used to represent the attribute weight of the business attribute corresponding to the i-th attribute value. i *w i Used to represent c i and w i The product of N and W. When N is 3, formula ② can be understood as C = c1*w1 + c2*w2 + c3*w3.

[0137] Optional, N business attributes include business attribute B. i N attribute values, including attribute value A i Attribute value A i For business attribute B i The corresponding attribute value, i, is a positive integer less than or equal to N. The computer device can obtain the business attribute B from the first logarithmic probability value sequence. i The corresponding second logarithmic probability value fragment is then integrated to obtain business attribute B. i The corresponding attribute confidence score. Among them, determining business attribute B. i The specific implementation process of the corresponding second logarithmic probability value sharding can be: obtaining business attribute B i The included R characters are used to form business attribute B by taking the logarithmic probability values ​​corresponding to the R characters in the first logarithmic probability value sequence. i The corresponding second logarithmic probability value slice. Where R is a positive integer. For business attribute B... i The R log probability values ​​in the corresponding second log probability value slice are averaged to obtain business attribute B. i The corresponding attribute confidence value.

[0138] Furthermore, the computer equipment can obtain the first weighted weight and the second weighted weight, based on business attribute B. i The corresponding attribute weights and the first weighted weight determine the business attribute B. i The attribute weighting. Specifically, computer equipment can assign weights to business attribute B. iThe product of the corresponding attribute weight and the first weighted weight is used to determine business attribute B. i The attribute weighting. This can be based on business attribute B. i The corresponding attribute weight and second weight are used to determine the attribute value A. i The values ​​are weighted. Specifically, computer equipment can assign weights to business attribute B. i The product of the corresponding attribute weight and the second weight is determined for attribute value A. i The values ​​are weighted. The first weight represents the weight percentage of a business attribute within the same group of business attributes and attribute values; the second weight represents the weight percentage of an attribute value within the same group of business attributes and attribute values. For example, given a group of business attributes and attribute values ​​denoted as {Business Attribute 1a: Attribute Value 1b}, if the weight percentage of business attribute 1a in {Business Attribute 1a: Attribute Value 1b} is 0.1, then its first weight is 0.1, and the weight percentage of attribute value 1b is 0.9, meaning its second weight is 0.9.

[0139] Furthermore, computer equipment can be based on business attribute B. i The attribute weights and attribute values ​​A i The value of the weighted weight is applied to business attribute B. i The corresponding attribute confidence value and attribute value A i The corresponding first local confidence scores are weighted and averaged to obtain the result for attribute value A. i The weighted confidence score. Specifically, computer equipment can determine business attribute B. i The attribute weighting and business attribute B i The product of the corresponding attribute confidence scores is used to determine the attribute value A. i The weighted average of the values ​​and the attribute value A i The product of the corresponding first local confidence scores will determine the business attribute B. i The attribute weighting and business attribute B i The product of the corresponding attribute confidence scores and the attribute value A i The weighted average of the values ​​and the attribute value A i The sum of the products of the corresponding first local confidence scores is used to determine the attribute value A. i (or business attribute B) i The weighted confidence values ​​corresponding to the N attribute values ​​are used to determine the first global confidence value for the first text result.

[0140] Optionally, the computer equipment can also be based on business attribute B. iThe attribute weights and attribute values ​​A i The value of the weighted weight is applied to business attribute B. i The corresponding attribute confidence value and attribute value A i The corresponding first local confidence scores are weighted and summed to obtain the result for attribute value A. i The weighted confidence level is not limited here and can be determined based on the specific circumstances.

[0141] Please see also Figure 7 , Figure 7 This is a schematic diagram of a weighted processing scenario provided in an embodiment of this application, used to illustrate one of the processes described above, which involves using the attribute weights corresponding to N business attributes to perform a weighted summation of the first local confidence scores corresponding to the N attribute values, thereby obtaining a first global confidence score for the first text result. For example... Figure 7 As shown, N is 3, including three key-value pairs that make up the first structured text, namely key-value pair 701, key-value pair 702, and key-value pair 703. Key-value pair 701 includes business attribute 1a and attribute value 1b, key-value pair 702 includes business attribute 2a and attribute value 2b, and key-value pair 703 includes business attribute 3a and attribute value 3b. The computer device can obtain the first weighted weight and the second weighted weight. The following explanation uses an example where the first weighted weight is 0.1, the second weighted weight is 0.2, the attribute weight 1 corresponding to business attribute 1a is 0.2, the attribute weight 2 corresponding to business attribute 2a is 0.3, and the attribute weight 3 corresponding to business attribute 3a is 0.5.

[0142] The computer equipment can determine the attribute weight 1c (0.02) corresponding to business attribute 1a by multiplying attribute weight 1 and the first weighted weight; it can determine the value weight 1d (0.18) corresponding to attribute value 1b by multiplying attribute weight 1 and the second weighted weight; it can determine the attribute weight 2c (0.03) corresponding to business attribute 2a by multiplying attribute weight 2 and the second weighted weight; it can determine the value weight 2d (0.27) corresponding to attribute value 2b by multiplying attribute weight 3 and the first weighted weight; it can determine the attribute weight 3c (0.05) corresponding to business attribute 3a by multiplying attribute weight 3 and the second weighted weight; and it can determine the value weight 3d (0.45) corresponding to attribute value 3b by multiplying attribute weight 3 and the first weighted weight.

[0143] Furthermore, the computer equipment can perform a weighted average of the attribute confidence value 1e corresponding to business attribute 1a and the first local confidence value 1f corresponding to attribute value 1b, based on the attribute weighting weight 1c of business attribute 1a and the value weighting weight 1d of attribute value 1b, to obtain a weighted confidence value 1g for attribute value 1b; similarly, it can perform a weighted average of the attribute confidence value 2e corresponding to business attribute 2a and the first local confidence value 2f corresponding to attribute value 2b, based on the attribute weighting weight 2c of business attribute 2a and the value weighting weight 2d of attribute value 2b, to obtain a weighted confidence value 2g for attribute value 2b; and it can perform a weighted average of the attribute confidence value 3e corresponding to business attribute 3a and the first local confidence value 3f corresponding to attribute value 3b, based on the attribute weighting weight 3c of business attribute 3a and the value weighting weight 3d of attribute value 3b, to obtain a weighted confidence value 3g for attribute value 3b. One possible implementation of the weighted average method can be found in formula ③:

[0144]

[0145] As shown in formula ③, D represents the weighted confidence score, t represents the attribute weight, s represents the value weight, d represents the attribute confidence score, and b represents the first local confidence score. For example, when calculating the weighted confidence score 1g, that is, when D represents the weighted confidence score 1g, t represents the attribute weight 1c (for example, 0.02), s represents the value weight 1d (for example, 0.18), d represents the attribute confidence score 1e (for example, 0.6), and b represents the first local confidence score 1f (for example, 8). The calculated weighted confidence score 1g is 0.78.

[0146] Optionally, the computer device can also perform a weighted summation process on the attribute confidence value 1e corresponding to business attribute 1a and the first local confidence value 1f corresponding to attribute value 1b, based on the attribute weighting weight 1c of business attribute 1a and the value weighting weight 1d of attribute value 1b, to obtain the weighted confidence value 1g for attribute value 1b. A possible implementation of the weighted summation process can be found in formula ④:

[0147] E=d*t+b*s ④

[0148] As shown in Formula ④, E represents the weighted confidence score, t represents the attribute weight, s represents the value weight, d represents the attribute confidence score, and b represents the first local confidence score. For example, when calculating the weighted confidence score 1g, that is, when E represents the weighted confidence score 1g, t represents the attribute weight 1c (for example, 0.02), s represents the value weight 1d (for example, 0.18), d represents the attribute confidence score 1e (for example, 0.6), and b represents the first local confidence score 1f (for example, 8). The calculated weighted confidence score 1g is 0.156.

[0149] Furthermore, when the weighted confidence values ​​corresponding to the three attribute values ​​are obtained, the sum of the three weighted confidence values ​​can be determined as the first global confidence value for the first text result. That is, the sum of the weighted confidence values ​​1g, 2g, and 3g is determined as the first global confidence value for the first text result.

[0150] Step S505: Detect whether the first local confidence value and the first global confidence value meet the threshold requirements.

[0151] In this embodiment, the computer device can detect whether the first local confidence value and the first global confidence value meet the threshold requirements. If the first local confidence value and the first global confidence value meet the threshold requirements, step S507 is executed; if the first local confidence value and the first global confidence value do not meet the threshold requirements, step S506 is executed. Specifically, if the first local confidence value corresponding to the key attribute is less than the local confidence threshold, or the first global confidence value is less than the global confidence threshold, then step S506 is executed; if the first local confidence value corresponding to the key attribute is greater than or equal to the local confidence threshold, and the first global confidence value is greater than or equal to the global confidence threshold, then step S507 is executed. Here, the key attribute refers to a business attribute that has key significance in the first text result or the first structured result, that is, a business attribute whose attribute weight is greater than or equal to the key weight threshold among N business attributes. The local confidence threshold is used to measure whether the key attribute corresponding to the first local confidence value is reliable. More precisely, it is used to measure whether the attribute value corresponding to the key attribute is reliable. If the first local confidence value is higher than or equal to the local confidence threshold, the attribute value corresponding to the first local confidence value is considered reliable. If the first local confidence value is lower than the local confidence threshold, the attribute value corresponding to the first local confidence value is considered unreliable. Similarly, the global confidence threshold is used to measure whether the first text result corresponding to the first global confidence value is reliable. Optionally, the computer device can directly detect whether the first global confidence value meets the threshold requirement based on the first global confidence value. That is, if the first global confidence value meets the threshold requirement (i.e., the first global confidence value is less than the global confidence threshold), then proceed to step S506; if the first global confidence value does not meet the threshold requirement (i.e., the first global confidence value is greater than or equal to the global confidence threshold), then proceed to step S507.

[0152] It is important to note that when computer devices use different weighting methods to obtain the weighted confidence scores for each attribute value, the corresponding global confidence thresholds will differ. For example, when a computer device uses business attribute B... i The attribute weights and attribute values ​​A i The value of the weighted weight is applied to business attribute B. i The corresponding attribute confidence value and attribute value A i The corresponding first local confidence scores are weighted and summed to obtain the result for attribute value A. i When calculating the weighted confidence value, the corresponding global confidence threshold can be 0.8. Computer equipment is based on business attribute B. i The attribute weights and attribute values ​​A i The value of the weighted weight is applied to business attribute B. iThe corresponding attribute confidence value and attribute value A i The corresponding first local confidence scores are weighted and averaged to obtain the result for attribute value A. i When calculating the weighted confidence value, the corresponding global confidence threshold can be reset to a global confidence threshold within a reasonable range, or it can be the product of the previous global confidence threshold and the number of attribute values ​​(0.8*N). For example, if N is 3, the corresponding global confidence threshold can be 2.4.

[0153] Step S506: Determine the confidence anomaly result for the first text result, and generate a data prompt text based on the confidence anomaly result; the data prompt text is used to prompt the business object associated with the first business data to input updated business data in the business model.

[0154] In this embodiment, the computer device can determine an anomaly in the confidence level of the first text result based on the first local confidence value, the local confidence threshold, the first global confidence value, and the global confidence threshold corresponding to the key attribute. The anomaly in the confidence level indicates that the first text result is unreliable or has low credibility. For example, an anomaly in the confidence level can be expressed as "the first local confidence value corresponding to the key attribute is less than the local confidence threshold, and the first global confidence value is less than the global confidence threshold, therefore the first text result can be determined to be unreliable." Furthermore, the computer device can obtain a prompt text template from a template library, update the prompt text template based on the anomaly in the confidence level, and obtain a data prompt text. This data prompt text can be sent to the business object associated with the first business data. The data prompt text is used to prompt the business object associated with the first business data to input updated business data in the business model.

[0155] Optionally, the computer device can extract the structured feature vector of the first business data, input the structured feature vector into the relational database to match the corresponding business object identifier, and initiate a query request to the metadata repository based on the business object identifier to obtain the attribute information set of the business object. The attribute information set includes at least the business object name and historical data update records. Further, the computer device can construct a dynamic text generation template, embedding the confidence anomaly result, local confidence threshold, global confidence threshold, and business object name into a predefined semantic framework of the dynamic text generation template to obtain data prompt text. The data prompt text can be natural language prompt text including the data anomaly type, a description of the scope of impact, and a forced update instruction. While the historical data update records are not directly embedded in the data prompt text, they serve as a background knowledge base for data updates in the business model and can be used to dynamically analyze the correlation between the current confidence anomaly result and historical update events in the historical data update records. For example, when a current confidence level anomaly is detected to be similar to an anomaly event in a historical data update record caused by a data format error, the computer device can automatically adjust the impact range description logic in the dynamic text generation template, increase the intensity of the mandatory prompts for data format verification, and thus indirectly improve the accuracy of the data prompt text.

[0156] Step S507: Obtain the business processing result for the first business data based on the first structured result, and output the business processing result.

[0157] In this embodiment, the computer device can obtain the business processing result for the first business data from the first structured result and output the business processing result. Specifically, when the first text result is a request result for the first business data, the computer device can obtain the attribute value corresponding to the first business attribute from the N business attributes in the first structured result. Here, the first business attribute refers to the business attribute whose attribute category is result attribute among the N business attributes. The attribute value corresponding to the first business attribute can be determined as the business processing result for the first business data. For example, if the first structured result is “{"reasoning":"There are 15 trees originally.Then there were 21 trees after some more were planted.So there must have been 21-15=6.","answer":6}”, ​​the computer device can determine the attribute value “6” corresponding to the first business attribute “answer” as the business processing result for the first business data.

[0158] Optionally, when the first text result is the result of intent parsing for the first business data, the computer device can obtain H media data associated with the first structured result through a downstream processing program. The downstream processing program can reside in the computer device or other business devices; there is no limitation on this. Furthermore, the computer device can obtain the data correlation degree between each of the H media data and the first structured result. Here, H is a positive integer.

[0159] Specifically, the computer equipment can extract features from H media data points to obtain data extraction features corresponding to each of the H media data points; it can also extract features from the first structured result to obtain structured result features; and it can perform feature comparisons between the H data extraction features and the structured result features to obtain the feature correlation degree between each of the H data extraction features and the structured result features. The feature correlation degree between each of the H data extraction features and the structured result features is determined as the data correlation degree between each of the H media data points and the first structured result. The feature comparison method used to compare the features between the H data extraction features and the structured result features can be vector similarity calculation or cross-attention calculation, etc., and is not limited here.

[0160] Understandably, each attribute value corresponds to a first local confidence score and a first global confidence score for the overall result. This allows downstream processing programs to better manage request risks and prevent chain reactions caused by unreliable results generated by the business model that downstream processing programs are unaware of. Optionally, downstream processing programs can also separately check the first local confidence score and the first global confidence score corresponding to the first text result (in this case, the confidence threshold used for detection can be the same as or different from the local confidence threshold or the global confidence threshold). If the downstream processing program determines that the first text result is unreliable, it can reject the first text result or the first structured result, or take additional fallback strategies to improve the usability of the first text result or the first structured result, thereby improving the usability of model intelligence based on the business model. Among these fallback strategies, it is possible to obtain historical business data (which has been confirmed to be reliable) that is highly similar to or related to the first business data from the metadata database or the storage database corresponding to the business model, and determine the historical text result corresponding to the historical business data as the updated text result of the first business data, so that the downstream processing program can perform subsequent processing based on the updated text result.

[0161] Furthermore, the computer device can extract the associated media data corresponding to the first structured result from the H media data based on the data correlation degree between each of the H media data and the first structured result. Specifically, the computer device can obtain a data correlation degree threshold and determine the media data corresponding to the data correlation degree greater than or equal to the data correlation degree threshold among the H data correlation degrees as the associated media data corresponding to the first structured result. For example, if there are three media data, namely media data x (data correlation degree of 0.8), media data y (data correlation degree of 0.88), and media data z (data correlation degree of 0.6), when the data correlation degree threshold is 0.77, the associated media data corresponding to the first structured result includes media data x and media data y.

[0162] Furthermore, the computer equipment can integrate related media data based on the first business data to obtain a business processing result for the first business data. Integrating related media data can refer to actions such as deletion and splicing. Specifically, the computer equipment can detect each media data in the related media data, determine the similarity between each media data, delete each media data based on the similarity, and then splice the deleted media data to obtain a business processing result for the first business data. For example, if the related media data includes three media data (media data 1, media data 2, and media data 3), and media data 1 and media data 2 are similar (partially identical), the computer equipment can merge media data 1 and media data 2 (retaining the identical parts and merging the different parts) to obtain merged media data. This merged media data is then spliced ​​with media data 3 to obtain the business processing result for the first business data. If media data 1 and media data 2 are the same, the computer device will only retain one of media data 1 and media data 2. Taking the retention of media data 1 as an example, the computer device can concatenate media data 1 and media data 3 to obtain the business processing result for the first business data.

[0163] Step S508: When second business data associated with the first business data is obtained through the business model, a second text result for the second business data is generated, a second global confidence value for the second text result is determined, and the second global confidence value is adjusted based on the first local confidence value corresponding to the associated attribute value to obtain an updated global confidence value for the second text result.

[0164] In this embodiment, when second business data associated with the first business data is obtained through a business model, the computer device can parse the second business data to generate a second text result for the second business data and a second logarithmic probability value sequence corresponding to the second text result. The specific implementation process of the computer device generating the second text result for the second business data and the second logarithmic probability value sequence corresponding to the second text result can be found in [reference needed]. Figure 3 The specific description of obtaining the first text result output by the business model for the first business data and the first logarithmic probability value sequence corresponding to the first text result in step S301 shown will not be repeated here.

[0165] Furthermore, the computer device can generate a second global confidence score for the second text result based on the second text result and the second logarithmic probability value sequence. The specific implementation process for generating the second global confidence score can be found in [link to documentation]. Figure 3 and Figure 5 In the embodiments described in detail, the specific process of obtaining the first global confidence data will not be repeated here.

[0166] The computer device can extract features from N attribute values ​​in the first structured result, determine the attribute value features corresponding to each of the N attribute values, then extract features from the second text result to obtain the second text features corresponding to the second text result. The N attribute value features and the second text features are then compared to obtain the correlation values ​​between the N attribute values ​​and the second text result. Based on the N correlation values, the associated attribute values ​​related to the second text result are obtained from the first structured result. Based on the first local confidence value corresponding to the associated attribute values, the second global confidence value is adjusted to obtain an updated global confidence value for the second text result. Here, the N attribute values ​​include associated attribute values.

[0167] An optional adjustment method could be: if the number of associated attribute values ​​is 1, the computer device can determine the updated global confidence value for the second text result as the product between the first local confidence value and the second global confidence value corresponding to that associated attribute value. Optionally, the average value between the first local confidence value and the second global confidence value corresponding to that associated attribute value can also be determined as the updated global confidence value for the second text result. If there are multiple associated attribute values, the computer device can determine the average value, or weighted average, of the first local confidence values ​​corresponding to the multiple associated attribute values, and determine the updated global confidence value for the second text result based on the average value, or weighted average, of the first local confidence values ​​corresponding to the multiple associated attribute values ​​and the second global confidence value.

[0168] Through the above process, the first text output of the model is aligned to obtain the first structured result. Based on the first structured result, local and global confidence scores are determined for the business model's output, achieving structured parsing of the model's output. This ensures that the confidence score determination considers not only the model's own output but also its structured information, thus improving the comprehensiveness and accuracy of the information referenced in the confidence score determination. Furthermore, generating a global confidence score based on the local confidence score ensures that the global confidence score fully considers the structured data composition of the first text output of the business model, achieving attribute-based parsing of the first text output. This allows for a more accurate evaluation of the accuracy and usability of the first text output of the business model through two dimensions of confidence scores (first local confidence score and first global confidence score). This further reduces the risk of downstream processing programs processing based solely on the first text or first structured result.

[0169] Further, please see Figure 8 , Figure 8 This is a schematic diagram of a data processing apparatus provided in an embodiment of this application. The data processing apparatus 800 can be a computer program (including program code, etc.) running on a computer device; for example, the data processing apparatus 800 can be an application software. The data processing apparatus 800 can be used to execute corresponding steps in the method provided in the embodiments of this application. Figure 8 As shown, the data processing device 800 can be used for Figure 3 and Figure 5 Specifically, the data processing device 800 in the corresponding embodiment may include: a text result acquisition module 11, an alignment processing module 12, a local confidence determination module 13, a global confidence determination module 14, a business processing result acquisition module 15, and a confidence adjustment module 16.

[0170] The text result acquisition module 11 is used to acquire the first text result output by the business model for the first business data, and the first log probability value sequence corresponding to the first text result; the first log probability value sequence includes the log probability value corresponding to each character in the first text result;

[0171] Alignment processing module 12 is used to perform key-value pair alignment processing on the first text result to obtain a first structured result for the first business data; the first structured result includes N business attributes and the attribute values ​​corresponding to the N business attributes; N is a positive integer;

[0172] The local confidence determination module 13 is used to obtain the first log probability value slices corresponding to N attribute values ​​from the first log probability value sequence, integrate the data of the first log probability value slices corresponding to each attribute value, and obtain the first local confidence value corresponding to the N attribute values ​​respectively.

[0173] The global confidence determination module 14 is used to perform data statistics on the first local confidence values ​​corresponding to the N attribute values ​​respectively, and obtain the first global confidence value for the first text result; the first local confidence value and the first global confidence value are used to represent the credibility of the first text result when data processing is performed based on the first text result.

[0174] In one possible implementation, the alignment processing module 12 is used to perform key-value pair alignment processing on the first text result to obtain the first structured result for the first business data. Specifically, the alignment processing module 12 is used to perform the following operations:

[0175] The first text result is parsed character by character to obtain the character type of each character in the first text result;

[0176] Based on the characters of type 1 symbol in the first text result, the first text result is split into N character sequences; the N character sequences include character sequence C. k k is a positive integer less than or equal to N; the first symbol type character is used to distinguish characters associated with different business attributes;

[0177] Character sequence C k The first character preceding the character of type second symbol is determined as character sequence C. k The corresponding business attribute will be the character sequence C. k The second character following the character of type 2 symbol is determined as character sequence C. k The attribute value of the corresponding business attribute;

[0178] When the business attributes and attribute values ​​corresponding to N character sequences are obtained, the business attributes and attribute values ​​corresponding to each character sequence are combined to obtain the first structured result for the first business data.

[0179] In one possible implementation, the N attribute values ​​include attribute value A. i, where i is a positive integer less than or equal to N; the local confidence determination module 13 is used to obtain the first logarithmic probability value slices corresponding to N attribute values ​​from the first logarithmic probability value sequence, and to integrate the data of the first logarithmic probability value slices corresponding to each attribute value to obtain the first local confidence values ​​corresponding to N attribute values. Specifically, the local confidence determination module 13 is used to perform the following operations:

[0180] Get attribute value A i The included M characters are used to form attribute value A by combining the logarithmic probability values ​​corresponding to the M characters in the first logarithmic probability value sequence. i The corresponding first logarithmic probability value slice; M is a positive integer;

[0181] For attribute value A i The attribute value A is obtained by averaging the M log probability values ​​in the first log probability value slice. i The corresponding first local confidence value.

[0182] In one possible implementation, when the global confidence determination module 14 performs data statistics on the first local confidence values ​​corresponding to the N attribute values ​​to obtain the first global confidence value for the first text result, the global confidence determination module 14 is specifically used to perform the following operations:

[0183] Obtain the attribute weights corresponding to N business attributes;

[0184] By using the attribute weights corresponding to N business attributes, the first local confidence scores corresponding to the N attribute values ​​are weighted and summed to obtain the first global confidence score for the first text result.

[0185] In one possible implementation, when the global confidence determination module 14 obtains the attribute weights corresponding to N business attributes, the global confidence determination module 14 is specifically used to perform the following operations:

[0186] The N business attributes and the first business data are input into the importance detection model. The importance detection model is used to detect the correlation between the N business attributes and the first business data, and the attribute weights corresponding to the N business attributes are obtained.

[0187] Alternatively, in the importance detection model, the importance of N business attributes is detected based on the first business data to obtain the importance of each business attribute, and an attribute weight is assigned to each business attribute based on the importance of each business attribute.

[0188] In one possible implementation, when the global confidence determination module 14 obtains the attribute weights corresponding to N business attributes, the global confidence determination module 14 is specifically used to perform the following operations:

[0189] Obtain the first attribute category corresponding to each of the N business attributes, as well as the preset category weight information; the preset category weight information includes the attribute category and the category weight of the attribute category; the attribute category includes the first attribute category;

[0190] Obtain the category weights of the first attribute category for each of the N business attributes from the preset category weight information;

[0191] Normalize the category weights corresponding to the N business attributes to obtain the attribute weights corresponding to the N business attributes.

[0192] In one possible implementation, the N business attributes include business attribute B. i N attribute values, including attribute value A i Attribute value A i For business attribute B i The corresponding attribute value, i, is a positive integer less than or equal to N; the global confidence determination module 14 is used to perform a weighted summation of the first local confidence values ​​corresponding to the N attribute values ​​using the attribute weights corresponding to the N business attributes, in order to obtain the first global confidence value for the first text result. Specifically, the global confidence determination module 14 is used to perform the following operations:

[0193] Obtain business attribute B from the first logarithmic probability value sequence. i The corresponding second logarithmic probability value fragment is then integrated to obtain business attribute B. i The corresponding attribute confidence value;

[0194] Obtain the first weighted weight and the second weighted weight based on business attribute B. i The corresponding attribute weights and the first weighted weight determine the business attribute B. i The attribute weighting is based on business attribute B. i The corresponding attribute weight and second weight are used to determine the attribute value A. i The values ​​are weighted; the first weight is used to represent the weight percentage of the business attribute in the same group of business attributes and attribute values; the second weight is used to represent the weight percentage of the attribute value in the same group of business attributes and attribute values.

[0195] Based on business attribute B i The attribute weights and attribute values ​​A i The value of the weighted weight is applied to business attribute B. iThe corresponding attribute confidence value and attribute value A i The corresponding first local confidence scores are weighted and averaged to obtain the result for attribute value A. i The weighted confidence score;

[0196] When the weighted confidence scores corresponding to the N attribute values ​​are obtained, the sum of the N weighted confidence scores is determined as the first global confidence score for the first text result.

[0197] In one possible implementation, the data processing apparatus further includes a business processing result acquisition module 15, which is specifically used to perform the following operations:

[0198] If the first local confidence value corresponding to the key attribute is less than the local confidence threshold, or the first global confidence value is less than the global confidence threshold, then an abnormal confidence result is determined for the first text result. Based on the abnormal confidence result, a data prompt text is generated. The key attribute refers to the business attribute among N business attributes whose attribute weight is greater than or equal to the key weight threshold. The data prompt text is used to prompt the business object associated with the first business data to input updated business data in the business model.

[0199] If the first local confidence value corresponding to the key attribute is greater than or equal to the local confidence threshold, and the first global confidence value is greater than or equal to the global confidence threshold, then the business processing result for the first business data is obtained based on the first structured result, and the business processing result is output.

[0200] In one possible implementation, the first text result is the request result for the first business data; when the business processing result acquisition module 15 is used to acquire the business processing result for the first business data based on the first structured result, the business processing result acquisition module 15 is specifically used to perform the following operations:

[0201] From the N business attributes in the first structured result, obtain the attribute value corresponding to the first business attribute; the first business attribute refers to the business attribute whose attribute category is result attribute among the N business attributes;

[0202] The attribute value corresponding to the first business attribute is determined as the business processing result for the first business data.

[0203] In one possible implementation, the first text result is the result of intent parsing for the first business data; when the business processing result acquisition module 15 is used to acquire the business processing result for the first business data based on the first structured result, the business processing result acquisition module 15 is specifically used to perform the following operations:

[0204] Obtain H media data associated with the first structured result, and obtain the data correlation degree between each of the H media data and the first structured result; H is a positive integer;

[0205] Based on the data correlation between H media data and the first structured result, the associated media data corresponding to the first structured result is obtained from the H media data;

[0206] Based on the first business data, the associated media data is integrated to obtain the business processing results for the first business data.

[0207] In one possible implementation, the data processing apparatus further includes a confidence adjustment module 16, which is specifically used to perform the following operations:

[0208] When second business data associated with the first business data is obtained through the business model, the second business data is parsed to generate a second text result for the second business data, as well as a second log probability value sequence corresponding to the second text result;

[0209] Based on the second text result and the second log probability value sequence, a second global confidence value is generated for the second text result;

[0210] The association attribute values ​​associated with the second text result are obtained from the first structured result. Based on the first local confidence value corresponding to the association attribute value, the second global confidence value is adjusted to obtain the updated global confidence value for the second text result. The N attribute values ​​include the association attribute values.

[0211] In one possible implementation, when the text result acquisition module 11 is used to acquire the first text result output by the business model for the first business data, and the first log probability value sequence corresponding to the first text result, the text result acquisition module 11 is specifically used to perform the following operations:

[0212] The first business data and the format prompt instructions for the first business data are input into the business model. The business model performs text splitting on the first business data to obtain the word sequence corresponding to the first business data. The format prompt instructions are used to instruct the business model to output data with a structured format.

[0213] Encode each word in the word sequence to obtain the text encoding features of the first business data;

[0214] The format prompt instructions guide the business model to perform text prediction based on text encoding features, generating multiple characters and the corresponding log probability value for each character;

[0215] The first text result is composed of multiple characters, and the log probability value corresponding to each character is used to form the first log probability value sequence corresponding to the first text result.

[0216] Through the above process, the first text output of the model is aligned to obtain the first structured result. Based on the first structured result, local and global confidence scores are determined for the business model's output, achieving structured parsing of the model's output. This ensures that the confidence score determination considers not only the model's own output but also its structured information, thus improving the comprehensiveness and accuracy of the information referenced in the confidence score determination. Furthermore, generating a global confidence score based on the local confidence score ensures that the global confidence score fully considers the structured data composition of the first text output of the business model, achieving attribute-based parsing of the first text output. This allows for a more accurate evaluation of the accuracy and usability of the first text output of the business model through two dimensions of confidence scores (first local confidence score and first global confidence score). This further reduces the risk of downstream processing programs processing based solely on the first text or first structured result.

[0217] Please see Figure 9 , Figure 9 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Figure 9 As shown, the computer device 900 in this embodiment may include a processor 901, a network interface 904, and a memory 905. Furthermore, the computer device 900 may also include a user interface 903 and at least one communication bus 902. The communication bus 902 is used to enable communication between these components. The user interface 903 may include a display screen and a keyboard; optionally, the user interface 903 may also include a standard wired interface or a wireless interface. The network interface 904 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 905 may be a high-speed RAM or non-volatile memory, such as at least one disk storage device. Optionally, the memory 905 may also be at least one storage device located remotely from the processor 901. Figure 9 As shown, the memory 905, which is a computer-readable storage medium, may include an operating system, a network communication module, a user interface module, and a device control application program.

[0218] Network interface 904 provides network communication elements; user interface 903 is mainly used to provide an input interface for users; and processor 901 can be used to call the device control application stored in memory 905 to perform the following operations:

[0219] Obtain the first text result output by the business model for the first business data, and the first log probability value sequence corresponding to the first text result; the first log probability value sequence includes the log probability value corresponding to each character in the first text result;

[0220] The first text result is subjected to key-value pair alignment to obtain the first structured result for the first business data; the first structured result includes N business attributes and the attribute values ​​corresponding to the N business attributes; N is a positive integer;

[0221] From the first log probability value sequence, obtain the first log probability value slices corresponding to N attribute values ​​respectively, integrate the data of the first log probability value slices corresponding to each attribute value, and obtain the first local confidence values ​​corresponding to N attribute values ​​respectively;

[0222] The first local confidence scores corresponding to the N attribute values ​​are statistically analyzed to obtain the first global confidence score for the first text result. The first local confidence score and the first global confidence score are used to represent the credibility of the first text result when data processing is performed based on the first text result.

[0223] Furthermore, it should be noted that embodiments of this application also provide a computer-readable storage medium storing a computer program adapted to be loaded and executed by the processor. Figure 3 or Figure 5 For details on the methods provided in each step, please refer to the document. Figure 3 or Figure 5 The implementation methods provided for each step are not repeated here. Furthermore, the beneficial effects of using the same method are also not repeated. For technical details not disclosed in the computer-readable storage medium embodiments involved in this application, please refer to the description of the method embodiments of this application. As an example, a computer program may be deployed to execute on a single computer device, or on multiple computer devices located in one location, or on multiple computer devices distributed across multiple locations and interconnected via a communication network.

[0224] The computer-readable storage medium can be the apparatus provided in any of the foregoing embodiments or the internal storage unit of the computer device, such as the hard disk or memory of the computer device. The computer-readable storage medium can also be an external storage device of the computer device, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., provided on the computer device. Furthermore, the computer-readable storage medium can include both internal storage units and external storage devices of the computer device. The computer-readable storage medium is used to store the computer program and other programs and data required by the computer device. The computer-readable storage medium can also be used to temporarily store data that has been output or will be output.

[0225] This application also provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform... Figure 3 or Figure 5 The methods provided in the various optional modes are already available, so they will not be elaborated upon here.

[0226] The terms "first," "second," etc., in the specification, claims, and drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the term "comprising," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, apparatus, product, or device that includes a series of steps or units is not limited to the listed steps or modules, but may optionally include steps or modules not listed, or may optionally include other step units inherent to these processes, methods, apparatuses, products, or devices.

[0227] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0228] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of each example have been generally described in terms of functionality. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this application.

[0229] The methods and related apparatus provided in this application are described with reference to the method flowcharts and / or structural diagrams provided in this application. Specifically, each block of the method flowchart and / or structural diagram, as well as combinations of blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable device to create a machine, such that the instructions, which execute via the processor of the computer or other programmable device, generate instructions for implementing the process. Figure 1 A schematic diagram of one or more processes and / or structures. Figure 1 The computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable device to function in a particular manner, causing the instructions stored in the computer-readable storage medium to produce an article of manufacture including the instruction means, or to be transmitted via a computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The instruction means is implemented in the process. Figure 1 A schematic diagram of one or more processes and / or structures. Figure 1 The functions specified in one or more boxes. These computer program instructions may also be loaded onto a computer or other programmable device to cause a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable device for implementing the process. Figure 1 A process or multiple processes and / or structures illustrate the steps of the functions specified in one or more boxes.

[0230] The steps in the method of this application embodiment can be adjusted, combined, or deleted according to actual needs.

[0231] The modules in the device of this application embodiment can be merged, divided, and deleted according to actual needs.

[0232] The above-disclosed embodiments are merely preferred embodiments of this application and should not be construed as limiting the scope of this application. Therefore, any equivalent variations made in accordance with the claims of this application shall still fall within the scope of this application.

Claims

1. A data processing method, characterized in that, The method includes: Obtain the first text result output by the business model for the first business data, and the first log probability value sequence corresponding to the first text result; the first log probability value sequence includes the log probability value corresponding to each character in the first text result; The first text result is subjected to key-value pair alignment processing to obtain a first structured result for the first business data; the first structured result includes N business attributes and attribute values ​​corresponding to the N business attributes; N is a positive integer; From the first log probability value sequence, obtain the first log probability value slices corresponding to N attribute values ​​respectively, integrate the data of the first log probability value slices corresponding to each attribute value, and obtain the first local confidence value corresponding to the N attribute values ​​respectively. The first local confidence scores corresponding to the N attribute values ​​are statistically analyzed to obtain the first global confidence score for the first text result; the first local confidence score and the first global confidence score are used to represent the credibility of the first text result when data processing is performed based on the first text result.

2. The method according to claim 1, characterized in that, The step of performing key-value pair alignment on the first text result to obtain a first structured result for the first business data includes: The first text result is parsed character by character to obtain the character type of each character in the first text result; Based on the characters of type 1 symbol in the first text result, the first text result is split into N character sequences; the N character sequences include character sequence C. k k is a positive integer less than or equal to N; the character of the first symbol type is used to distinguish characters associated with different business attributes; The character sequence C k The first character preceding the character of type second symbol is determined to be the character sequence C. k The corresponding business attributes will be the character sequence C k The second character following the character of type second symbol is determined to be the character sequence C. k The attribute value of the corresponding business attribute; When the business attributes and attribute values ​​corresponding to N character sequences are obtained, the business attributes and attribute values ​​corresponding to each character sequence are combined to obtain the first structured result for the first business data.

3. The method according to claim 1, characterized in that, N attribute values, including attribute value A i , where i is a positive integer less than or equal to N; the step of obtaining N log-probability value slices corresponding to each attribute value from the first log-probability value sequence, integrating the data of the first log-probability value slices corresponding to each attribute value, and obtaining the first local confidence values ​​corresponding to the N attribute values ​​respectively includes: Obtain the attribute value A i The attribute value A is formed by taking the logarithmic probability values ​​corresponding to the M characters in the first logarithmic probability value sequence. i The corresponding first logarithmic probability value slice; M is a positive integer; For the attribute value A i The attribute value A is obtained by averaging the M log probability values ​​in the first log probability value slice. i The corresponding first local confidence value.

4. The method according to claim 1, characterized in that, The step of performing data statistics on the first local confidence scores corresponding to the N attribute values ​​respectively to obtain the first global confidence score for the first text result includes: Obtain the attribute weights corresponding to the N business attributes respectively; By using the attribute weights corresponding to the N business attributes respectively, the first local confidence scores corresponding to the N attribute values ​​are weighted and summed to obtain the first global confidence score for the first text result.

5. The method according to claim 4, characterized in that, The step of obtaining the attribute weights corresponding to the N business attributes includes: The N business attributes and the first business data are input into the importance detection model. The importance detection model is used to detect the correlation between the N business attributes and the first business data, and the attribute weights corresponding to the N business attributes are obtained. Alternatively, in the importance detection model, the importance of the N business attributes is detected based on the first business data to obtain the importance of each business attribute, and an attribute weight is assigned to each business attribute based on the importance of each business attribute.

6. The method according to claim 4, characterized in that, The step of obtaining the attribute weights corresponding to the N business attributes includes: Obtain the first attribute category corresponding to each of the N business attributes, and the preset category weight information; the preset category weight information includes the attribute category and the category weight of the attribute category; the attribute category includes the first attribute category; From the preset category weight information, obtain the category weights of the first attribute categories corresponding to the N business attributes respectively; The category weights corresponding to the N business attributes are normalized to obtain the attribute weights corresponding to the N business attributes.

7. The method according to claim 4, characterized in that, The N business attributes include business attribute B. i The N attribute values ​​include attribute value A. i The attribute value A i For the business attribute B i The corresponding attribute value, i, is a positive integer less than or equal to N; the step of using the attribute weights corresponding to the N business attributes to perform a weighted summation of the first local confidence values ​​corresponding to the N attribute values ​​to obtain the first global confidence value for the first text result includes: Obtain the business attribute B from the first logarithmic probability value sequence. i The corresponding second logarithmic probability value slice is then used to integrate the data to obtain the business attribute B. i The corresponding attribute confidence value; Obtain the first weighted weight and the second weighted weight based on the business attribute B. i The business attribute B is determined by the corresponding attribute weight and the first weighted weight. i The attribute weighting is based on the business attribute B. i The corresponding attribute weight and the second weighting weight are used to determine the attribute value A. i The first weighted weight is used to represent the weight percentage of the business attribute in the same group of business attributes and attribute values; the second weighted weight is used to represent the weight percentage of the attribute value in the same group of business attributes and attribute values. Based on the aforementioned business attribute B i The attribute weighting and the attribute value A i The value of the weighted weight is applied to the business attribute B. i The corresponding attribute confidence value and attribute value A i The corresponding first local confidence scores are weighted and averaged to obtain the result for attribute value A. i The weighted confidence score; When the weighted confidence scores corresponding to the N attribute values ​​are obtained, the sum of the N weighted confidence scores is determined as the first global confidence score for the first text result.

8. The method according to claim 1, characterized in that, The method further includes: If the first local confidence value corresponding to the key attribute is less than the local confidence threshold, or the first global confidence value is less than the global confidence threshold, then an abnormal confidence result is determined for the first text result. Based on the abnormal confidence result, a data prompt text is generated. The key attribute refers to the business attribute among the N business attributes whose attribute weight is greater than or equal to the key weight threshold. The data prompt text is used to prompt the business object associated with the first business data to input updated business data in the business model. If the first local confidence value corresponding to the key attribute is greater than or equal to the local confidence threshold, and the first global confidence value is greater than or equal to the global confidence threshold, then the business processing result for the first business data is obtained based on the first structured result, and the business processing result is output.

9. The method according to claim 8, characterized in that, The first text result is the request result for the first business data; obtaining the business processing result for the first business data based on the first structured result includes: From the N business attributes in the first structured result, obtain the attribute value corresponding to the first business attribute; the first business attribute refers to the business attribute whose attribute category is result attribute among the N business attributes; The attribute value corresponding to the first business attribute is determined as the business processing result for the first business data.

10. The method according to claim 8, characterized in that, The first text result is the result of intent parsing of the first business data; obtaining the business processing result for the first business data based on the first structured result includes: Obtain H media data associated with the first structured result, and obtain the data correlation degree between the H media data and the first structured result respectively; H is a positive integer; Based on the data correlation between the H media data and the first structured result, the associated media data corresponding to the first structured result is obtained from the H media data; The associated media data is integrated based on the first business data to obtain the business processing result for the first business data.

11. The method according to claim 1, characterized in that, The method further includes: When second business data associated with the first business data is obtained through the business model, the second business data is parsed to generate a second text result for the second business data and a second logarithmic probability value sequence corresponding to the second text result; Based on the second text result and the second log probability value sequence, a second global confidence value is generated for the second text result; The association attribute value associated with the second text result is obtained from the first structured result. Based on the first local confidence value corresponding to the association attribute value, the second global confidence value is adjusted to obtain an updated global confidence value for the second text result. The N attribute values ​​include the association attribute value.

12. The method according to claim 1, characterized in that, The acquisition of the first text result output by the business model for the first business data, and the first logarithmic probability value sequence corresponding to the first text result, includes: The first business data and the format prompt instruction for the first business data are input into the business model. The business model performs text splitting on the first business data to obtain the word sequence corresponding to the first business data. The format prompt instruction is used to instruct the business model to output data with a structured format. Each word in the word sequence is encoded to obtain the text encoding features of the first business data; Using the aforementioned format prompt instructions, the business model is guided to perform text prediction on the text encoding features, generating multiple characters and the corresponding log probability value for each character; The multiple characters are combined to form a first text result, and the log probability value corresponding to each character is combined to form a first log probability value sequence corresponding to the first text result.

13. A data processing apparatus, characterized in that, The device includes: The text result acquisition module is used to acquire the first text result output by the business model for the first business data, and the first log probability value sequence corresponding to the first text result; the first log probability value sequence includes the log probability value corresponding to each character in the first text result; The alignment processing module is used to perform key-value pair alignment processing on the first text result to obtain a first structured result for the first business data; the first structured result includes N business attributes and attribute values ​​corresponding to the N business attributes; N is a positive integer; The local confidence determination module is used to obtain the first log probability value slices corresponding to N attribute values ​​from the first log probability value sequence, integrate the data of the first log probability value slices corresponding to each attribute value, and obtain the first local confidence value corresponding to the N attribute values ​​respectively. The global confidence determination module is used to perform data statistics on the first local confidence values ​​corresponding to the N attribute values ​​respectively, and obtain the first global confidence value for the first text result; the first local confidence value and the first global confidence value are used to represent the credibility of the first text result when data processing is performed based on the first text result.

14. A computer device, characterized in that, Includes processor, memory, and input / output interfaces; The processor is connected to the memory and the input / output interface respectively, wherein the input / output interface is used to receive data and output data, the memory is used to store computer programs, and the processor is used to call the computer programs so that the computer device executes the method according to any one of claims 1-12.

15. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program adapted to be loaded and executed by a processor to cause a computer device having the processor to perform the method of any one of claims 1-12.

16. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the method described in any one of claims 1-12.

Images