Question and answer processing method and device based on flow chart

Abstract

The invention provides a question and answer processing method and device based on a flow chart, and relates to the technical field of artificial intelligence. The method comprises the steps of obtaining a to-be-processed problem and a to-be-processed flow; identifying element information in the to-be-processed flow chart according to a specified dimension to generate content description of the to-be-processed flow chart; calculating the matching degree between the content description and the to-be-processed flow chart; generating target description of the to-be-processed flow chart based on the content description and the specified comparison result; and analyzing and reasoning the to-be-processed question by using the target description, and generating a target reply corresponding to the to-be-processed question. In the embodiment of the invention, the target description of the to-be-processed flow chart can be accurately generated based on the matching degree and the content description, and finally the target reply of the to-be-processed question is analyzed and reasoned by using the target description, so that the question answering efficiency and accuracy based on the flow chart can be effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a question-answering processing method and device based on a flowchart. Background Art

[0002] As an important information display tool, flowcharts play a key role in many fields. They can intuitively display complex workflows, system architectures or algorithm logic, and present each step, link and the relationship between them in a graphical way, making it easier for people to understand and grasp. In flowchart-based question-answering tasks, it is necessary to accurately and comprehensively understand the content of the flowchart in order to give appropriate responses to user questions.

[0003] However, the interpretation of flowcharts is currently mainly done manually, which is not only time-consuming and labor-intensive, but also has inconsistent understanding levels among different interpreters, resulting in uneven interpretation quality, which in turn leads to low execution efficiency and accuracy of flowchart-based question-and-answer tasks. Summary of the invention

[0004] In view of the above problems, the purpose of the present invention is to provide a flowchart-based question and answer processing method and device, which can automatically and accurately interpret the flowchart to improve the efficiency and accuracy of flowchart-based question and answer processing.

[0005] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0006] In one aspect, the present invention provides a question-answering processing method based on a flowchart, comprising:

[0007] Obtain pending issues and pending process diagrams;

[0008] Identifying element information in the flowchart to be processed according to a specified dimension to generate a content description of the flowchart to be processed, wherein the content description includes a sub-description corresponding to the specified dimension;

[0009] Calculating the content description and the matching degree and a specified comparison result to generate a target description of the flowchart to be processed, wherein the specified comparison result is a comparison result of the matching degree and a preset threshold value;

[0010] The target description is used to analyze and reason about the problem to be processed, and a target answer corresponding to the problem to be processed is generated.

[0011] On the other hand, the present invention also provides a question-answering processing device based on a flowchart, comprising:

[0012] An acquisition module is used to obtain pending issues and pending flow charts;

[0013] An identification module, used for identifying element information in the flowchart to be processed according to a specified dimension to generate a content description of the flowchart to be processed, wherein the content description includes a sub-description corresponding to the specified dimension;

[0014] A calculation module, used for calculating the matching degree between the content description and the flowchart to be processed;

[0015] A target generation module, used to generate a target description of the flowchart to be processed based on the content description and a specified comparison result, wherein the specified comparison result is a comparison result between the matching degree and a preset threshold value;

[0016] The question-answering module is used to analyze and reason about the problem to be processed using the target description to generate a target answer corresponding to the problem to be processed.

[0017] On the other hand, the present invention also provides an electronic device, including a processor and a memory, wherein the memory stores multiple instructions; the processor loads instructions from the memory to execute the steps in any one of the flowchart-based question and answer processing methods provided by the present invention.

[0018] On the other hand, the present invention also provides a computer-readable storage medium, which stores multiple instructions, and the instructions are suitable for a processor to load to execute the steps in any one of the flowchart-based question and answer processing methods provided by the present invention.

[0019] On the other hand, an embodiment of the present invention further provides a computer program product, including a computer program / instructions, which, when executed by a processor, implements the steps of any one of the flowchart-based question-and-answer processing methods provided by the present invention.

[0020] The beneficial effects brought about by the technical solution provided by the present invention include at least:

[0021] In the embodiment of the present invention, the problem to be processed and the flowchart to be processed are obtained; the element information in the flowchart to be processed is identified according to the specified dimension to generate the content description of the flowchart to be processed; then the matching degree between the content description and the flowchart to be processed is calculated, and the consistency between the content description and the flowchart to be processed is measured by the matching degree and the specified comparison result of the preset threshold. Then, the target description of the flowchart to be processed is generated by using the content description and the specified comparison result, which can accurately and comprehensively describe the content in the flowchart to be processed. Finally, the target answer to the problem to be processed is analyzed and inferred using the target description, which can effectively improve the efficiency and accuracy of flowchart-based question and answer. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0023] Figure 1 It is a schematic diagram of an application scenario of a question-answering processing method based on a flowchart provided in an embodiment of the present invention;

[0024] Figure 2 is a flowchart diagram of a question-and-answer processing method based on a flowchart provided by an embodiment of the present invention;

[0025] Figure 3 is a schematic diagram of generating a target description provided by an embodiment of the present invention;

[0026] Figure 4 is a schematic diagram of correcting a content description to a target description provided by an embodiment of the present invention;

[0027] Figure 5 is a structural schematic diagram of a question-and-answer processing device based on a flowchart provided in an embodiment of the present invention;

[0028] Figure 6 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0029] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0030] The present invention provides a flow chart-based question and answer processing method and device, which can quickly and accurately interpret the flow chart and improve the efficiency and accuracy of question and answer.

[0031] It is understandable that in the specific implementation of the present invention, data related to user information, etc., needs to obtain user permission or consent, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

[0032] See also Figure 1, shows a schematic diagram of an application scenario of a question-answering processing method based on a flowchart. The application scenario may include a terminal 101 and a server 102, and data may be exchanged between the terminal 101 and the server 102 via a network, and a corresponding application may be installed on the terminal 101. The terminal 101 may be a mobile phone, a tablet computer, a smart Bluetooth device, a computer, a large screen device, a robot, etc.; the server 102 may be a single server or a server cluster consisting of multiple servers.

[0033] The user can send the problem to be processed and the flowchart to be processed to the server 102 through the terminal 101, so that the server 102 can identify the element information in the flowchart to be processed according to the specified dimension, and generate a sub-description corresponding to each specified dimension as a content description; calculate the matching degree between the content description and the flowchart to be processed, and then generate a target description of the flowchart to be processed based on the matching degree and the specified comparison result of the preset threshold and the content description; finally, the target description can be used to analyze and reason about the problem to be processed to generate a target answer corresponding to the problem to be processed.

[0034] The target answer may be sent by the server 102 to the terminal 101 so that the terminal 101 may present the target answer to the user.

[0035] In this embodiment, a question-answering processing method based on a flowchart is provided, such as Figure 2 As shown, the specific process of the question-answering processing method based on the flowchart can be as follows:

[0036] S110, obtaining pending issues and pending process flow charts.

[0037] The question to be processed is the question text input by the user, where the content input by the user can be in text form or in voice form. If the input content is in text form, all the content input by the user can be directly used as the text to be processed; if the input content is in voice form, it can be converted into text through language-to-text technology to obtain the question to be processed.

[0038] The flowchart to be processed is the flowchart required to solve the problem to be processed. The flowchart is a graphical tool used to represent the sequence and logical relationship of a process, system or algorithm. It uses standardized symbols and graphics to show the sequence, branches, loops and decision points between steps to clearly express complex processes. A flowchart can contain a variety of elements, such as nodes, arrows, text, etc.

[0039] The flowchart to be processed may be input together with the problem to be processed, for example, the flowchart to be processed and the problem to be processed may be directly input together. For another example, the location of the flowchart to be processed in the document may be described in words, and the document may be provided to the server. The server may locate and extract the flowchart in the document based on the content provided by the user as the flowchart to be processed.

[0040] S120: Identify element information in the flowchart to be processed according to a specified dimension to generate a content description of the flowchart to be processed.

[0041] The specified dimension refers to the direction of parsing and analyzing the process to be processed, and the number and content of the specified dimension can be set in advance according to actual needs. In an embodiment of the present invention, the specified dimension may include node type and text dimension, arrow and connection dimension, logical relationship dimension between nodes, and spatial position dimension of nodes.

[0042] Among them, the flowchart to be processed is identified according to the specified dimension, and the corresponding element information in the flowchart to be processed can be identified. For example, in the node type and text dimension, the type, shape, text content in the node, etc. of each node can be identified; in the arrow and connection dimension, the starting point, end point and flow direction of each arrow or line can be identified; in the logical relationship dimension between nodes, the logical relationship between nodes can be identified; in the spatial position dimension of the nodes, the relative positions between the nodes can be identified.

[0043] In each specified dimension, the corresponding element information can be identified and described in text form as the sub-description corresponding to the specified dimension, and then the content description of the flowchart to be processed can be generated. That is, the content description can include the sub-description corresponding to each specified dimension, so as to provide a comprehensive and complete description of the flowchart to be processed.

[0044] In some implementations, optical character recognition (OCR) technology may also be used to recognize text information, graphic element layout and structure, etc. in the flowchart to be processed, and then the large language model may be used to integrate the content recognized by OCR according to a specified dimension to obtain a content description.

[0045] In some embodiments, in order to improve the accuracy of the content description, when generating the content description of the flowchart to be processed, the image quality of the flowchart to be processed can be optimized to obtain an optimized flowchart; based on the optimized flowchart and the recognition template, recognition prompt words are generated, and the recognition prompt words include recognition requirements and output requirements corresponding to each specified dimension; with the recognition prompt words, the recognition model is guided to identify the element information corresponding to each specified dimension from the optimized flowchart according to the recognition requirements; for each specified dimension, the element information is organized according to the output requirements to obtain the sub-description corresponding to the specified dimension; the sub-descriptions corresponding to all specified dimensions are used as the content description of the flowchart to be processed.

[0046] The flowchart to be processed is usually in the form of an image, and its image quality can be optimized to improve the accuracy of subsequent recognition. The optimization process can include grayscale processing, denoising processing, enhancement processing, etc. Among them, grayscale processing can convert the colored flowchart to be processed into a grayscale image to reduce the interference of color, lighting, shadow, etc. Denoising processing can remove the noise in the flowchart to be processed through an algorithm, reduce interference factors, and improve the subsequent recognition effect. Enhancement processing can make the image clearer by adjusting the contrast, brightness and other parameters of the flowchart to be processed. That is, the image quality of the optimized flowchart obtained after the optimization process is higher, clearer and easier to identify later.

[0047] As an implementation method, when performing optimization processing to obtain an optimized flowchart, the flowchart to be processed can be grayscaled to obtain a grayscale flowchart; the grayscale flowchart is filtered to obtain a filtered flowchart; the edges of key structures in the filtered flowchart are enhanced to obtain an edge-enhanced flowchart; the filtered flowchart is contrast-enhanced to obtain a contrast-enhanced flowchart; the edge-enhanced flowchart and the contrast-enhanced flowchart are superimposed according to specified weights to obtain an optimized flowchart.

[0048] Grayscale processing can be to convert a color image into a grayscale image according to a conversion formula, wherein the conversion formula is: Y = 0.2989 R + 0.5870 G + 0.1140 B, wherein R is the pixel value of the red channel in the image, G is the pixel value of the green channel in the image; B is the pixel value of the blue channel in the image; and Y is the grayscale value after conversion.

[0049] The filtering process is to eliminate the noise in the image. There are various ways to obtain the flowchart to be processed. It may be an image converted from a paper document and may contain scanning noise. In order to remove noise while retaining edge details, the grayscale flowchart can be processed by bilateral filtering. The filtering process considers the spatial distance between each pixel and the surrounding pixels and the similarity of pixel values. The pixels in the neighborhood are weighted averaged. The neighborhood diameter determines the range of filtering, and the color space and coordinate space standard deviations affect the intensity of filtering and the degree of edge retention. Among them, the neighborhood diameter, color space, and coordinate space standard deviations can all be set according to actual needs. In the embodiment of the present invention, the neighborhood diameter is set to 9, and the color space and coordinate space standard deviations are both set to 75.

[0050] Edge enhancement is to highlight the key structure in the filtering flow chart. The key structure can be set according to actual needs. In the embodiment of the present invention, the key structure can include nodes, arrows, connecting lines, etc. Among them, when performing edge enhancement, the gradient amplitude and direction of each pixel in the image can be calculated based on the filtering flow chart. The amplitude can be used to characterize the intensity of the brightness change of the image at the pixel, and the gradient direction indicates the direction of the brightness change. Then non-maximum suppression can be performed, that is, the pixels whose gradient amplitude is not the local maximum value are set to 0, so as to refine the edge and make it a thin line with a single pixel width; set a high threshold and a low threshold, and perform threshold processing on the gradient image to obtain strong edges and weak edges. Track the weak edges, retain the weak edges connected to the strong edges, and remove the isolated weak edges. Use morphological dilation operation to make the edges more coarse. Morphological dilation is based on structural elements, and selecting appropriate structural elements can control the degree and shape of dilation. For example, using a 3×3 square structural element for dilation can make the edge extend outward by a distance of one pixel, so that the lines and edges in the filtering flow chart are more prominent.

[0051] Contrast enhancement is to improve the readability of text and details in the flowchart to be processed. Specifically, it can be achieved by using a contrast-limited adaptive histogram equalization method to divide the filtered flowchart into blocks, equalize the contrast within these blocks respectively, and set the contrast limit to 2.0.

[0052] After obtaining the edge enhancement flowchart and the contrast enhancement flowchart, the two flowcharts can be superimposed according to the specified weights to obtain an optimized flowchart. The specified weights can be set according to actual needs. In an embodiment of the present invention, the weight of the edge enhancement flowchart is set to 30%, and the weight of the contrast enhancement flowchart is set to 70%. After superimposing the images according to the weights, an intermediate flowchart is obtained. At this time, the intermediate flowchart is also a grayscale image. The intermediate flowchart can be converted into an RGB image to obtain the final optimized flowchart.

[0053] The recognition template refers to a pre-set prompt word template for parsing the flowchart, and the recognition template can be set in advance according to actual needs. The recognition template can be provided with corresponding flowchart slots, as well as recognition requirements and output requirements corresponding to each specified dimension. The flowchart slots can be used to fill the optimized flowchart. By filling the optimized flowchart into the recognition template, the recognition prompt words can be generated. The recognition prompt words can be used to guide the recognition model to parse and recognize the optimized flowchart. The recognition prompt words also include the recognition requirements and output requirements corresponding to each specified dimension. That is, for each specified dimension, the recognition requirements and the output requirements of the recognized content can be set in the recognition template according to actual needs to ensure that the parsed and recognized content has strong usability.

[0054] In the embodiment of the present invention, the recognition template may be:

[0055] Please analyze the following flowchart, list all the elements in the diagram and their relationships in a detailed text description, and output it in json format. Please include the following information:

[0056] Node type and text: Identify the type of each node (such as start node, end node, processing node, decision node, etc.) and describe its shape (such as rectangle, circle, diamond, etc.) and the text content in each node (such as start, processing, end, decision).

[0057] Arrows and Lines: Detect arrows or lines, describe the start and end points of each arrow or line, and their direction of flow (for example, from node A to node B).

[0058] Logical relationship between nodes: describes the logical relationship between nodes, such as "node A points to node B" or "node B points to node C or D based on the decision".

[0059] Spatial position of nodes: describes the relative position of nodes (such as node A is to the left of node B).

[0060] The recognition model refers to a model used to parse and recognize the flowchart to be processed. The recognition template can be a visual language model. The recognition prompt words are input into the recognition model. The recognition model can identify the corresponding element information from the optimized flowchart according to the recognition requirements of each specified dimension, and organize the element information for output according to the output requirements to obtain the sub-description corresponding to each specified dimension. The sub-descriptions corresponding to all specified dimensions are used as the content description of the flowchart to be processed.

[0061] S130: Calculate the matching degree between the content description and the flowchart to be processed.

[0062] The content description is a text that describes the information contained in the flowchart to be processed. When answering the pending questions input by the user, the answers can be directly based on the content description. It can be seen that the accuracy of the content description directly affects the accuracy of the question and answer. In order to improve the accuracy of question and answer processing, the generated content description and the flowchart to be processed can be checked for consistency. For example, the consistency between the content description and the flowchart to be processed can be checked by calculating the matching degree between the content description and the flowchart to be processed. The higher the matching degree, the stronger the consistency, and the lower the matching degree, the weaker the consistency.

[0063] In some implementations, the content description and the flowchart to be processed may be directly converted into feature vectors, and then the cosine similarity between the feature vectors may be calculated as the matching degree.

[0064] The content description is data in text mode, and the flowchart to be processed is data in image mode. In some embodiments, in order to accurately calculate the matching degree between data of different modes and better measure the internal structure and relationship of the data, when calculating the matching degree between the content description and the flowchart to be processed, the text features of the content description in the multimodal space and the image features of the flowchart to be processed in the multimodal space can be obtained; the text features are multiplied by the transpose of the text features to obtain a text kernel matrix; the image features are multiplied by the transpose of the image features to obtain an image kernel matrix; and the matching degree between the content description and the flowchart to be processed is calculated using the text kernel matrix and the image kernel matrix.

[0065] Among them, the optimized flowchart corresponding to the flowchart to be processed has been calculated above, and the content description and the optimized flowchart can be mapped to the same feature space and then calculated. Among them, the mapping can be realized with the help of a neural network model. In an embodiment of the present invention, the content description and the flowchart to be processed can be projected into the multimodal space by relying on a multimodal model, and text features and image features can be obtained.

[0066] Among them, the multimodal model needs to be trained in advance. A large number of sample flowcharts can be collected in advance, and each sample flowchart is optimized in the above-mentioned way. Then, prompt words are generated according to the optimized sample flowchart and the recognition template, and the prompt words are input into the multimodal model. The results of the multimodal model are manually corrected and sorted. Then, the multimodal model is fine-tuned using the corresponding data set, and the large language model part in the multimodal model is fixed. Only the visual encoder and projection layer are trained. After the training is completed, the mapping model can be obtained.

[0067] The word segmenter in the mapping model is used to encode the content description to obtain the text features corresponding to the content description; the visual encoder in the mapping model is used to encode the optimized flowchart and project it into the multimodal space through the projection layer to obtain the image features.

[0068] For text features, the text features and the transpose of the text features may be multiplied to calculate a text kernel matrix. Similarly, for image features, the image features and the transpose of the image features may be multiplied to calculate an image kernel matrix.

[0069] The degree of match between the content description and the flowchart to be processed can be calculated using the text kernel matrix and the image kernel matrix. Optionally, the text kernel matrix and the image kernel matrix can be multiplied to obtain a first matrix; the text kernel matrix and the text kernel matrix can be multiplied to obtain a second matrix; the image kernel matrix and the image kernel matrix can be multiplied to obtain a third matrix; the traces of the first matrix, the second matrix and the third matrix are calculated respectively, and the trace refers to the sum of the diagonal elements of the corresponding matrices to obtain the corresponding first trace, that is, the sum of the diagonal elements of the first matrix, the second trace and the third trace; the product of the second trace and the third trace is calculated and then the square root is taken to obtain a first intermediate result; the first trace is divided by the first intermediate result to obtain the degree of match.

[0070] Specifically, the matching degree can be calculated according to the following formula:

[0071] ;

[0072] Among them, C(v,t) represents the matching degree between the content description and the flowchart to be processed; Kv represents the text kernel matrix; Kt represents the image kernel matrix; Tr(KvKt) represents the first trace; Tr(KtKt) represents the second trace; Tr(KvKv) represents the third trace.

[0073] S140: Generate a target description of the flowchart to be processed based on the content description and the specified comparison result.

[0074] The comparison result is specified as the comparison result between the matching degree and the preset threshold. The preset threshold is a preset matching degree threshold. By comparing the matching degree with the preset threshold, the consistency between the content description and the flowchart to be processed can be roughly determined. The matching degree ranges from 0 to 1. The closer the matching degree is to 1, the more consistent the content description is with the flowchart to be processed. Using the comparison result between the matching degree and the preset threshold and the content description, a target description of the flowchart to be processed can be generated. The target description is the text content obtained by comprehensively and accurately describing the information contained in the flowchart to be processed. The target description is the basis for generating the answer corresponding to the problem to be processed.

[0075] In some embodiments, the preset threshold may include a first threshold and a second threshold, wherein the second threshold is less than the first threshold. When generating the target description of the flowchart to be processed, if the matching degree is not less than the first threshold, the content description is used as the target description of the flowchart to be processed; if the matching degree is less than the first threshold and not less than the second threshold, based on the sub-description corresponding to each specified dimension and the sub-matching degree of the flowchart to be processed, the content description is corrected to the target description; if the matching degree is less than the second threshold, the step of returning to perform the step of identifying the element information in the flowchart to be processed according to the specified dimension to generate the content description of the flowchart to be processed and the subsequent steps.

[0076] See also Figure 3 , shows a schematic diagram of generating a target description, comparing the matching degree with the first threshold. If the matching degree is not less than the first threshold, it indicates that the content description is highly consistent with the flowchart to be processed, and the content description has fully and accurately described the content in the flowchart to be processed, and the content description can be directly used as the target description. Among them, the first threshold can be set according to actual needs. In the embodiment of the present invention, the first threshold is 0.8, that is, when the matching degree falls within the range of [0.8,1], the content description can be directly determined as the target description.

[0077] If the matching degree is less than the first threshold, the matching degree can be compared with the second threshold. If the matching degree is not less than the second threshold, it indicates that the content description is highly consistent with the flowchart to be processed, but there are still some differences. Some content can be corrected according to the specified dimension.

[0078] Optionally, the content description can be corrected to the target description based on the sub-description of the specified dimension and the sub-matching degree of the flowchart to be processed. Figure 4 , showing a schematic diagram of correcting a content description to a target description, for example, for each specified dimension, the sub-description corresponding to the specified dimension is encoded as a sub-text feature; the sub-matching degree between each sub-text feature and the image feature of the flowchart to be processed is calculated; the sub-description corresponding to the sub-text feature with the lowest sub-matching degree is determined as the description to be corrected; using the prompt word corresponding to the dimension to be corrected and the flowchart to be processed, a corrected description corresponding to the description to be corrected is generated, and the dimension to be corrected is the specified dimension corresponding to the description to be corrected; the description to be corrected in the content description is replaced by the corrected description to obtain the target description.

[0079] For each specified dimension, the sub-description corresponding to each specified dimension can be obtained from the content description, and the word segmenter in the mapping model can be used to encode the content description to obtain the sub-text feature corresponding to the sub-description. The sub-matching degree between each sub-text feature and the image feature is calculated. The calculation method of the sub-matching degree is the same as the above-mentioned matching degree. To avoid repetition, it will not be repeated here. You can directly refer to the above-mentioned corresponding content.

[0080] The sub-matching degree corresponding to each sub-description can be obtained by calculation. The lower the sub-matching degree, the lower the consistency between the corresponding sub-description and the flowchart to be processed. As an implementation method, the sub-description corresponding to the sub-text feature with the lowest sub-matching degree can be directly determined as the description to be corrected. For example, Figure 4 In the example, if the sub-matching degree 1 is the minimum value, the corresponding sub-description 1 is the description to be corrected. As another implementation, a minimum threshold can be set according to actual needs or experience, and the sub-description corresponding to the sub-text feature with a sub-matching degree less than the minimum threshold is determined as the description to be corrected.

[0081] The corrected description corresponding to the description to be corrected is generated by using the prompt words corresponding to the dimension to be corrected and the process flow chart to be processed. The dimension to be corrected refers to the specified dimension corresponding to the description to be corrected. The prompt words corresponding to the dimension to be corrected can be obtained from the recognition prompt words, or can be set in advance. Figure 1 And input it into the recognition model for processing to obtain the corrected description.

[0082] For example, if the dimension to be corrected is the logical relationship dimension between nodes, the corresponding prompt word can be: Please analyze the following flowchart, list the logical relationship between the nodes in the diagram in a detailed text description, and output it in json format. Please include the following information: Describe the logical relationship between nodes, such as "node A points to node B" or "node B points to node C or D based on the decision."

[0083] After obtaining the corrected description, the corresponding sub-description in the content description can be replaced with the corrected description to repair part of the content description and obtain the target description. Figure 4 In the content description, there are n sub-descriptions, and sub-description 1 has the lowest matching degree. Therefore, after correcting sub-description 1, the corrected description is used to replace sub-description 1, and the other sub-descriptions remain unchanged, and the target description can be obtained.

[0084] If the matching degree is less than the second threshold, it indicates that the consistency between the content description and the flowchart to be processed is weak, the content description is not accurate enough, and needs to be regenerated. Therefore, the execution can be returned to identify the element information in the flowchart to be processed according to the specified dimension to generate the steps and steps of the content description in the flowchart to be processed.

[0085] S150: Analyze and reason about the problem to be processed using the target description to generate a target answer corresponding to the problem to be processed.

[0086] The target description can be considered as a complete and accurate description of the flowchart to be processed, that is, the content in the target description is consistent with the content in the flowchart to be processed. By using the target description to analyze and reason about the problem to be processed, the target answer corresponding to the problem to be processed can be generated.

[0087] In some implementations, the target description and the question to be processed can be directly input into the question-answering model, and the question-answering model is used to understand the target description and infer the target answer corresponding to the question to be processed. The question-answering model can be a large language model, which can be used according to actual needs.

[0088] In some embodiments, there are various questions about the flowchart. The questions to be processed may be questions about the entire flowchart to be processed, or they may be detailed questions about certain nodes in the flowchart to be processed. The scope of the flowchart to be processed involved in these two types of questions is different. In order to improve the efficiency and accuracy of question-answering, when generating the target answer corresponding to the question to be processed, the question to be processed may be subjected to text classification processing to determine the question type of the question to be processed, and the question type includes at least one of a detail question and an overall question; if the question type includes an overall question, the target description is transcribed, and the question-answering model is called to generate the target answer corresponding to the question to be processed based on the transcribed target description; if the question type includes a detail question, the question-answering model is controlled to analyze and reason the question to be processed according to the preset thinking chain to generate the target answer corresponding to the question to be processed. It is more accurate to classify the questions to be processed and select different reasoning methods according to different question types to obtain the target answer.

[0089] Text classification is performed on the questions to be processed to determine the type of the questions to be processed, wherein the question type may include at least one of a detail question and an overall question. Detail questions focus on the detailed information of a specific node or a step in the flowchart to be processed, and usually require locating a specific part of the flowchart to be processed to answer the question. Overall questions usually require understanding the entire flowchart to be processed from a macro level, which may involve the overall structure, main steps or key processes of the flowchart to be processed.

[0090] Optionally, both the overall question and the detailed question may contain some common words. Based on actual experience, common keywords in the corresponding type of questions can be summarized as preset keywords, and the preset keywords and preset types can be associated in advance as type association relationships. For example, the preset keywords associated with the overall question may include overall, main, etc., and the preset keywords associated with the detailed question may include specific, specific steps, a certain node, etc. When determining the type of question, the preset keywords contained in the question to be processed can be directly determined, and then the type of question can be determined based on the type association relationship.

[0091] Optionally, when determining the problem type of the problem to be processed, it can also be analyzed and determined with the help of a large language model, which can be pre-set with corresponding prompt words, and the prompt words are input into the large language model so that the large language model can analyze and reason about the problem to be processed and output the corresponding problem type.

[0092] In order to determine the type of question more quickly and accurately, optionally, a type mapping relationship can be obtained, wherein the type mapping relationship includes an association relationship between a preset type and a preset keyword; if the problem to be processed contains any preset keyword, the preset type corresponding to the preset keyword is used as the problem type of the problem to be processed; if the problem to be processed does not contain any preset keyword, the problem to be processed is merged with a classification template to obtain a classification prompt word, and the classification template is a prompt word template for classifying the problem to be processed; the classification prompt word is input into a classification model so that the classification model can semantically understand and classify the problem to be processed to obtain the problem type of the problem to be processed.

[0093] The keyword matching algorithm can be used to determine all preset keywords contained in the problem to be processed. If the problem to be processed contains preset keywords, the problem type corresponding to the preset keywords can be used as the problem type. If the problem to be processed does not contain preset keywords, it means that the problem type cannot be determined using keywords. The classification model can be used to determine the problem type.

[0094] The classification template refers to a prompt word template used to classify the pending problems. The classification template may contain a question slot for filling the pending problems. After the pending problems and the classification template are combined, the classification prompt words can be obtained, and then the classification prompt words are input into the classification model so that the classification model outputs a specific problem type. Optionally, since the overall problem and the detailed problem are artificially defined concepts, the relevant concepts and explanations of the overall problem and the detailed problem can be added to the classification template.

[0095] For example, the classification template in the embodiment of the present invention may be:

[0096] “Overall questions: require understanding the entire flowchart from a macro level, involving the overall structure, main steps or key processes of the flowchart.

[0097] Detail issues: Focus on the detailed information of a specific node or step in the flowchart, usually requiring locating a specific part of the flowchart.

[0098] Determine whether the following question is about the overall problem of the flowchart or a detail problem:\nQuestion: {user_question}\nAnswer: "

[0099] The classification model may be a large language model, and the problem type corresponding to the problem to be processed may be determined based on the reasoning analysis of the classification model.

[0100] If the question type includes an overall question, the target description can be transcribed, and the question-answering model can be called to generate the target answer corresponding to the pending question based on the written target description. Among them, the target description contains sub-descriptions corresponding to each specified dimension, but it is not a complete and fluent text. The question-answering model can be used to transcribe or rewrite the target description to generate a text describing the main process path. Then, based on the transcribed text, the pending question is analyzed and inferred, and the target answer is generated and output.

[0101] If the question type includes detail questions, the question-answering model can be controlled to analyze and reason the problem to be processed according to the preset thinking chain to generate the target answer corresponding to the problem to be processed. For example, it can be to identify the intention of the problem to be processed to determine the target dimension associated with the problem to be processed from the specified dimension; locate the target node associated with the problem to be processed; combine the target dimension and the target node to generate a preliminary answer corresponding to the problem to be processed; reflect and verify the preliminary answer to obtain the target answer.

[0102] In order to ensure the accuracy of the answers to the detailed questions, the question-answering model can be guided to reason step by step according to the chain of thought to obtain the final target answer. Among them, the specific reasoning steps of the preset chain of thought can be set according to actual needs. In an embodiment of the present invention, intent recognition may be performed first, that is, intent recognition is performed on the problem to be processed to identify the user's intention and determine the target dimension specifically associated with the problem to be processed. For example, the problem to be processed is "What is the next step of node A?" This problem to be processed involves the flow direction of a certain node, then the target dimension can be the arrow and line dimension.

[0103] Then, node positioning can be performed to locate the target node associated with the problem to be processed, that is, to analyze the specific nodes involved in the problem to be processed. For example, in the problem to be processed in the above example, the target node can be node A. Through intent recognition and node positioning, user questions and the specified dimensions and nodes that need to be focused on can be clearly answered. Then analyze the input, output, and relationship between each node, focus on the information related to the target node, and then combine the sub-description corresponding to the target dimension to give a preliminary answer to the problem to be processed. Then reflect and verify the preliminary answer to check whether there are any errors in the generated preliminary answer. When it is determined that there are no errors, the final target answer is obtained.

[0104] The problem to be processed can be decomposed into multiple small problems by the way of thinking chain. The question-answering model can solve each small problem in turn to get the final target answer, which can effectively improve the logic of reasoning, and thus effectively improve the accuracy of question-answering. It should be noted that if the problem to be processed contains both overall problems and detailed problems, the overall problem part of the problem to be processed can be split out, and reasoning can be performed according to the reasoning method of the overall problem, and the detailed problem part of the problem to be processed can be split out, and reasoning can be performed according to the reasoning method of the detailed problem. Finally, the reasoning results of the two parts are combined together as the target answer. Optionally, if the problem to be processed contains an overall problem, the target answer can be obtained directly according to the reasoning method of the overall problem; if the problem to be processed only contains detailed problems, the target answer can be obtained by reasoning in the way of thinking chain. It can be set according to actual needs, and no specific limitation is made here.

[0105] The flowchart-based question-and-answer solution provided by the embodiment of the present invention can be applied in various question-and-answer scenarios. For example, taking the internal question-and-answer system of an enterprise as an example, the internal management of an enterprise involves multiple processes, such as reimbursement process, approval process, etc. These processes can all be drawn with corresponding flowcharts for employees to query. The solution provided by the embodiment of the present invention can accurately interpret the flowchart and directly give the corresponding answer to the user's question without the user having to repeatedly check the flowchart.

[0106] The method provided by the embodiment of the present invention can obtain the problem to be processed and the flowchart to be processed; identify the element information in the flowchart to be processed according to the specified dimension to generate the content description of the flowchart to be processed; calculate the matching degree between the content description and the flowchart to be processed, so as to use the matching degree to verify the consistency of the content description and the content of the flowchart to be processed, and use it directly as the target description when the content is highly consistent, otherwise the content description can be corrected in blocks or as a whole to ensure the accuracy of the target description, and finally, according to different problem types, different reasoning methods are used to answer the problem to be processed based on the target description, which can effectively improve the efficiency and accuracy of flowchart-based question answering.

[0107] In order to better implement the above method, the embodiment of the present invention also provides a flow chart-based question-answer processing device, which can be integrated in an electronic device, which can be a terminal, a server, etc. Among them, the terminal can be a mobile phone, a tablet computer, a smart Bluetooth device, a laptop, a personal computer, etc.; the server can be a single server or a server cluster composed of multiple servers.

[0108] For example, in this embodiment, the method of the embodiment of the present invention is described in detail by taking the flowchart-based question and answer processing device specifically integrated in the server as an example.

[0109] For example, Figure 5 As shown, the question-and-answer processing device 200 based on the flowchart may include an acquisition module 210 , an identification module 220 , a calculation module 230 , a target generation module 240 and a question-and-answer module 250 .

[0110] An acquisition module 210 is used to acquire a problem to be processed and a flow chart to be processed;

[0111] An identification module 220, configured to identify element information in the flowchart to be processed according to a specified dimension to generate a content description of the flowchart to be processed, wherein the content description includes a sub-description corresponding to the specified dimension;

[0112] A calculation module 230, used to calculate the matching degree between the content description and the flowchart to be processed;

[0113] A target generation module 240, configured to generate a target description of the flowchart to be processed based on the content description and a specified comparison result, wherein the specified comparison result is a comparison result between the matching degree and a preset threshold value;

[0114] The question-answering module 250 is used to analyze and reason about the problem to be processed using the target description, and generate a target answer corresponding to the problem to be processed.

[0115] In some embodiments, the identification module 220 is specifically used to:

[0116] Optimizing the image quality of the flowchart to be processed to obtain an optimized flowchart;

[0117] Based on the optimized flow chart and the recognition template, generate recognition prompt words, wherein the recognition prompt words include recognition requirements and output requirements corresponding to each specified dimension;

[0118] Using the recognition prompt words, guiding the recognition model to recognize the element information corresponding to each specified dimension from the optimized flow chart according to the recognition requirements;

[0119] For each specified dimension, the element information is organized according to the output requirement to obtain a sub-description corresponding to the specified dimension;

[0120] The sub-descriptions corresponding to all designated dimensions are used as the content description of the flowchart to be processed.

[0121] In some embodiments, the identification module 220 is specifically used to:

[0122] Performing grayscale processing on the flowchart to be processed to obtain a grayscale flowchart;

[0123] Performing filtering processing on the grayscale flow chart to obtain a filtering flow chart;

[0124] Performing enhancement processing on the edges of the key structures in the filtering flowchart to obtain an edge enhancement flowchart;

[0125] Performing contrast enhancement on the filtering flow chart to obtain a contrast enhancement flow chart;

[0126] The edge enhancement flowchart and the contrast enhancement flowchart are superimposed according to the specified weights to obtain an optimized flowchart.

[0127] In some embodiments, the calculation module 230 is specifically used to:

[0128] Acquire text features of the content description in a multimodal space, and image features of the flowchart to be processed in the multimodal space;

[0129] Multiplying the text feature by the transpose of the text feature to obtain a text kernel matrix;

[0130] multiplying the image feature by the transpose of the image feature to obtain an image kernel matrix;

[0131] The matching degree between the content description and the flowchart to be processed is calculated using the text kernel matrix and the image kernel matrix.

[0132] In some embodiments, the preset threshold includes a first threshold and a second threshold, the second threshold is less than the first threshold, and the target generation module 240 is specifically used to:

[0133] If the matching degree is not less than a first threshold, taking the content description as a target description of the flowchart to be processed;

[0134] If the matching degree is less than the first threshold value and not less than the second threshold value, based on the matching degree between the sub-description corresponding to each specified dimension and the sub-description of the flowchart to be processed, correcting the content description to a target description;

[0135] If the matching degree is less than the second threshold, the process returns to execute the step of identifying the element information in the flowchart to be processed according to the specified dimension to generate the content description of the flowchart to be processed and the subsequent steps.

[0136] In some embodiments, the target generation module 240 is specifically used to:

[0137] For each specified dimension, encode the sub-description corresponding to the specified dimension as a sub-text feature;

[0138] Calculating the sub-matching degree between each of the sub-text features and the image features of the flowchart to be processed;

[0139] Determining the sub-description corresponding to the sub-text feature with the lowest sub-matching degree as the description to be corrected;

[0140] Generate a corrected description corresponding to the description to be corrected by using the prompt word corresponding to the dimension to be corrected and the flowchart to be processed, wherein the dimension to be corrected is a specified dimension corresponding to the description to be corrected;

[0141] The description to be corrected in the content description is replaced with the corrected description to obtain a target description.

[0142] In some embodiments, the question-and-answer module 250 is specifically used to:

[0143] Performing text classification processing on the problem to be processed to determine the problem type of the problem to be processed, wherein the problem type includes at least one of a detail problem and an overall problem;

[0144] If the question type includes an overall question, transcribe the target description, and call the question-answering model to generate a target answer corresponding to the question to be processed based on the transcribed target description;

[0145] If the question type includes a detail question, the control question-answering model analyzes and infers the question to be processed according to a preset thinking chain to generate a target answer corresponding to the question to be processed.

[0146] In some embodiments, the question-and-answer module 250 is specifically used to:

[0147] Acquire a type mapping relationship, where the type mapping relationship includes an association relationship between a preset type and a preset keyword;

[0148] If the problem to be processed contains any preset keyword, the preset type corresponding to the preset keyword is used as the problem type of the problem to be processed;

[0149] If the problem to be processed does not contain any preset keyword, the problem to be processed is combined with a classification template to obtain a classification prompt word, where the classification template is a prompt word template for classifying the problem to be processed;

[0150] The classification prompt words are input into the classification model so that the classification model can perform semantic understanding and classification on the problem to be processed to obtain the problem type of the problem to be processed.

[0151] In some embodiments, the question-and-answer module 250 is specifically used to:

[0152] Performing intent recognition on the problem to be processed to determine the target dimension associated with the problem to be processed from the specified dimensions;

[0153] Locating the target node associated with the problem to be processed;

[0154] Generate a preliminary answer corresponding to the problem to be processed by combining the target dimension and the target node;

[0155] Reflectively verify the preliminary responses to obtain the target responses.

[0156] In specific implementation, the above modules can be implemented as independent entities, or can be arbitrarily combined and implemented as the same or several entities. The specific implementation of the above modules can be found in the previous method embodiments, which will not be repeated here.

[0157] As can be seen from the above, the flowchart-based question-answering processing device of this embodiment can obtain the problem to be processed and the flowchart to be processed; identify the element information in the flowchart to be processed according to the specified dimension to generate the content description of the flowchart to be processed; then calculate the matching degree between the content description and the flowchart to be processed, and measure the consistency between the content description and the flowchart to be processed by the matching degree and the specified comparison result of the preset threshold. Then, the target description of the flowchart to be processed is generated by using the content description and the specified comparison result, which can accurately and comprehensively describe the content in the flowchart to be processed. Finally, the target description is used to analyze and infer the target answer to the problem to be processed, which can effectively improve the efficiency and accuracy of flowchart-based question-answering.

[0158] The embodiment of the present invention further provides an electronic device, which may be a terminal, a server, or the like. The terminal may be a mobile phone, a tablet computer, a smart Bluetooth device, a notebook computer, a personal computer, or the like; the server may be a single server or a server cluster composed of multiple servers, or the like.

[0159] In some embodiments, the flowchart-based question and answer processing device can also be integrated into multiple electronic devices. For example, the flowchart-based question and answer processing device can be integrated into multiple servers, and the flowchart-based question and answer processing method of the present invention can be implemented by multiple servers.

[0160] In this embodiment, the electronic device of this embodiment is a server as an example for detailed description, for example, Figure 6 As shown, it shows a schematic diagram of the structure of an electronic device involved in an embodiment of the present invention, specifically:

[0161] The electronic device may include components such as a processor 310 with one or more processing cores, a memory 320 with one or more computer-readable storage media, a power supply 330, an input module 340, and a communication module 350. Those skilled in the art will appreciate that Figure 6 The electronic device structure shown in the figure does not constitute a limitation on the electronic device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0162] The processor 310 is the control center of the electronic device. It uses various interfaces and lines to connect various parts of the entire electronic device. It executes various functions of the electronic device and processes data by running or executing software programs and / or modules stored in the memory 320, and calling data stored in the memory 320. In some embodiments, the processor 310 may include one or more processing cores; in some embodiments, the processor 310 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 310.

[0163] The memory 320 can be used to store software programs and modules. The processor 310 executes various functional applications and data processing by running the software programs and modules stored in the memory 320. The memory 320 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 320 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 320 may also include a memory controller to provide the processor 310 with access to the memory 320.

[0164] The electronic device also includes a power supply 330 for supplying power to various components. In some embodiments, the power supply 330 can be logically connected to the processor 310 through a power management system, so as to manage charging, discharging, and power consumption through the power management system. The power supply 330 can also include any components such as one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, and power status indicators.

[0165] The electronic device may further include an input module 340, which may be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function controls.

[0166] The electronic device may further include a communication module 350. In some embodiments, the communication module 350 may include a wireless module. The electronic device may perform short-range wireless transmission through the wireless module of the communication module 350, thereby providing the user with wireless broadband Internet access. For example, the communication module 350 may be used to help the user send and receive emails, browse web pages, and access streaming media.

[0167] Although not shown, the electronic device may further include a display unit, etc., which will not be described in detail herein. Specifically in this embodiment, the processor 310 in the electronic device will load the executable files corresponding to the processes of one or more application programs into the memory 320 according to the following instructions, and the processor 310 will run the application programs stored in the memory 320, thereby implementing the steps in the methods of the embodiments of the present invention.

[0168] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0169] As can be seen from the above, the electronic device provided by the embodiment of the present invention can obtain the problem to be processed and the flowchart to be processed; identify the element information in the flowchart to be processed according to the specified dimension to generate the content description of the flowchart to be processed; then calculate the matching degree between the content description and the flowchart to be processed, and measure the consistency between the content description and the flowchart to be processed by the matching degree and the specified comparison result of the preset threshold. Then, the target description of the flowchart to be processed is generated by using the content description and the specified comparison result, which can accurately and comprehensively describe the content in the flowchart to be processed. Finally, the target description is used to analyze and infer the target answer to the problem to be processed, which can effectively improve the efficiency and accuracy of flowchart-based question and answer.

[0170] A person of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0171] To this end, an embodiment of the present invention provides a computer-readable storage medium, in which multiple instructions are stored, and the instructions can be loaded by a processor to execute the steps of any one of the flowchart-based question and answer processing methods provided by the embodiment of the present invention.

[0172] The storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0173] According to one aspect of the present invention, a computer program product or computer program is provided, the computer program product or computer program including a computer program / instruction, the computer program / instruction being stored in a computer-readable storage medium. A processor of an electronic device reads the computer program / instruction from the computer-readable storage medium, and the processor executes the computer program / instruction, so that the electronic device executes the method provided in various optional implementations of the flowchart parsing aspect or the question-and-answer processing aspect provided in the above-mentioned embodiments.

[0174] Since the instructions stored in the storage medium can execute the steps in any one of the flowchart-based question and answer processing methods provided in the embodiments of the present invention, the beneficial effects that can be achieved by any one of the flowchart-based question and answer processing methods provided in the embodiments of the present invention can be achieved. Please see the previous embodiments for details and will not be repeated here.

[0175] The above is a detailed introduction to a flowchart-based question-and-answer processing method and device provided in an embodiment of the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for technical personnel in this field, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A question-answering processing method based on a flowchart, characterized in that: The method comprises: Obtain pending issues and pending process diagrams; Identifying element information in the flowchart to be processed according to a specified dimension to generate a content description of the flowchart to be processed, wherein the content description includes a sub-description corresponding to the specified dimension; Calculating the matching degree between the content description and the flowchart to be processed; Generate a target description of the flowchart to be processed based on the content description and a specified comparison result, wherein the specified comparison result is a comparison result between the matching degree and a preset threshold value; The target description is used to analyze and reason about the problem to be processed, and a target answer corresponding to the problem to be processed is generated.

2. The method according to claim 1, characterized in that The identifying the element information in the flowchart to be processed according to the specified dimension to generate a content description of the flowchart to be processed includes: Optimizing the image quality of the flowchart to be processed to obtain an optimized flowchart; Based on the optimized flow chart and the recognition template, generate recognition prompt words, wherein the recognition prompt words include recognition requirements and output requirements corresponding to each specified dimension; Using the recognition prompt words, guiding the recognition model to recognize the element information corresponding to each specified dimension from the optimized flow chart according to the recognition requirements; For each specified dimension, the element information is organized according to the output requirement to obtain a sub-description corresponding to the specified dimension; The sub-descriptions corresponding to all designated dimensions are used as the content description of the flowchart to be processed.

3. The method according to claim 2, characterized in that The step of optimizing the image quality of the flowchart to be processed to obtain an optimized flowchart includes: Performing grayscale processing on the flowchart to be processed to obtain a grayscale flowchart; Performing filtering processing on the grayscale flow chart to obtain a filtering flow chart; Performing enhancement processing on the edges of the key structures in the filtering flowchart to obtain an edge enhancement flowchart; Performing contrast enhancement on the filtering flow chart to obtain a contrast enhancement flow chart; The edge enhancement flowchart and the contrast enhancement flowchart are superimposed according to the specified weights to obtain an optimized flowchart.

4. The method according to claim 1, characterized in that: The calculating the matching degree between the content description and the flowchart to be processed includes: Acquire text features of the content description in a multimodal space, and image features of the flowchart to be processed in the multimodal space; Multiplying the text feature by the transpose of the text feature to obtain a text kernel matrix; multiplying the image feature by the transpose of the image feature to obtain an image kernel matrix; The matching degree between the content description and the flowchart to be processed is calculated using the text kernel matrix and the image kernel matrix.

5. The method according to claim 1, characterized in that The preset threshold includes a first threshold and a second threshold, the second threshold is less than the first threshold, and the generating the target description of the flowchart to be processed based on the content description and the specified comparison result includes: If the matching degree is not less than a first threshold, taking the content description as a target description of the flowchart to be processed; If the matching degree is less than the first threshold value and not less than the second threshold value, based on the matching degree between the sub-description corresponding to each specified dimension and the sub-description of the flowchart to be processed, correcting the content description to a target description; If the matching degree is less than the second threshold, the process returns to execute the step of identifying the element information in the flowchart to be processed according to the specified dimension to generate the content description of the flowchart to be processed and the subsequent steps.

6. The method according to claim 5, characterized in that The correcting the content description to a target description based on the sub-description corresponding to each specified dimension and the sub-matching degree of the flowchart to be processed includes: For each specified dimension, encode the sub-description corresponding to the specified dimension as a sub-text feature; Calculating the sub-matching degree between each of the sub-text features and the image features of the flowchart to be processed; Determining the sub-description corresponding to the sub-text feature with the lowest sub-matching degree as the description to be corrected; Generate a corrected description corresponding to the description to be corrected by using the prompt word corresponding to the dimension to be corrected and the flowchart to be processed, wherein the dimension to be corrected is a specified dimension corresponding to the description to be corrected; The description to be corrected in the content description is replaced with the corrected description to obtain a target description.

7. The method according to claim 1, characterized in that The using the target description to analyze and reason the problem to be processed to generate a target answer corresponding to the problem to be processed includes: Performing text classification processing on the problem to be processed to determine the problem type of the problem to be processed, wherein the problem type includes at least one of a detail problem and an overall problem; If the question type includes an overall question, transcribe the target description, and call the question-answering model to generate a target answer corresponding to the question to be processed based on the transcribed target description; If the question type includes a detail question, the control question-answering model analyzes and infers the question to be processed according to a preset thinking chain to generate a target answer corresponding to the question to be processed.

8. The method according to claim 7, characterized in that The performing text classification processing on the problem to be processed to determine the problem type of the problem to be processed includes: Acquire a type mapping relationship, where the type mapping relationship includes an association relationship between a preset type and a preset keyword; If the problem to be processed contains any preset keyword, the preset type corresponding to the preset keyword is used as the problem type of the problem to be processed; If the problem to be processed does not contain any preset keyword, the problem to be processed is combined with a classification template to obtain a classification prompt word, where the classification template is a prompt word template for classifying the problem to be processed; The classification prompt words are input into the classification model so that the classification model can perform semantic understanding and classification on the problem to be processed to obtain the problem type of the problem to be processed.

9. The method according to claim 7, characterized in that: The control question-answering model analyzes and infers the problem to be processed according to a preset thinking chain to generate a target answer corresponding to the problem to be processed, including: Performing intent recognition on the problem to be processed to determine the target dimension associated with the problem to be processed from the specified dimensions; Locating the target node associated with the problem to be processed; Generate a preliminary answer corresponding to the problem to be processed by combining the target dimension and the target node; Reflectively verify the preliminary responses to obtain the target responses.

10. A question-answer processing device based on a flowchart, the device being used to implement the method according to any one of claims 1 to 9, characterized in that: The device comprises: An acquisition module is used to obtain pending issues and pending flow charts; An identification module, used for identifying element information in the flowchart to be processed according to a specified dimension to generate a content description of the flowchart to be processed, wherein the content description includes a sub-description corresponding to the specified dimension; A calculation module, used for calculating the matching degree between the content description and the flowchart to be processed; A target generation module, used to generate a target description of the flowchart to be processed based on the content description and a specified comparison result, wherein the specified comparison result is a comparison result between the matching degree and a preset threshold value; The question-answering module is used to analyze and reason about the problem to be processed using the target description to generate a target answer corresponding to the problem to be processed.

Images