Flow chart review method and device, electronic equipment and storage medium

Abstract

The invention provides a flow chart review method and device, electronic equipment and a storage medium, and relates to the technical field of artificial intelligence, and the method comprises the steps: inputting a preset problem of a to-be-reviewed flow chart into a visual language model under the condition that the to-be-reviewed flow chart uploaded by a user is received, obtaining flow chart information of the to-be-reviewed flow chart output by the visual language model; the preset problem is used for guiding the visual language model to analyze the to-be-reviewed flow chart; determining a review question of the to-be-reviewed flow chart according to the flow chart information; inputting the review question into the visual language model to obtain a review result of the to-be-reviewed flow chart output by the visual language model; wherein the visual language model is obtained by performing joint training on the image data and the text data. According to the method and the device, the flow chart is analyzed and reviewed automatically and intelligently, so that the dependence on professionals is reduced, the labor cost is reduced, and meanwhile, the accuracy and the efficiency of flow chart review are improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a flowchart review method, device, electronic device and storage medium. Background Art

[0002] At present, in special planning and design materials, the high-precision analysis and review requirements for flowchart contents are all done manually, resulting in low efficiency in flowchart review. Summary of the invention

[0003] The present invention provides a flowchart review method, device, electronic device and storage medium, which are used to solve the defect of low efficiency of flowchart review in the prior art and improve the accuracy and efficiency of flowchart review.

[0004] The present invention provides a flow chart review method, comprising the following steps: When receiving a flowchart to be reviewed uploaded by a user, inputting preset questions of the flowchart to be reviewed into the visual language model, and obtaining flowchart information of the flowchart to be reviewed output by the visual language model; the preset questions are used to guide the visual language model to parse the flowchart to be reviewed; Determining review questions for the flowchart to be reviewed based on the flowchart information; Inputting the review question into the visual language model to obtain the review result of the flowchart to be reviewed output by the visual language model; The visual language model is obtained by jointly training image data and text data.

[0005] According to a flowchart review method provided by the present invention, the flowchart information includes at least one of text information, color information, symbol information, arrow information, image structure information, image relationship information and visual elements; the color information includes text color information, symbol color information and line color information.

[0006] According to a flowchart review method provided by the present invention, the image structure information includes the layout mode and hierarchical structure of the process nodes; the image structure information is determined based on the following method: Identifying a process node in the flowchart to be reviewed; Determining the spatial distribution of the process nodes according to the position information of the process nodes; Determining the layout mode according to the spatial distribution of the process nodes; The hierarchical structure is determined according to the nesting relationship and grouping relationship between the process nodes.

[0007] According to a flowchart review method provided by the present invention, the image relationship information includes dependency relationships and sequence relationships between process nodes; the image relationship information is determined based on the following method: Identifying arrows in the flowchart to be reviewed; The dependency relationship and the sequence relationship between the process nodes in the flowchart to be reviewed are determined according to the direction and type of the arrow.

[0008] According to a flowchart review method provided by the present invention, determining review questions for the flowchart to be reviewed based on the flowchart information includes: Slicing the flowchart information to obtain text entries of the flowchart to be reviewed; The name of the flowchart to be reviewed is concatenated with the text entry to obtain the review question of the flowchart to be reviewed.

[0009] According to a flowchart review method provided by the present invention, the step of inputting the review question into the visual language model to obtain the review result of the flowchart to be reviewed output by the visual language model includes: In the case where the review question is a content review question, inputting the content review request into the visual language model to obtain a content review result of the flowchart to be reviewed output by the visual language model; In the case where the review question is a structural review question, the structural review request is input into the visual language model to obtain a structural review result of the flowchart to be reviewed output by the visual language model; The review result includes the content review result and the structure review result.

[0010] According to a flowchart review method provided by the present invention, after inputting the review question into the visual language model and obtaining the review result of the flowchart to be reviewed output by the visual language model, the method further includes: The content review result and the structure review result are integrated to generate a comprehensive review result of the flowchart to be reviewed.

[0011] The present invention also provides a flow chart review device, comprising the following modules: The parsing module is used to input preset questions of the flowchart to be reviewed into the visual language model when receiving the flowchart to be reviewed uploaded by the user, and obtain the flowchart information of the flowchart to be reviewed output by the visual language model; the preset questions are used to guide the visual language model to parse the flowchart to be reviewed; An evaluation question module, used to determine the evaluation questions of the flowchart to be evaluated according to the flowchart information; The review module is used to input the review questions into the visual language model to obtain the review results of the flowchart to be reviewed output by the visual language model; wherein the visual language model is obtained by jointly training image data and text data.

[0012] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the review method of any of the above-mentioned flowcharts is implemented.

[0013] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the review method of any of the flowcharts described above is implemented.

[0014] The flowchart review method, device, electronic device and storage medium provided by the present invention input preset questions of the flowchart to be reviewed into a visual language model when receiving a flowchart to be reviewed uploaded by a user, and obtain flowchart information of the flowchart to be reviewed output by the visual language model; the preset questions are used to guide the visual language model to parse the flowchart to be reviewed; based on the flowchart information, the review questions of the flowchart to be reviewed are determined; the review questions are input into the visual language model, and the review results of the flowchart to be reviewed output by the visual language model are obtained; wherein the visual language model is obtained by joint training of image data and text data. The present invention reduces the dependence on professionals by automatically and intelligently parsing and reviewing flowcharts, thereby reducing labor costs, and at the same time, improving the accuracy and efficiency of flowchart review. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0016] Figure 1 It is a schematic diagram of the flow chart of the review method provided by the present invention.

[0017] Figure 2 It is a flow chart of the method of large-model intelligent identification and review flow chart provided by the present invention.

[0018] Figure 3 It is a structural schematic diagram of the review device of the flow chart provided by the present invention.

[0019] Figure 4It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] In related technologies, image recognition and processing mainly focus on basic element detection, identification and classification, such as identifying graphics, colors and text. However, these methods lack a deep understanding of the complex logic and structural relationships in images. Especially in images such as flowcharts that contain a lot of information and logical relationships, related technologies cannot accurately interpret and process the dynamic logic and complex relationships in the diagram, resulting in insufficient information interpretation or misunderstanding.

[0022] Since the information contained in the flowchart usually involves multiple dimensions such as the order of operation steps, conditional judgment, data flow, etc., if these features cannot be accurately identified and understood, the practical value of the flowchart in project management and decision support will be greatly limited. In addition, manual processing of flowcharts is time-consuming and labor-intensive, and is prone to errors due to differences in personal experience and subjective understanding.

[0023] Since the review of the flowchart is achieved based on the recognition result of the flowchart, the above flowchart recognition method will affect the review efficiency and accuracy of the flowchart.

[0024] Based on the above problems, the present invention proposes a flowchart review method, specifically a flowchart review method based on a large model intelligence.

[0025] Combine the following Figure 1-Figure 4 The invention describes a flowchart review method, device, electronic device and storage medium.

[0026] Figure 1 It is a schematic diagram of the flow chart of the review method provided by the present invention, such as Figure 1 As shown, the method includes the following: Step 101: upon receiving a flowchart to be reviewed uploaded by a user, inputting preset questions of the flowchart to be reviewed into the visual language model, and obtaining flowchart information of the flowchart to be reviewed output by the visual language model.

[0027] In an embodiment of the present invention, the review system is provided with an interface for users to upload flowchart images. Through the interface, the review system receives the flowchart to be reviewed uploaded by the user in the form of images, wherein the flowchart to be reviewed is a flowchart in a special plan or design scheme.

[0028] A pre-trained visual language model is deployed in the review system, wherein the visual language model may be a VLM (Visual Language Model) model. The VLM model is a model that combines visual and language understanding capabilities. It can not only understand and generate natural language text, but also process and analyze image content. The VLM model is based on deep learning technology. Through joint training of image and text data, the model can simultaneously process visual and language information, thereby achieving more complex understanding and generation tasks. In an embodiment of the present invention, the VLM model has the following functions: 1) Image description generation: Generate a natural language description of the image content based on the input image; 2) Visual Question Answering: answering natural language questions about the content of an image; 3) Image classification and annotation: semantic understanding and label generation of images; 4) Flowchart parsing: Parse and understand the nodes, connections, and structures in the flowchart, and generate corresponding natural language explanations or reviews.

[0029] In one embodiment, the visual language model is trained based on the following method: (1) Data preparation. Image data: Use large-scale labeled image datasets, such as ImageNet and COCO, which contain images of various categories and scenes. Text data: This can be image annotations with descriptions or other forms of text data, which are used to guide the model to learn the relationship between image content and semantic expression.

[0030] (2) Model architecture selection. Based on the deep neural network architecture, a model suitable for processing sequence data such as Transformer is used as the basic architecture. The model usually includes a visual processor for processing images (such as a convolutional neural network) and a language processor for processing text sequences (such as a Transformer encoder).

[0031] (3) Joint training process. Image encoder: First, the input image is converted into a feature vector or feature map through a convolutional neural network. These features capture different levels of abstract information in the image, such as edges, textures, objects, and scenes. Text encoder: For text input, a language processor (such as a Transformer encoder) is used to encode natural language text into semantic vectors or hidden representations. These vectors capture the semantic and contextual information of the text. Cross-modal attention mechanism: At the top layer of the model, a cross-modal attention mechanism is introduced, which enables the model to pay attention to the information of the image and text at the same time and establish meaningful associations between them.

[0032] (4) Objective function. The goal during training is to minimize the language model loss in the image description generation task and the visual loss in the image classification / labeling task. For the image description generation task, the model is optimized by maximizing the similarity between the generated text and the true description. For the image classification or labeling task, the model is optimized by minimizing the difference between the predicted label and the true label.

[0033] (5) Iterative optimization. The training process is an iterative optimization process. The model parameters are adjusted through optimization methods such as back propagation algorithm and gradient descent, so that the model gradually improves its performance on the training set and validation set.

[0034] (6) Evaluation and tuning. After training is completed, the model needs to be evaluated and tuned. The evaluation includes quality evaluation of generated text (such as BLEU score, etc.) and accuracy evaluation of image understanding tasks on the test set.

[0035] Through the above training process, the visual language model can learn the complex relationship between images and language, thereby demonstrating strong capabilities in a variety of visual and language tasks, such as image description generation, visual question answering, and multimodal content understanding.

[0036] Pre-construct and preset questions in the review system to guide the visual language model to parse the flowchart to be reviewed. For example, the visual language model is guided to focus on specific architectural content in the flowchart for detailed analysis. Among them, the preset questions can be set according to specific needs and the characteristics of the flowchart, such as: a. Confirm whether the steps in the flowchart are coherent and complete. b. Evaluate whether the functions of each node in the flowchart are clearly defined. c. Check whether the logical path in the flowchart is reasonable and effective. d. Analyze possible repeated or redundant steps in the flowchart. e. Identify possible errors or inconsistencies in the flowchart. Through these preset questions, the visual language model can analyze and interpret the flowchart in a more targeted manner, thereby generating more accurate and useful analysis results and review reports.

[0037] When receiving a flowchart to be reviewed uploaded by a user, the preset questions of the flowchart to be reviewed are input into the visual language model to obtain the flowchart information of the flowchart to be reviewed output by the visual language model. For example, questions are asked to the visual language model based on the preset questions, so that the visual language model automatically parses the uploaded flowchart to be reviewed based on the questions, and extracts key information from the diagram, including but not limited to text information, color information, symbol information, arrow information, image structure information, image relationship information, and visual elements. The parsing of each information is as follows: Text information: Identify text content in the flowchart, including labels and descriptions of process nodes; Color information: including text color information, symbol color information and line color information. Different colors are used to identify different process states, priorities or other important information. Symbol information: various symbols used in flowcharts, such as process boxes, decision boxes, start / end boxes, etc., to indicate different process steps, directions or decision points; Arrow information: Arrows are used to connect the nodes in the flowchart to indicate the sequence, direction or dependency of the process; Image structure information: describes the overall structure of the flowchart, including the arrangement and layout of each element in the diagram; Image relationship information: the relationship between different elements in the flowchart, such as dependency, sequential, parallel, etc. Other key visual elements: covers all other important visual elements in the flowchart besides text, color and symbols, such as charts, tables, etc.

[0038] Step 102: Determine review questions for the flowchart to be reviewed based on the flowchart information.

[0039] The flowchart information is sliced ​​to obtain text entries of the flowchart to be reviewed, and then the name of the flowchart to be reviewed is concatenated with the text entries to obtain review questions of the flowchart to be reviewed.

[0040] For example, the review system calls the content slicing plug-in to itemize the flowchart information parsed by the model, that is, to organize and display the various key information extracted from the model parsing flowchart in a certain structured way. The itemization process can be understood as dividing the important parts and elements in the flowchart into clear items or items, each of which contains specific information or attributes. Among them, itemization includes the following: Key element extraction: The system will identify and extract key elements in the flowchart, such as nodes, connecting lines, arrow indicators, text labels, etc.

[0041] Attribute description: For each identified element, the system will describe its attributes and characteristics, such as the name, type (such as transportation hub, road intersection), location coordinates of the node; the direction and meaning of the connection line (such as path, flow direction), etc.

[0042] Relationships and connections: The system will analyze and describe the relationships and connections between various elements, including the association between nodes, the functional description of connecting lines, the meaning of arrows (such as indicating direction, flow direction), etc.

[0043] Semantic understanding: The system attempts to understand and interpret the semantic meaning of these items, that is, the specific role and significance of these elements and relationships in the overall plan or design.

[0044] Structured output: This information will be output in a structured form to facilitate subsequent automated processing, analysis or presentation, which can include generating readable text descriptions, generating data tables, creating relationship diagrams, etc.

[0045] Through item-based processing, the system not only makes the information in the flowchart easier to understand and analyze, but also provides strong support for subsequent evaluation, comparison and decision-making.

[0046] The name of the flowchart to be reviewed is further spliced ​​with the text entry to obtain the review questions of the flowchart to be reviewed. For example, a customized prompt for flowchart review is generated through a flowchart analysis prompt splicing plug-in to effectively splice the name of the document to be reviewed with the functional entries parsed from the flowchart.

[0047] Step 103: input the review question into the visual language model to obtain the review result of the flowchart to be reviewed output by the visual language model.

[0048] In the embodiment of the present invention, the review result includes a review score value and a review description, and the review result includes a content review result and a structure review result.

[0049] In the case where the review question is a content review question, the content review request is input into the visual language model to obtain the content review result of the flowchart to be reviewed output by the visual language model. The content review of the flowchart to be reviewed may include: Existence and completeness of function points: Evaluate whether the function points described in the flowchart are clear and unambiguous, and confirm whether all necessary steps and activities are covered. Function points refer to the specific operations, steps or stages described in the flowchart. Each function point describes a specific task or activity in the process, which can be a node or step in the flowchart.

[0050] Accuracy of information: Check the text descriptions, symbols, colors and other elements in the flowchart to ensure that the information is accurate.

[0051] Logical coherence: Analyze the logical relationship between the steps in the flowchart to confirm whether the sequence of steps in the process is reasonable and conflict-free.

[0052] When the review question is a structural review question, the structural review request is input into the visual language model to obtain the structural review result of the flowchart to be reviewed output by the visual language model. The structural review of the flowchart to be reviewed may include: Normativity: Evaluate whether the layout and structure of the flowchart conform to general flowchart design specifications or standards, including the use of symbols, arrow directions, legend descriptions, etc.

[0053] Clarity and readability: Check the visual presentation of the flowchart to ensure that the arrangement and design of the elements in the diagram make the information easy to understand and convey.

[0054] Through comprehensive review of both content and structure, the quality and effectiveness of flowcharts can be comprehensively evaluated, helping to ensure the accuracy and efficiency of project management and design solution review.

[0055] In one embodiment, the content review results and the structural review results are integrated to generate a comprehensive review result of the flowchart to be reviewed, wherein the comprehensive review result includes comprehensive suggestions and improvement suggestions. For example, in view of the clarity of the functional point description and the rationality of the logical relationship, it is recommended to further optimize the sequence of process steps, especially in the resource allocation and execution stages. It is proposed to improve the overall layout and readability of the flowchart by rearranging the positions of some nodes so as to better display the logical relationship of the process. The embodiment of the present invention can provide clear feedback and improvement suggestions to the designer through the comprehensive review results to ensure that the flowchart meets the optimal standards in both content and structure, thereby effectively supporting and guiding the actual business process execution.

[0056] The flowchart review method provided by the embodiment of the present invention inputs the preset questions of the flowchart to be reviewed into the visual language model when receiving the flowchart to be reviewed uploaded by the user, and obtains the flowchart information of the flowchart to be reviewed output by the visual language model; the preset questions are used to guide the visual language model to parse the flowchart to be reviewed; according to the flowchart information, the review questions of the flowchart to be reviewed are determined; the review questions are input into the visual language model, and the review results of the flowchart to be reviewed output by the visual language model are obtained; wherein the visual language model is obtained by joint training of image data and text data. The present invention reduces the dependence on professionals by automatically and intelligently parsing and reviewing flowcharts, thereby reducing labor costs, and at the same time, improves the accuracy and efficiency of flowchart review.

[0057] In one embodiment, the image structure information includes the layout and hierarchy of the process nodes; the image structure information is determined based on the following methods: Identify the process nodes in the flowchart to be reviewed; determine the spatial distribution of the process nodes based on their location information; determine the layout method based on the spatial distribution of the process nodes; determine the hierarchical structure based on the nested and grouped relationships between the process nodes.

[0058] For example, image recognition technology (such as computer vision models) is used to detect and identify process nodes in flowcharts. These nodes can be graphic objects of different shapes and sizes, representing specific process steps or decision points. Then, the position of each process node such as process box, judgment box, start / end box, etc. is determined. According to the position and arrangement of process nodes, the relative position relationship between nodes is analyzed to understand the overall layout of the flowchart, such as vertical arrangement, horizontal arrangement or mixed arrangement. By detecting the nesting and grouping relationship of nodes, the hierarchical structure of the flowchart can be inferred, including the identification of parallel processes, sub-processes and conditional branches, which helps to understand the complexity and organizational structure of the process.

[0059] The embodiment of the present invention can improve the review accuracy of the flowchart by analyzing the image structure information in the flowchart to be reviewed.

[0060] In one embodiment, the image relationship information includes dependency and sequence relationships between process nodes; the image relationship information is determined based on the following method: Identify the arrows in the flowchart to be reviewed; based on the direction and type of the arrows, determine the dependency and sequence relationships between the process nodes in the flowchart to be reviewed.

[0061] For example, identify the arrows in the flowchart to be reviewed, and then analyze the direction and position of the arrows to determine the execution order and control flow of the process. Arrows in different directions can represent different logical relationships such as sequential execution, conditional branches or loop structures. At the same time, identify different types of arrows, such as ordinary arrows, conditional arrows, loop arrows, etc. Different types of arrows represent different logical relationships, such as conditional judgments, loops, etc. The system needs to be able to identify these different types of arrows and perform corresponding logical reasoning. Optionally, the text labels and descriptions on the process nodes are also extracted to understand the specific meaning and function of each process node. These text information helps the system to infer the logic of the process more accurately. Based on the above information, the dependency and sequence relationships between the process nodes in the flowchart to be reviewed can be determined.

[0062] The embodiment of the present invention can improve the review accuracy of the flowchart by parsing the image relationship information in the flowchart to be reviewed.

[0063] In order to further analyze and explain the review method of the flowchart provided by the present invention, refer to the following examples.

[0064] The embodiment of the present invention proposes a method for intelligently identifying and reviewing flowcharts based on a large model, aiming to provide an efficient and accurate technical solution for the review requirements of flowcharts in special plans, design plans (including project lists), etc. The method includes four main stages: preliminary recognition, comprehensive content analysis, logical structure analysis, and review scoring. By integrating technical means such as image content analysis, image structure analysis, and image relationship analysis, deep learning and processing are carried out based on the powerful computing and analysis capabilities of the large model.

[0065] The first stage, preliminary recognition, focuses on reading text and color in the image, so as to preliminarily judge the basic content information of the flowchart. Through advanced image recognition technology, the system can accurately identify and distinguish different elements and colors, thus laying the foundation for subsequent in-depth analysis.

[0066] The second stage, content comprehensiveness analysis, is mainly to determine whether the flowchart contains the key information or necessary information required for the review, and to evaluate whether the content information provided by the flowchart is comprehensive. This stage uses highly advanced semantic understanding and information extraction technology to conduct an in-depth analysis of the content in the flowchart to ensure that the necessary information points in the review process are not omitted.

[0067] The third stage is logical structure analysis. Based on information such as arrow indications and diagram colors, the logical structure of the flowchart can be analyzed. This stage accurately reveals the logical relationship and structural layout of the process through intelligent interpretation of visual elements such as arrow directions and color distinctions in the flowchart.

[0068] In the fourth stage, based on the analysis results of the flowchart, the content and structure of the flowchart are reviewed and scored.

[0069] The above method can not only effectively perform intelligent recognition and analysis of flowchart content, but also comprehensively score the flowchart according to its content and associated logic, and output corresponding modification opinions and review conclusions. By integrating with large model technology, this method improves the efficiency and accuracy of flowchart review in special planning and design solutions.

[0070] The method for intelligently identifying and reviewing flowcharts based on a large model provided in an embodiment of the present invention is implemented by an automatic flowchart recognition and review system, which includes: (1) An interface for receiving flowchart images uploaded by users; (2) An image parsing module integrated with the multimodal big model, which is used to automatically parse text, color, and other key visual elements in the flowchart. The multimodal big model integrates image content parsing, image structure parsing, image relationship parsing, and natural language processing technologies to achieve high-precision and comprehensive parsing of flowcharts. (3) A question presetting and parsing module, which is configured with preset questions (prompts) for guiding the large model to deeply parse the content of the flowchart architecture. The module is also configured with an interface for constructing questions, allowing users or system administrators to predefine the parsing and review logic for specific flowcharts. (4) Content entry and text generation module, used to convert the parsed flowchart information into entry text; (5) Flowchart review module, which is used to automatically review the existence and completeness of function points and the standardization of flowchart structure based on analysis results and preset logic; (6) Comprehensive scoring and report generation module, used to summarize the review results and generate a review report.

[0071] refer to Figure 2 ,The method based on large model intelligent identification and review flow chart implemented by the above system mainly includes the following steps: Step 1, receiving the flowchart image uploaded by the user; Step 2, perform VLM analysis on the flow chart; Step 3: Generate parsed content; Step 4: Use prompt to ask whether the content of the large model flowchart meets the preset specifications; Step 5: Call the content slicing plug-in to generate text entries from the flowchart content; Step 6: Call the flowchart analysis prompt splicing plug-in to splice the prompt words; Step 7: VLM asks questions and analyzes the flowchart design module to determine whether the reported materials are described; if so, call the review summary plug-in to integrate the results of the two directions and output the flowchart review; if not, proceed to step 8; Step 8: Provide a flow chart evaluation analysis; Step 9: VLM questions to assess the specification structure; Step 10. Call the review summary plug-in to integrate the results of the two directions and output the flowchart review.

[0072] The method provided by the embodiment of the present invention has the following advantages: 1. Improve efficiency and speed. Automated processing flow: Automating the process of manual processing and analysis of flowcharts significantly reduces the time and labor of manual input, especially when processing a large number of documents. Fast feedback and decision support: Through instant intelligent review and feedback mechanisms, relevant decision makers can quickly obtain the required information, thereby accelerating the decision-making process and increasing the speed of project advancement.

[0073] 2. Improve accuracy and reliability. Deep content analysis: By using advanced artificial intelligence technology, each element in the flowchart is accurately identified and parsed to reduce human errors. Comprehensive logic verification: The system ensures the rationality and coherence between steps by analyzing the logical structure of the flowchart, which helps to discover and correct potential errors and discrepancies.

[0074] 3. Enhance operability and practicality. Detailed review report and improvement suggestions: The review report not only lists the scoring results in detail, but also provides improvement suggestions, providing practical guidance for project improvement. User-friendly interface design: The entire system, from uploading to report generation, is operated through a user-friendly interface, which is easy to use and easy to understand.

[0075] 4. Reduce costs. Reduce resource investment: Automated and intelligent processing reduces dependence on professionals and reduces labor costs accordingly. Avoid duplication of work: Highly accurate automated processes reduce repeated correction work caused by errors and omissions, saving additional time and resources.

[0076] 5. Enhanced decision support. Data-driven decision-making: The data analysis and conclusions provided by the technical solution support decision-making based on data and facts, increasing the transparency and scientific nature of decision-making. Strategy and planning optimization: Through in-depth analysis and system feedback, future project planning strategies can be optimized to avoid potential design and planning defects.

[0077] By implementing this intelligent flowchart identification and review technology solution, while ensuring high-quality output, it can significantly improve work efficiency, reduce operating costs, and enhance the scientificity and practicality of decision-making and planning. This has important strategic significance for managing complex design and planning projects and can bring sustainable and long-term competitive advantages to enterprises or institutions.

[0078] The flowchart review device provided by the present invention is described below. The flowchart review device described below and the flowchart review method described above can be referenced to each other.

[0079] refer to Figure 3 The flowchart review device provided by the present invention includes a parsing module 301, a review questioning module 302 and a review module 303.

[0080] The parsing module 301 is used to input preset questions of the flowchart to be reviewed into the visual language model when receiving the flowchart to be reviewed uploaded by the user, and obtain the flowchart information of the flowchart to be reviewed output by the visual language model; the preset questions are used to guide the visual language model to parse the flowchart to be reviewed; The review question module 302 is used to determine the review questions of the flowchart to be reviewed according to the flowchart information; The review module 303 is used to input the review questions into the visual language model to obtain the review results of the flowchart to be reviewed output by the visual language model; wherein the visual language model is obtained by jointly training image data and text data.

[0081] The flowchart review device provided by the embodiment of the present invention inputs the preset questions of the flowchart to be reviewed into the visual language model when receiving the flowchart to be reviewed uploaded by the user, and obtains the flowchart information of the flowchart to be reviewed output by the visual language model; the preset questions are used to guide the visual language model to parse the flowchart to be reviewed; according to the flowchart information, the review questions of the flowchart to be reviewed are determined; the review questions are input into the visual language model, and the review results of the flowchart to be reviewed output by the visual language model are obtained; wherein the visual language model is obtained by joint training of image data and text data. The present invention reduces the dependence on professionals by automatically and intelligently parsing and reviewing flowcharts, thereby reducing labor costs, and at the same time, improving the accuracy and efficiency of flowchart review.

[0082] In one embodiment, the flowchart information includes at least one of text information, color information, symbol information, arrow information, image structure information, image relationship information and visual elements; the color information includes text color information, symbol color information and line color information.

[0083] In one embodiment, the image structure information includes the layout mode and hierarchical structure of the process nodes; the image structure information is determined based on the following methods: identifying the process nodes in the flowchart to be reviewed; determining the spatial distribution of the process nodes based on the position information of the process nodes; determining the layout mode based on the spatial distribution of the process nodes; determining the hierarchical structure based on the nested relationship and grouping relationship between the process nodes.

[0084] In one embodiment, the image relationship information includes dependency relationships and sequential relationships between process nodes; the image relationship information is determined based on the following method: identifying arrows in the flowchart to be reviewed; and determining the dependency relationships and sequential relationships between process nodes in the flowchart to be reviewed based on the direction and type of the arrows.

[0085] In one embodiment, the review question module 302 is specifically used to: slice the flowchart information to obtain text entries of the flowchart to be reviewed; and concatenate the name of the flowchart to be reviewed with the text entries to obtain review questions of the flowchart to be reviewed.

[0086] In one embodiment, the review module 303 is specifically used for: when the review question is a content review question, inputting the content review request into the visual language model to obtain the content review result of the flowchart to be reviewed output by the visual language model; when the review question is a structure review question, inputting the structure review request into the visual language model to obtain the structure review result of the flowchart to be reviewed output by the visual language model; the review result includes the content review result and the structure review result.

[0087] In one embodiment, the review module 303 is further used to: integrate the content review result and the structure review result to generate a comprehensive review result of the flowchart to be reviewed.

[0088] Figure 4 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 4 As shown, the electronic device may include: a processor 410, a communication interface 420, a memory 430 and a communication bus 440, wherein the processor 410, the communication interface 420 and the memory 430 communicate with each other through the communication bus 440. The processor 410 may call the logic instructions in the memory 430 to execute the flowchart review method, which includes: when receiving a flowchart to be reviewed uploaded by a user, inputting the preset questions of the flowchart to be reviewed into the visual language model, and obtaining the flowchart information of the flowchart to be reviewed output by the visual language model; the preset questions are used to guide the visual language model to parse the flowchart to be reviewed; according to the flowchart information, determining the review questions of the flowchart to be reviewed; inputting the review questions into the visual language model, and obtaining the review results of the flowchart to be reviewed output by the visual language model; wherein the visual language model is obtained by joint training of image data and text data.

[0089] In addition, the logic instructions in the above-mentioned memory 430 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0090] On the other hand, the present invention also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the flowchart review method provided by the above-mentioned methods, the method including: when receiving a flowchart to be reviewed uploaded by a user, inputting preset questions of the flowchart to be reviewed into a visual language model, and obtaining flowchart information of the flowchart to be reviewed output by the visual language model; the preset questions are used to guide the visual language model to parse the flowchart to be reviewed; based on the flowchart information, determining the review questions of the flowchart to be reviewed; inputting the review questions into the visual language model, and obtaining the review results of the flowchart to be reviewed output by the visual language model; wherein the visual language model is obtained by joint training of image data and text data.

[0091] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a method for reviewing flowcharts provided by the above methods, the method comprising: upon receiving a flowchart to be reviewed uploaded by a user, inputting preset questions of the flowchart to be reviewed into a visual language model, and obtaining flowchart information of the flowchart to be reviewed output by the visual language model; the preset questions are used to guide the visual language model to parse the flowchart to be reviewed; based on the flowchart information, determining review questions for the flowchart to be reviewed; inputting the review questions into the visual language model, and obtaining review results of the flowchart to be reviewed output by the visual language model; wherein the visual language model is obtained by joint training of image data and text data.

[0092] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0093] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0094] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A flowchart review method, characterized in that: include: When receiving a flowchart to be reviewed uploaded by a user, inputting preset questions of the flowchart to be reviewed into a visual language model, and obtaining flowchart information of the flowchart to be reviewed output by the visual language model; The preset questions are used to guide the visual language model to parse the flowchart to be reviewed; Determining review questions for the flowchart to be reviewed based on the flowchart information; Inputting the review question into the visual language model to obtain the review result of the flowchart to be reviewed output by the visual language model; The visual language model is obtained by jointly training image data and text data.

2. The method for reviewing a flowchart according to claim 1, characterized in that: The flowchart information includes at least one of text information, color information, symbol information, arrow information, image structure information, image relationship information and visual elements; the color information includes text color information, symbol color information and line color information.

3. The method for reviewing a flowchart according to claim 2, characterized in that: The image structure information includes the layout mode and hierarchy of the process nodes; the image structure information is determined based on the following methods: Identifying a process node in the flowchart to be reviewed; Determining the spatial distribution of the process nodes according to the position information of the process nodes; Determining the layout mode according to the spatial distribution of the process nodes; The hierarchical structure is determined according to the nesting relationship and grouping relationship between the process nodes.

4. The method for reviewing a flowchart according to claim 2, characterized in that: The image relationship information includes dependency and sequence relationships between process nodes; the image relationship information is determined based on the following method: Identifying arrows in the flowchart to be reviewed; The dependency relationship and the sequence relationship between the process nodes in the flowchart to be reviewed are determined according to the direction and type of the arrow.

5. The method for reviewing a flowchart according to claim 1, characterized in that: Determining review questions for the flowchart to be reviewed based on the flowchart information includes: Slicing the flowchart information to obtain text entries of the flowchart to be reviewed; The name of the flowchart to be reviewed is concatenated with the text entry to obtain the review question of the flowchart to be reviewed.

6. The method for reviewing a flowchart according to claim 1, characterized in that: The step of inputting the review question into the visual language model to obtain the review result of the flowchart to be reviewed output by the visual language model includes: In the case where the review question is a content review question, inputting the content review request into the visual language model to obtain a content review result of the flowchart to be reviewed output by the visual language model; In the case where the review question is a structural review question, the structural review request is input into the visual language model to obtain a structural review result of the flowchart to be reviewed output by the visual language model; The review result includes the content review result and the structure review result.

7. The method for reviewing a flowchart according to claim 6, characterized in that: After inputting the review question into the visual language model and obtaining the review result of the flowchart to be reviewed output by the visual language model, the method further includes: The content review result and the structure review result are integrated to generate a comprehensive review result of the flowchart to be reviewed.

8. A flow chart review device, characterized in that: include: A parsing module, for, upon receiving a flowchart to be reviewed uploaded by a user, inputting preset questions of the flowchart to be reviewed into a visual language model, and obtaining flowchart information of the flowchart to be reviewed output by the visual language model; The preset questions are used to guide the visual language model to parse the flowchart to be reviewed; An evaluation question module, used to determine the evaluation questions of the flowchart to be evaluated according to the flowchart information; The review module is used to input the review questions into the visual language model to obtain the review results of the flowchart to be reviewed output by the visual language model; wherein the visual language model is obtained by jointly training image data and text data.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the review method of the flowchart according to any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the review method of the flowchart according to any one of claims 1 to 7 is implemented.

Images