Flow chart processing method and device, equipment, storage medium and program product

Abstract

The invention discloses a flow chart processing method and device, equipment, a storage medium and a program product, and the method comprises the steps: carrying out the image coding of a flow image, and obtaining an image coding feature; performing text decoding on the image coding features to obtain a process node sequence set, a plurality of node identifiers and a connection relationship among the node identifiers in the target process image; determining the flow node corresponding to each node identifier according to the position information of each flow node in the flow node set and the position information contained in the node identifier; and generating a text corresponding to the flow image according to the flow node, the node identifier corresponding to the flow node and the connection relationship between the node identifiers. According to the method and the device, the accuracy of the generated description text of the process image can be improved, the training effect of the model is improved, and the subsequent process identification result of the large-size process image containing the complex process information is improved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a flowchart processing method, device, equipment, storage medium and program product. Background Art

[0002] With the development of artificial intelligence, there is a need to answer questions about document information. Since there are flowcharts in documents, when users need to ask questions about the information in the flowcharts, they often need to first identify the flowcharts as specific structured text data, and then use language models to implement the flowchart question-answering function. Summary of the invention

[0003] In response to the above technical problems, the embodiments of the present application provide a flowchart processing method, device, equipment, storage medium and program product to solve the technical problem of low recognition effect of flowchart information existing in the related art.

[0004] In a first aspect, an embodiment of the present application provides a flowchart processing method, including:

[0005] Perform image coding on the process image to obtain image coding features;

[0006] Performing text decoding on the image coding features to obtain a process node set, a plurality of node identifiers, and a connection relationship between the node identifiers in the target process image;

[0007] Determine the process node corresponding to each of the node identifiers according to the position information of each of the process nodes in the process node set and the position information contained in the node identifier;

[0008] The text corresponding to the process image is generated according to each of the process nodes, the node identifier of each of the process nodes and the connection relationship between the node identifiers.

[0009] In a second aspect, an embodiment of the present application provides a flowchart processing device, including:

[0010] A coding module, used for performing image coding on the process image to obtain image coding features;

[0011] A decoding module, used to perform text decoding on the image coding feature to obtain a process node set, a plurality of node identifiers and a connection relationship between the node identifiers of the process image;

[0012] A determination module, used to determine the process node corresponding to each of the node identifiers according to the position information of each of the process nodes in the process node set and the position information contained in the node identifier;

[0013] A generating module is used to generate text corresponding to the process image according to each of the process nodes, the node identifier of each of the process nodes and the connection relationship between the node identifiers.

[0014] In a third aspect, an embodiment of the present application further provides a computer device, comprising a memory storing a plurality of instructions; a processor loads instructions from the memory to execute the steps of the flowchart processing method described above.

[0015] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, which stores a plurality of instructions, and the instructions are suitable for a processor to load to execute the steps of the flowchart processing method described above.

[0016] In a fifth aspect, an embodiment of the present application further provides a computer program product, wherein the computer program product comprises a computer program, and the computer program is executed by a processor to perform the steps of the flowchart processing method as described above.

[0017] By adopting the scheme of the application embodiment, after performing image encoding processing and text decoding processing on the process image to obtain the process node set, node identifiers and the connection relationship between the node identifiers, the position information of each process node in the process node set and the position information contained in the node identifier are used to determine the process node corresponding to each node identifier, and finally the final text is generated in combination with the process nodes, the node identifiers corresponding to the process nodes and the connection relationship between the node identifiers. Compared with directly obtaining a complex process description text by performing text decoding processing on the image encoding features of the target process image, by first identifying the relatively simple process node set, node identifiers and the connection relationship between the node identifiers in the flowchart to further generate text for the process description, the problem of deviation in the directly generated process description text due to image encoding and decoding errors can be effectively avoided, thereby improving the accuracy of the generated text. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0019] Figure 1 A schematic diagram of a flow chart image requiring process identification in the related art;

[0020] Figure 2 A schematic diagram of the steps of a flowchart processing method provided in an embodiment of the present application;

[0021] Figure 3 A schematic diagram of a process flow of selecting a suitable image coding model based on image size to perform image coding processing according to an embodiment of the present application;

[0022] Figure 4 A schematic diagram of another step flow of selecting a suitable image coding model based on image size to perform image coding processing provided in an embodiment of the present application;

[0023] Figure 5a A schematic flow chart of the steps of dividing a process image to obtain different image coding features according to an embodiment of the present application;

[0024] Figure 5b A schematic diagram of the effect of a regional image obtained by dividing a process image provided in an embodiment of the present application;

[0025] Figure 5c A schematic diagram of the effect of a regional image obtained by dividing a process image according to an embodiment of the present application;

[0026] Figure 6 A schematic diagram of a process flow for performing text decoding processing on image coding features provided in an embodiment of the present application;

[0027] Figure 7 A schematic diagram of a step flow of training a text decoding model provided in an embodiment of the present application;

[0028] Figure 8 A schematic diagram of a process flow for determining a model loss value provided in an embodiment of the present application;

[0029] Fig. 9 A schematic diagram of the structure of a flow chart processing device provided in an embodiment of the present application;

[0030] Fig.10 A schematic diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0031] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present application. At the same time, in the description of the embodiments of the present application, the terms "first", "second", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more features. In the description of the embodiments of the present application, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0032] In order to better understand the flowchart processing method, device, equipment, storage medium and program product provided by the embodiments of the present application, the application environment applicable to the embodiments of the present application is described below. Specifically, in the field of artificial intelligence, there is usually a need for users to ask questions about a certain process, and because the language model in the field of artificial intelligence is usually unable to directly recognize the process image, it is often necessary to first process the flowchart into specific structured text data through process recognition, so that the text data can be used to assist the language model to complete the response to the conversation text initiated by the user. Specifically, for ease of understanding, such as Figure 1 As shown, Figure 1 This is a schematic diagram of a flow chart image that needs to be identified in the related art. Through the process identification method in the related art, the process image can be processed into the following structured text:

[0033] [Type]|Flowchart

[0034] Start -> Apply

[0035] Application -> Acceptance

[0036] Acceptance->Notify the required materials at one time | Condition: Incomplete materials

[0037] Notify the required supplementary materials at one time -> Application

[0038] Acceptance->Inform the reason and return the materials | Condition: Not meeting the conditions

[0039] Inform the reason and return the materials -> End

[0040] Acceptance->Review|Conditions: Meet the conditions

[0041] Audit->Inform the reason and return the materials | Condition: Audit failed

[0042] Inform the reason and return the materials -> End

[0043] Review->Disbursement|Condition: Review passed

[0044] Payment->Completion

[0045] Finished->End

[0046] After obtaining the above structured text, when the user asks questions about the flowchart, the language model can answer the user's questions in combination with the structured text provided above, thereby ensuring the service effect of artificial intelligence.

[0047] However, in actual application, it is found that the recognition of process images through the above scheme has high requirements on the training of image encoding model and text decoding model. Figure 1 For the relatively simple flowchart shown, the number of characters of the structured text finally output by the text decoding model exceeds 150 characters. In other more complex processes, the number of characters required to be output will be even greater. This places higher demands on the text decoding model to understand the image encoding features and output structured text of a certain sequence length. As a result, there will be a problem that the text decoding model cannot accurately recognize the flowchart content and output structured text of an appropriate length due to the complexity of the flowchart.

[0048] It is precisely to solve the above problems that the present application provides a flowchart processing method, device, equipment, storage medium and program product, which obtains a simplified process node set and node identifier in the process image by encoding and decoding the image. Subsequently, the position information of each process node in the process node set and the implicit position information of the node identifier can be further used to determine the process node corresponding to each node identifier, and finally the connection relationship between the node identifiers is combined to generate the final text. Through the above scheme, compared with directly decoding the image encoding features of the process image to obtain a complex process description text, the problem of deviation in the directly generated process description text due to image encoding and decoding errors can be effectively avoided, thereby improving the accuracy of the generated text.

[0049] In addition, in the process of encoding and decoding images, related technologies often require the use of complex process description texts as training labels for the model to train the corresponding encoding and decoding model, which places high requirements on the training of the model. The present application replaces the complex process description texts with simpler process node sets, node identifiers, and the connection relationship between node identifiers as the training labels for the model, which can reduce the training requirements for the model and improve the training effect of the model. Specifically, the flowchart processing method is usually installed on the flowchart processing device in the form of a computer program, and the flowchart processing device is usually set in the form of a processor on a computer device, such as a server on a business platform. The flowchart processing device in the computer device executes the computer program corresponding to the flowchart processing method to implement the flowchart processing method provided in the embodiment of the present application.

[0050] Specifically, Figure 2 As shown, Figure 2 A flowchart diagram of a flowchart processing method provided in an embodiment of the present application includes steps S210 to S240:

[0051] S210, performing image coding on the process image to obtain image coding features.

[0052] In the embodiment of the present application, image encoding of the process image generally refers to using appropriate image processing technology to encode the feature information in the process image, that is, processing the image into a vector matrix readable by a computer program for subsequent processing, that is, the image encoding feature generally refers to a feature vector or a feature matrix, which is not limited in the embodiment of the present application.

[0053] Specifically, there are many implementation schemes for image coding of process images. For example, a more common one is to use an image coding model trained by machine learning, wherein the image coding model usually processes the pixel values ​​in the process image through trained parameters, thereby finally outputting a set of feature vectors or feature matrices as image coding features. Among them, there are many structures of image coding models here, for example, the more common ones are image coding models based on transformer (a common encoder structure) structure, image models based on CNN (Convolutional Neural Network) structure, or image coding models based on autoencoder structure. Of course, in addition to the several image coding models mentioned above, it is also feasible to use other image coding models that are not mentioned but have image coding functions to encode process images, and the embodiments of the present application will not be repeated here.

[0054] However, considering that the complexity of the process information contained in different flowcharts is different, and the image coding models with different model structures have different principles for image coding, when the image sizes of the process images are different, it is often necessary to use an image coding model with a suitable structure for processing. Otherwise, the image coding model used will be unreasonable, resulting in the inability to generate accurate image coding features for flowcharts of larger sizes. Therefore, as a feasible implementation scheme of the present application, Figure 3 As shown, a schematic flow chart of the steps of selecting a suitable image coding model based on the image size for image coding processing is provided, specifically, including steps S310 to S330:

[0055] S310, determining the image size of the process image.

[0056] In the embodiment of the present application, the image size is a kind of image attribute, which is usually stored in association with the image. Therefore, by parsing the process image, the image size of the process image can be determined, wherein the image size can be understood as the number of pixels of the length and width of the image, and of course, it can also be simply understood as the storage size of the process image. The embodiment of the present application does not limit the definition of the image size.

[0057] S320: Determine a target image coding model from preset image coding models according to the image size.

[0058] In combination with the above-mentioned related description, it can be known that, since the image coding principles of image coding models with different model structures are different, the image coding effect of the image coding model on flow charts with different image sizes is often affected. Therefore, in the embodiment of the present application, the preset image coding model usually includes a plurality of pre-trained image coding models with different structures, and when determining the image size of the flow image, a suitable image coding model is selected from the preset image coding model according to the set mapping relationship as the target image coding model.

[0059] To facilitate understanding of the image coding effect of the flowchart of the image coding model on different image sizes, as a further embodiment of the present application, the preset image coding model generally includes a first image coding model for sequentially encoding images and a second image coding model for selectively encoding images. For example, the more common first image coding model for sequentially encoding images generally refers to an image coding model that adopts a transformer structure or other improved structures implemented based on the transformer, while the second image coding model generally refers to an image coding model that can selectively encode images. For example, the more common ones include a mamba (a neural network model) model that adopts a selective state space to selectively process an input sequence or other improved model structures based on the mamba model. At this time, for the specific implementation steps of determining the target image coding model in the preset image coding model according to the image size, please refer to Figure 4 , specifically including steps S410 to S430:

[0060] S410, comparing the image size with a preset size threshold to determine whether the image size is smaller than the preset size threshold. If yes, execute step S420, if no, execute step S430.

[0061] In the embodiment of the present application, the preset size threshold can usually be set based on actual needs, and can reflect to a certain extent whether the information in the flowchart is complex enough. Specifically, if the image size is smaller than the preset size threshold, it can be considered that the information contained in the flowchart is relatively simple, and when the image size is larger than the preset size threshold, it can be considered that the information contained in the flowchart is relatively complex.

[0062] S420: Determine the first image coding model as a target image coding model.

[0063] In an embodiment of the present application, in combination with the aforementioned related description, when the information contained in the flowchart is relatively simple, the first image coding model for sequentially encoding the image can be selected as the target image coding model, such as the aforementioned transformer structure or an image coding model with other improved structures implemented based on the transformer. Through the above-mentioned image coding model, the model is allowed to focus on different areas of the image at the same time, thereby improving its ability to learn from different areas, so as to improve the feature extraction effect of the image coding model on the image, and improve the information richness in subsequent image coding features.

[0064] S430: Determine the second image coding model as a target image coding model.

[0065] In the embodiment of the present application, combined with the above-mentioned related description, when the information contained in the flowchart is relatively complex, if the image is sequentially encoded, it often takes up more resources, affecting the efficiency of model reasoning. Therefore, a second image coding model for selectively encoding the image can be adopted, for example, a mamba model of the state space or other improved models based on the mamba model is improved by introducing a selection mechanism, so that the coding model can selectively propagate or forget the information in the process based on the input flowchart, while effectively reducing the spatial complexity, ensuring the expression ability of the model, and being able to extract relatively accurate image coding features in the flowchart.

[0066] By using the comparison result of the image size and the preset size threshold provided above, the first image coding model for sequentially encoding the image or the second image coding model for selectively encoding the image is selectively adopted as the target image coding model for processing the target flowchart, thereby ensuring the efficiency of processing the target flowchart while ensuring the effect of the extracted image coding features.

[0067] S330, inputting the process image into the target image coding model to obtain image coding features.

[0068] In the embodiment of the present application, by inputting the process image into a suitable target image coding model for processing, the image coding features containing the corresponding image features in the process diagram can be obtained. The embodiment of the present application will not be described in detail here.

[0069] Of course, as another feasible implementation scheme of the present application, considering that the subsequent text decoding processing of the image coding features needs to be performed, it is necessary to obtain the process node set, node identifiers and the connection relationship between the node identifiers respectively. In the flowchart, the process node set and the connection relationship between the node identifiers are usually related to the process frame lines in the process flowchart. For example, the text content within the process frame lines is usually the process node, and the connecting lines between the process boxes are usually related to the connection relationship between the node identifiers. Therefore, as a further feasible implementation scheme of the present application, the process image can also be reasonably divided based on the recognition results of the lines in the process image to obtain images of different areas containing the process node information and the connection relationship between the node identifiers, so that when input into the target image coding model for processing, the first image coding feature related to the process node information and the second image coding feature related to the connection relationship between the node identifiers can be obtained. For specific implementation schemes, please refer to Figure 5a ,like Figure 5a As shown, Figure 5a A schematic flow chart of a step of dividing a process image to obtain different image coding features is provided in an embodiment of the present application, specifically, comprising steps S510 to S530:

[0070] S510: Perform edge recognition on the process image to obtain line information in the process image.

[0071] In an embodiment of the present application, the recognition of lines in an image usually relies on the edge recognition of the image. Specifically, in a flowchart, the process frame line will usually be in a different color, or the thickness of the frame line will be different, and there will be a significant difference from the area outside the process frame line. Therefore, based on edge recognition, for example, the process image can be processed through various edge operators to identify the line information in the process image, that is, the process frame line.

[0072] S520: Divide the process image according to the line information to obtain a first region image and a second region image in the process image.

[0073] In the embodiment of the present application, based on the line information obtained in the process image, that is, the process frame line, the process image is divided based on the line information, that is, the image is masked, so that a first area image located within the process frame line and a second area image located outside the process frame line can be obtained. It can be understood that the first area image usually contains the text information of a process node within the process frame line, and the second area image usually contains the sequential connection relationship between different process nodes. Specifically, to facilitate understanding of the above content, please refer to Figure 5b and Figure 5c , specifically Figure 1 The flowchart image shown is used as an example for explanation, and schematic diagrams of the effects of the first area image and the second area image in the flowchart image are respectively shown.

[0074] For details, please refer to Figure 5b After identifying the line information in the process image, the area surrounded by the line information is divided according to the line information, so that the first area image of the process image containing the text information of a process node in the process frame line can be obtained, for example Figure 5b As shown, the area within each process frame line can be regarded as a first region image, and the encoding model can accurately identify the text information within the process frame line when processing these first region images.

[0075] See also Figure 5c ,and Figure 5bCorrespondingly, the area surrounded by the line information is masked according to the line information, and only the content outside the process frame is retained, so that a second area image containing the sequential connection relationship between different process nodes in the process image can be obtained. At this time, when the encoding model processes the second area image, it can pay more attention to the sequential connection relationship between the process nodes without being disturbed by the text information in the process frame, thereby ensuring the accuracy of the sequential connection relationship between the identified process nodes.

[0076] S530: Input the first region image and the second region image into the target image coding model to obtain a first image coding feature and a second image coding feature.

[0077] In the embodiment of the present application, the first region image and the second region image are respectively input into the target image coding model for processing, so that the image coding model can focus on different regions in the process image, thereby obtaining different image coding features. For example, by encoding the first region image containing the text information of a process node in the process frame line, the first image coding feature associated with the text information of a process node in the process frame line can often be obtained, and by encoding the second image coding feature containing the sequential connection relationship between different process nodes, the second image coding feature associated with the sequential connection relationship between process nodes can often be obtained. In this way, when the image coding feature is subsequently decoded by the text decoding model, the corresponding text information can be generated.

[0078] Of course, in addition to the aforementioned implementation steps for dividing the process image, in order to improve the feature encoding effect of the process image, in the actual process of image encoding processing of the target process image, the process image can also be enhanced or other preprocessing operations can be performed in other ways. For example, as a common feasible implementation scheme, the target process image can also be processed by Gaussian blur and Gaussian noise with a certain probability, and then switched through random channels of RGB (three primary colors of red, green and blue), and random values ​​within a specified range are added or subtracted at each pixel with corresponding probability, thereby obtaining an enhanced process image. The embodiment of the present application does not limit the implementation scheme of enhancing the process image or performing other preprocessing operations.

[0079] S220, text decoding is performed on the image coding feature to obtain a process node set, a plurality of node identifiers, and a connection relationship between the node identifiers in the process image.

[0080] After completing the image encoding process of the process image through the above-mentioned embodiments and obtaining the image encoding features containing the image information, the encoded image features can be effectively restored to the corresponding text description content by further text decoding process on the image encoding features. However, unlike the related art, in the embodiments of the present application, the text decoding process of the image encoding features does not directly generate a complete process description text corresponding to the process image, but will output the simplified text data, specifically including a process node set, multiple node identifiers, and the connection relationship between the node identifiers.

[0081] Among them, the process node set includes multiple process nodes and the location information of each process node. The process node is used to indicate the text content within the process frame, that is, the node name. For example, the process nodes include start, application, acceptance, review, completion, end and other process nodes. The location information of the process node can be used to indicate the position of the process node in the entire flowchart, for example, the first process node, the second process node..., there is no limitation here.

[0082] Among them, the node identifier uses a simplified identifier to distinguish different process nodes, and uses the connection relationship between the node identifiers to describe the association relationship between the process nodes. For example, node identifiers such as 1, 2, and 3 can be used to replace each process node respectively. Node identifier 1 represents the first process node "start", node identifier 2 represents the second process node "apply"..., and the connection relationship between different node identifiers can be represented by 1->2, 5->10, and 3->6, etc.

[0083] Optionally, the flowchart can be input into a preset text decoding model to obtain a process node set of the process image, the position information of each process node in the process node set, the node identifier, and the connection relationship between the node identifiers. The training process of the preset text decoding model will be described in detail below and will not be repeated here.

[0084] Specifically, in order to facilitate understanding of the process node set, node identifiers, and connection relationships between node identifiers provided in the embodiments of the present application, Figure 1 A schematic diagram of a flowchart is provided as an example for explanation. Different from the process description text of more than 150 characters obtained by processing the process identification of the flowchart in the related art, the process node set, node identifiers and the connection relationship between the node identifiers obtained by the embodiment of the present application are specifically as follows:

[0085] [Type]|Flowchart

[0086] [Node]|Start|Apply|Accept|Notify the required materials at one time|Notify the reason and return the materials|Review|Notify the reason and return the materials|Pay|Complete|End

[0087] [Relationship]|1->2|2->3|3->4% Incomplete materials|4->2|3->5% Not eligible|5->10|3->6% Eligible|6->7% Failed review|7->10|8->9|9->10

[0088] It can be seen that compared with the process description text directly generated above, the text information generated by the present application based on text decoding mainly includes a process node set, node identifiers and the connection relationship between node identifiers, wherein the process node set is composed of multiple process nodes in a certain order, for example, the start, application, acceptance and other process nodes in the above output results, and in the node identifier, a simplified digital node identifier, such as 1, 2, 3, etc., is used to replace each different process node, and the connection relationship between different node identifiers is given, such as 1->2, 5->10 and 3->6, etc. At this time, it can be seen that the characters of the text information generated by the process identification method provided by the present application do not exceed 90, which is significantly reduced compared to the process description text of more than 150 characters obtained by the process identification of the flowchart by the related technology, and in the flowchart with more complex processes, the output sequence length can be reduced by about 50%, that is, there is a lower requirement for the text sequence length required to be output by the text decoding model, which reduces the resource occupation of the text decoding model, thereby significantly improving the efficiency and effect of text decoding.

[0089] Specifically, text decoding processing of image coding features can usually be implemented based on a preset text decoding model, wherein the specific model structure of the text decoding model is not limited in this embodiment of the present application, and any method that can decode and restore a given feature vector or feature matrix into a text sequence with a given style is feasible. Of course, in the embodiment of the present application, the text sequence obtained by text decoding processing of image coding features includes a set of process nodes, multiple node identifiers, and a connection relationship between the node identifiers.

[0090] Furthermore, considering that in the aforementioned process of inputting the process image into the image coding model for processing to obtain the image coding features, the line information in the process image is extracted based on the edge recognition of the process image, and the process image is linearly divided based on the line, thereby obtaining the first area image in the process image that is more relevant to the text in the process box and the second area image that is more relevant to the connecting arrows between the nodes in the process box, so that when input into the image coding model, the first image coding features that are more relevant to the text in the process box and the second image coding features that are more relevant to the connecting arrows between the nodes in the process box can be obtained. Therefore, as a feasible embodiment of the present application, the image coding features are text-decoded by a preset text decoding model, and different image coding features in the image coding features can also be processed by using the preset text decoding model, so as to obtain the process node sequence and node relationship in the target process image respectively.

[0091] For details, please refer to Figure 6 , Figure 6 A schematic flow chart of steps for performing text decoding processing on image coding features provided in an embodiment of the present application, specifically, comprising steps S610 to S630:

[0092] S610: Input a first image coding feature among the image coding features into a preset text decoding model to obtain a process node set of the process image.

[0093] In the embodiment of the present application, since the first image coding feature in the image coding feature usually refers to the image coding model processing the first area image within the process frame line that is more relevant to the text in the process frame, therefore, the first image coding feature is input into the preset text decoding model for processing, so that the text decoding model can pay more attention to the text information in the process frame, thereby outputting a set of process nodes that are more related to the text information, such as the aforementioned "|start|apply|accept|notify the materials to be supplemented at one time|notify the reasons and return the materials|review|notify the reasons and return the materials|disburse|complete|end" and so on.

[0094] S620: Input a second image coding feature in the image coding features into a preset text decoding model to obtain a plurality of node identifiers of the process image and connection relationships between the node identifiers.

[0095] In the embodiment of the present application, similar to the aforementioned first image coding feature, since the second image coding feature in the image coding feature generally refers to the image coding model processing the second area image outside the process frame line that is more related to the sequential connection arrows between the process frames. Therefore, inputting the second image coding feature into the preset text decoding model for processing can make the text decoding model pay more attention to the sequential connection relationship outside the process frame, thereby outputting the node identifiers that are more related to the sequential connection relationship and the connection relationship between the node identifiers, such as the aforementioned "|1->2|2->3|3->4|4->2|3->5|5->10|3->6|6->7|7->10|8->9|9->10" and so on.

[0096] Of course, it should be noted that in addition to the multiple node identifiers provided above and the connection relationship between the node identifiers, the node relationship output by text decoding processing of the image coding features usually also includes conditional information corresponding to the connection relationship, such as the aforementioned "3->4% incomplete materials", "3->5% not meeting the conditions", etc. Therefore, as a feasible embodiment of the present application, text decoding processing of image coding features can further output conditional information in the node relationship, as follows.

[0097] S630: Input the image coding feature into a preset text decoding model to obtain condition information of the process image.

[0098] In an embodiment of the present application, by inputting image coding features into a preset text decoding model, conditional information of the node relationship in the target process image can be further obtained. For example, in the connection relationship between nodes 3 and 4 mentioned above, the conditional information of "incomplete materials" needs to exist, and in the connection relationship between nodes 3 and 5, the conditional information of "not meeting the conditions" needs to exist.

[0099] Of course, it should be noted that the above is only an example of a simple text decoding model. In fact, in the process of using the text decoding model to perform text decoding processing on image coding features, other relevant knowledge in machine learning can also be added to the text decoding model, such as introducing a self-attention mechanism or other mechanisms to improve the processing effect of text decoding processing on image coding features, thereby further improving the accuracy of the output process node sequence and node relationship. The embodiments of the present application are not limited here.

[0100] Of course, it should be noted that in order to enable the text decoding model provided above to perform text decoding processing on the image encoding features to generate a set of process nodes, node identifiers, and connection relationships between node identifiers, it is usually necessary to train the text decoding model based on the idea of ​​machine learning. Specifically, for ease of understanding, as another feasible implementation scheme of this application, please refer to Figure 7 , Figure 7 A flowchart of a training text decoding model provided in an embodiment of the present application, specifically, includes steps S710 to S740:

[0101] S710, determining a sample process image, and node set labels, node identification labels, and connection relationship labels between the node identification labels corresponding to the sample process image.

[0102] In the embodiment of the present application, the sample process image can usually be a sample process image extracted from different documents based on technologies such as web crawlers. Generally, these sample process images have different process information. Further, the node set labels, node identification labels, and connection relationship labels corresponding to the sample process image can usually be pre-labeled or manually annotated, similar to the aforementioned label information such as "|Start|Apply|Accept|Notify the materials to be supplemented at one time|Notify the reasons and return the materials|Review|Notify the reasons and return the materials|Disburse|Complete|End" and "|1->2|2->3|3->4% materials are incomplete|4->2|3->5% do not meet the conditions|5->10|3->6% meet the conditions|6->7% fail the review|7->10|8->9|9->10".

[0103] S720, inputting the image coding features of the sample process image into the initial text decoding model to obtain a prediction node set, prediction node identifiers, and prediction connection relationships between the prediction node identifiers corresponding to the sample process image.

[0104] In an embodiment of the present application, the image coding features of the sample process image can be similar to the steps of performing image coding processing on the target process image mentioned above, and can be obtained by processing the sample process image through a trained image coding model. Of course, the image coding model can also be trained synchronously with the text decoding model. At this time, the image coding features of the sample process image are obtained by processing the sample process image through the initial image coding model.

[0105] In an embodiment of the present application, by inputting the image coding features of the sample process image into the initial text decoding model, the initial text decoding model can encode the image coding features and predict and generate corresponding prediction node sets, prediction node identifiers, and predicted connection relationships between the prediction node identifiers. Of course, under the condition of parameter initialization in the initial text decoding model, the prediction node sets, prediction node identifiers, and predicted connection relationships between the prediction node identifiers predicted and generated by encoding the image coding features will often be significantly different from the node set labels, node identifier labels, and connection relationship labels between the node identifier labels corresponding to the sample process image.

[0106] S730, determining a model loss value corresponding to the initial text decoding model according to the difference between the predicted node set and the node set label, the difference between the predicted node identifier and the node identifier label, and the difference between the predicted connection relationship and the connection relationship label.

[0107] In an embodiment of the present application, on the basis of the foregoing, by comparing the difference between the predicted node set and the node set label, the difference between the predicted node identifier and the node identifier label, and the difference between the predicted connection relationship and the connection relationship label, the model loss value corresponding to the initial text decoding model can be determined. It can be understood that the greater the difference between the predicted node set and the node set label, the greater the difference between the predicted node identifier and the node identifier label, or the greater the difference between the predicted connection relationship and the connection relationship label, it indicates that the initial text decoding model cannot accurately restore the connection relationship between the process node set and the node identifier from the image coding features. At this time, it is often necessary to iteratively update the initial text decoding model or the parameters in the initial text decoding model and the initial image coding model.

[0108] Specifically, since the prediction node set includes the process nodes and the node location information, the model loss value corresponding to the initial text decoding model is also determined based on the difference between the above different information. For details, please refer to Figure 8 , Figure 8 A schematic flow chart of a step of determining a model loss value provided in an embodiment of the present application, specifically, comprising steps S810 to S850:

[0109] S810, determining a first loss value corresponding to the initial text decoding model according to the predicted process nodes in the predicted node set and the process node labels in the node set labels.

[0110] In an embodiment of the present application, the predicted process nodes in the predicted node set include several process nodes decoded from the image coding features, such as the aforementioned "start", "acceptance", "review", etc. Of course, due to the parameter initialization in the initial text decoding model, these predicted process nodes often have certain differences from the process node labels in the node set label. Therefore, the first loss value corresponding to the initial text decoding model can be determined based on the difference between the predicted process nodes in the predicted node set and the process node labels in the node set label. For example, the loss value can be determined based on the number or number proportion of process nodes in the predicted node set that do not exist in the node set label, or based on the number or number proportion of process nodes in the node set label that do not exist in the predicted node set. The more the number or the higher the number proportion, the higher the first loss value, and the worse the model's effect on text decoding. Of course, it is also feasible to determine the loss value based on the number or number proportion of process nodes coexisting in the predicted node set and the process set label. At this time, the lower the number or the smaller the number proportion, the higher the first loss value, and the worse the model's effect on text decoding.

[0111] S820, determining a second loss value corresponding to the initial text decoding model according to the position prediction information of each predicted process node in the predicted node set and the position label information of each process node label in the node set label.

[0112] In an embodiment of the present application, on the basis of the foregoing, there is also position prediction information of each predicted process node in the prediction node set. For example, the position prediction information is determined by the order of each predicted process node in the prediction set sequence, and each process node label in the corresponding node set label also has corresponding position label information. Therefore, since the position information will affect the determination of subsequent node relationships, in addition to the need to compare the predicted process nodes and the process node labels, it is also necessary to compare the difference between the position prediction information of the predicted process nodes and the position label information of the process node labels. It can be understood that the higher the position deviation between the position prediction information of the predicted process nodes and the position label information of the process node labels, the greater the second loss value corresponding to the initial text decoding model, and the worse the effect of the model on text decoding.

[0113] S830: Determine a third loss value corresponding to the initial text decoding model according to the predicted node identifier and the node identifier label.

[0114] In an embodiment of the present application, in addition to the difference between the predicted node sequence and the node sequence label provided above, in order for the model to accurately predict the node relationship, it is also necessary to compare the predicted node identifier and the node identifier label. For example, the numerical node identifiers mentioned above (such as 1, 2, 3) are compared. Obviously, the greater the difference between the predicted node identifier and the node identifier label, the greater the third loss value corresponding to the initial text decoding model, and the worse the model's effect on text decoding.

[0115] S840, determining a fourth loss value corresponding to the initial text decoding model according to the predicted connection relationship and the connection relationship label.

[0116] In an embodiment of the present application, of course, in addition to the aforementioned differences between predicted node identifiers and node identifier labels, it also includes predicted connection relationships between predicted node identifiers and connection relationship labels between node identifier labels. For example, the loss value can be determined based on the number or proportion of connection relationships in the predicted connection relationship that do not exist in the connection relationship label, or based on the number or proportion of connection relationships in the connection relationship label that do not exist in the predicted connection relationship. The greater the number or the higher the proportion, the higher the fourth loss value, and the worse the model's text decoding effect. Of course, it is also feasible to determine the loss value based on the number or proportion of connection relationships coexisting in the predicted connection relationship and the connection relationship label. At this time, the lower the number or the smaller the proportion, the higher the fourth loss value, and the worse the model's text decoding effect.

[0117] Of course, it should be noted that the above provides an implementation scheme for determining the loss values ​​of the initial text decoding model in different dimensions based on different information to describe the text decoding effect of the initial text decoding model. The specific calculation formula for the loss value can be set based on actual needs, such as the common cross-entropy loss value calculation formula. Of course, it is also feasible to use other loss value calculation formulas, which is not limited in the embodiments of the present application.

[0118] S850, determining a model loss value corresponding to the initial text decoding model according to the first loss value, the second loss value, the third loss value and the fourth loss value.

[0119] In an embodiment of the present application, on the basis of the foregoing, by weighted fusion of the first loss value, the second loss value, the third loss value and the fourth loss value according to a preset weight coefficient, the model loss value corresponding to the initial text decoding model can be further determined. Generally, the higher the model loss value corresponding to the initial text decoding model, the less ideal the text decoding effect of the initial text decoding model.

[0120] By comparing the differences between predicted process nodes and process node labels, the differences between predicted position prediction information of predicted process nodes and position label information of process node labels, the differences between predicted node identifiers and node identifier labels, and the differences between predicted connection relationships between predicted node identifiers and connection relationship labels between node identifier labels, the initial text decoding model can learn from sample data and labels the ability to decode text to obtain accurate process node sequences and node relationships, thereby improving the training effect of the initial text decoding model.

[0121] S740, updating the initial text decoding model according to the model loss value until the updated text decoding model meets the preset training conditions, and determining the current updated text decoding model as the preset text decoding model.

[0122] In an embodiment of the present application, after determining the model loss value corresponding to the initial text decoding model, the initial text decoding model is updated based on the back-propagation idea to obtain an updated text decoding model, which improves the effect of the text decoding model on decoding the process node sequence and node relationships. At the same time, the training samples can be input into the updated text decoding model again to calculate the loss value until the updated text decoding model meets the preset training conditions. For example, when the loss value obtained by several consecutive calculations is less than a certain threshold, it can be considered that the text decoding model has been trained, and the currently updated text decoding model can be determined as the preset text decoding model.

[0123] Of course, if the initial image coding model needs to be trained synchronously, it is necessary to update the initial image coding model at the same time based on the back-propagation idea to update the image coding features of the training samples until the preset training conditions are met. The currently updated image coding model can also be used as the preset image coding model for subsequent image coding processing of the target process image.

[0124] Through the training of the text decoding model provided above, the text decoding model can complete the decoding of the image encoding features, and recover the process node set, node identifiers and the connection relationship between node identifiers in the process image for further use in subsequent calculations.

[0125] S230, determining the process node corresponding to each of the node identifiers according to the position information of each of the process nodes in the process node set and the position information included in the node identifier.

[0126] In an embodiment of the present application, after obtaining the aforementioned process node sequence set such as "|start|apply|accept|notify of materials to be supplemented at one time|notify of reasons and return of materials|review|notify of reasons and return of materials|disbursement|completion|end", and node identifiers and connection relationships such as "|1->2|2->3|3->4% incomplete materials|4->2|3->5% do not meet the conditions|5->10|3->6% meet the conditions|6->7% fail the review|7->10|8->9|9->10", the process node corresponding to each node identifier can be determined by further using the position information of each process node in the process node set and the position information implied by the node identifier.

[0127] For example, the position information of the process node "start" in the process node set is the first process node, and the position information of the process node "apply" in the process node set is the second process node, and the position information implied by the node identifiers "1" and "2" are the first process node and the second process node, respectively. Therefore, it can be determined that the process nodes corresponding to the node identifiers "1" and "2" contained in the connection relationship "|1->2|" are "start" and "apply". Similarly, the process nodes corresponding to the node identifiers "3" and "4" in the connection relationship "|3->4|" are the third and fourth process nodes in the process node set, which are "acceptance" and "one-time notification of materials to be supplemented" respectively.

[0128] S240: Generate text corresponding to the process image according to each of the process nodes, the node identifier of each of the process nodes, and the connection relationship between the node identifiers.

[0129] In the embodiment of the present application, after determining the process node corresponding to each node identifier, that is, after determining the node identifier of each process node, the text corresponding to the process image can be generated in combination with the process node and the connection relationship in the node relationship, which is used to describe the process information in the process image. For example, "3->4% incomplete materials" describes that under the condition of incomplete materials, the process needs to be changed from the "acceptance" process to the "one-time notification of materials to be supplemented" process. Based on the same principle, a complete process description text corresponding to the process image can be finally generated.

[0130] Of course, considering that process recognition is usually applied in the field of artificial intelligence services, that is, it is usually necessary to answer by using the recognized process description text and a large language model after determining the user's conversation text. At this time, that is, after the step of generating the text corresponding to the process image according to each process node, the node identifier of each process node and the connection relationship between the node identifiers, the flowchart processing method also includes:

[0131] Determine the user's session text;

[0132] The conversation text and the text corresponding to the process image are input into a language model for processing to obtain a response text of the conversation text.

[0133] Among them, the user's conversation text usually refers to the "question text" related to the flowchart input by the user through the client or other means, such as "What should I do if I am prompted that the materials are incomplete?" At this time, the text corresponding to the aforementioned process image is associated with the conversation text as prompt information and input into the language model, such as the GPT (Generative Pre-Trained) language model, the LLaMA (Large Language Model Meta AI, a large-scale language model) language model, the language model can process the conversation text, and obtain the response text of the conversation text, such as "Inform the materials that need to be completed at one time" and output it to the user.

[0134] By adopting the scheme of the application embodiment, after performing image encoding processing and text decoding processing on the process image to obtain the process node set, node identifiers and the connection relationship between the node identifiers, the position information of each process node in the process node set and the position information contained in the node identifier are used to determine the process node corresponding to each node identifier, and finally the final text is generated in combination with the process nodes, the node identifiers corresponding to the process nodes and the connection relationship between the node identifiers. Compared with directly obtaining a complex process description text by performing text decoding processing on the image encoding features of the target process image, by first identifying the relatively simple process node set, node identifiers and the connection relationship between the node identifiers in the flowchart to further generate text for the process description, the problem of deviation in the directly generated process description text due to image encoding and decoding errors can be effectively avoided, thereby improving the accuracy of the generated text.

[0135] In addition, in the process of encoding and decoding images, related technologies often need to use complex process description texts as training labels for the model to train the corresponding encoding and decoding model, which places high requirements on model training. The present application replaces the complex process description texts with simpler process node sets, node identifiers, and the connection relationships between node identifiers as model training labels, which can reduce the training requirements for the model and improve the training effect of the model, especially for large-size process images containing complex process information, which can further improve the processing effect of the flowchart.

[0136] In order to better implement the flowchart processing method provided in the embodiment of the present application, the present embodiment also provides a flowchart processing device, which can be integrated in a terminal device. Fig. 9As shown, the flowchart processing device may include:

[0137] The encoding module 901 is used to perform image encoding on the process image to obtain image encoding features;

[0138] A decoding module 902 is used to perform text decoding processing on the image coding feature to obtain a process node set, a plurality of node identifiers and a connection relationship between the node identifiers of the process image;

[0139] A determination module 903, configured to determine the process node corresponding to each of the node identifiers according to the position information of each of the process nodes in the process node set and the position information contained in the node identifier;

[0140] The generating module 904 is used to generate text corresponding to the process image according to each of the process nodes, the node identifier of each of the process nodes and the connection relationship between the node identifiers.

[0141] In some embodiments, the encoding module is further used to determine the image size of the process image; determine the target image encoding model from a preset image encoding model according to the image size; and input the process image into the target image encoding model to obtain image encoding features.

[0142] In some embodiments, the encoding module is also used to determine the target image coding model from the preset image coding models based on the image size, including: comparing the image size with a preset size threshold; when the image size is smaller than the preset size threshold, determining the first image coding model as the target image coding model; when the image size is greater than or equal to the preset size threshold, determining the second image coding model as the target image coding model.

[0143] In some embodiments, the encoding module is also used to perform edge recognition on the process image to obtain line information in the process image; divide the process image according to the line information to obtain a first area image and a second area image in the process image; input the first area image and the second area image into the target image encoding model to obtain a first image encoding feature and a second image encoding feature.

[0144] In some embodiments, the decoding module is also used to input the first image coding feature among the image coding features into a preset text decoding model to obtain a set of process nodes of the process image; and input the second image coding feature among the image coding features into a preset text decoding model to obtain multiple node identifiers of the target process image and the connection relationship between the node identifiers.

[0145] In some embodiments, the decoding module is further used to input the image coding features into a preset text decoding model to obtain conditional information of the node relationship in the process image.

[0146] In some embodiments, the decoding module is also used to determine a sample process image, and a node set label, a node identification label, and a connection relationship label between the node identification labels corresponding to the sample process image; input the image coding features of the sample process image into the initial text decoding model to obtain a predicted node set, a predicted node identification, and a predicted connection relationship between the predicted node identifications corresponding to the sample process image; determine a model loss value corresponding to the initial text decoding model based on the difference between the predicted node set and the node set label, the difference between the predicted node identification and the node identification label, and the difference between the predicted connection relationship and the connection relationship label; update the initial text decoding model based on the model loss value, and determine the current updated text decoding model as the preset text decoding model when the updated text decoding model meets the preset training conditions.

[0147] In some embodiments, the decoding module is also used to determine a first loss value corresponding to the initial text decoding model based on the predicted process nodes in the predicted node set and the process node labels in the node set labels; determine a second loss value corresponding to the initial text decoding model based on the position prediction information of each predicted process node in the predicted node set and the position label information of each process node label in the node set labels; determine a third loss value corresponding to the initial text decoding model based on the predicted node identifier and the node identifier label; determine a fourth loss value corresponding to the initial text decoding model based on the predicted connection relationship and the connection relationship label; determine the model loss value corresponding to the initial text decoding model based on the first loss value, the second loss value, the third loss value and the fourth loss value.

[0148] In some embodiments, the generation module is further used to determine the user's conversation text; input the conversation text and the text corresponding to the process image into the language model for processing to obtain a response text for the conversation text.

[0149] By adopting the scheme of the application embodiment, after performing image encoding processing and text decoding processing on the process image to obtain the process node set, node identifiers and the connection relationship between the node identifiers, the position information of each process node in the process node set and the position information contained in the node identifier are used to determine the process node corresponding to each node identifier, and finally the final text is generated in combination with the process nodes, the node identifiers corresponding to the process nodes and the connection relationship between the node identifiers. Compared with directly obtaining a complex process description text by performing text decoding processing on the image encoding features of the target process image, by first identifying the relatively simple process node set, node identifiers and the connection relationship between the node identifiers in the flowchart to further generate text for the process description, the problem of deviation in the directly generated process description text due to image encoding and decoding errors can be effectively avoided, thereby improving the accuracy of the generated text.

[0150] In addition, in the process of encoding and decoding images, related technologies often need to use complex process description texts as training labels for the model to train the corresponding encoding and decoding model, which places high requirements on model training. The present application replaces the complex process description texts with simpler process node sets, node identifiers, and the connection relationships between node identifiers as model training labels, which can reduce the training requirements for the model and improve the training effect of the model, especially for large-size process images containing complex process information, which can further improve the processing effect of the flowchart.

[0151] Based on the same inventive concept, the embodiment of the present application also provides a computer device, which may be a server or a terminal device, and includes a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps of the above flowchart processing method when executing the computer program. Thereby, various functions are realized, for example:

[0152] Perform image coding on the process image to obtain image coding features;

[0153] Performing text decoding on the image coding feature to obtain a process node set, a plurality of node identifiers, and a connection relationship between the node identifiers of the target process image;

[0154] Determine the process node corresponding to each of the node identifiers according to the position information of each of the process nodes in the process node set and the position information contained in the node identifier;

[0155] The text corresponding to the process image is generated according to each of the process nodes, the node identifier of each of the process nodes and the connection relationship between the node identifiers.

[0156] In one embodiment, the computer device is a terminal device, for example, its internal structure diagram can be as follows: Fig.10 As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a flowchart processing method is implemented. The display unit of the computer device is used to form a visually visible image, and can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer covered on the display screen, or a button, trackball or touchpad set on the computer device casing, or an external keyboard, touchpad or mouse, etc.

[0157] Those skilled in the art will understand that Fig.10 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0158] Based on the same inventive concept, an embodiment of the present application also provides a computer-readable storage medium, which may include: a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc.

[0159] Since the computer program stored in the computer-readable storage medium can execute any one of the flowchart processing methods provided in the embodiments of the present application, the beneficial effects that can be achieved by any one of the flowchart processing methods provided in the embodiments of the present application can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0160] Based on the same inventive concept, the embodiment of the present application further provides a computer program product or a computer program, which includes a computer instruction, and the computer instruction is stored in a computer-readable storage medium. The processor of the computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device executes the method provided in various optional implementations in the above embodiments.

[0161] It should be noted that the object data (including but not limited to user device information, user personal information, etc.) and conversation data involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions. A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods.

[0162] Any reference to the memory, database or other media used in the embodiments provided in the present application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (Re RAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory may include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0163] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but is not limited thereto.

[0164] In the above-mentioned flowchart processing device, computer-readable storage medium, computer equipment, and computer program product embodiments, the description of each embodiment has its own emphasis. For parts not described in detail in a certain embodiment, reference can be made to the relevant description of other embodiments. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process and beneficial effects of the process identification device, computer-readable storage medium, computer program product, computer equipment and its corresponding units described above can refer to the description of the process identification method in the above embodiment, and will not be repeated here.

[0165] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0166] The above is a detailed introduction to a flowchart processing method, device, equipment, storage medium and computer program product provided in the embodiments of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, according to the ideas of the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A flowchart processing method, characterized in that: include: Perform image coding on the process image to obtain image coding features; Performing text decoding on the image coding feature to obtain a process node set, a plurality of node identifiers, and a connection relationship between the node identifiers of the process image; Determine the process node corresponding to each of the node identifiers according to the position information of each process node in the process node set and the position information contained in the node identifier; The text corresponding to the process image is generated according to each of the process nodes, the node identifier of each of the process nodes and the connection relationship between the node identifiers.

2. The method according to claim 1, characterized in that The performing image coding processing on the process image to obtain image coding features includes: Determine the image size of the process image; Determining a target image coding model from preset image coding models according to the image size; The process image is input into the target image coding model to obtain image coding features.

3. The method according to claim 2, characterized in that The preset image coding model includes a first image coding model for sequentially coding images and a second image coding model for selectively coding images; The step of determining a target image coding model from a preset image coding model according to the image size comprises: Comparing the image size with a preset size threshold; When the image size is smaller than a preset size threshold, determining the first image coding model as the target image coding model; When the image size is greater than or equal to the preset size threshold, the second image coding model is determined as the target image coding model.

4. The method according to claim 2, characterized in that: The image coding feature includes a first image coding feature and a second image coding feature; the step of inputting the process image into the target image coding model to obtain the image coding feature includes: Performing edge recognition on the process image to obtain line information in the process image; Dividing the process image according to the line information to obtain a first region image and a second region image in the process image; The first region image and the second region image are input into the target image coding model to obtain the first image coding feature and the second image coding feature.

5. The method according to claim 1, characterized in that The text decoding of the image coding feature to obtain a process node set of the process image, a plurality of node identifiers, and a connection relationship between the node identifiers includes: Inputting a first image coding feature among the image coding features into a preset text decoding model to obtain a process node set of the process image; A second image coding feature among the image coding features is input into the preset text decoding model to obtain a plurality of node identifiers of the process image and connection relationships between the node identifiers.

6. The method according to claim 5, characterized in that The preset text decoding model is trained by the following steps: Determine a sample process image, and a node set label, a node identification label, and a connection relationship label between the node identification labels corresponding to the sample process image; Inputting the image coding features of the sample process image into the initial text decoding model to obtain a set of prediction nodes, prediction node identifiers, and prediction connection relationships between the prediction node identifiers corresponding to the sample process image; Determine a model loss value corresponding to the initial text decoding model according to a difference between the predicted node set and the node set label, a difference between the predicted node identifier and the node identifier label, and a difference between the predicted connection relationship and the connection relationship label; The initial text decoding model is updated according to the model loss value until the updated text decoding model meets the preset training conditions, and the current updated text decoding model is determined as the preset text decoding model.

7. The method according to claim 6, characterized in that Determining the model loss value corresponding to the initial text decoding model according to the difference between the predicted node set and the node set label, the difference between the predicted node identifier and the node identifier label, and the difference between the predicted connection relationship and the connection relationship label, includes: Determining a first loss value corresponding to the initial text decoding model according to the predicted process nodes in the predicted node set and the process node labels in the node set labels; Determine a second loss value corresponding to the initial text decoding model according to the position prediction information of each predicted process node in the predicted node set and the position label information of each process node label in the node set label; Determining a third loss value corresponding to the initial text decoding model according to the predicted node identifier and the node identifier label; Determining a fourth loss value corresponding to the initial text decoding model according to the predicted connection relationship and the connection relationship label; Determine a model loss value corresponding to the initial text decoding model according to the first loss value, the second loss value, the third loss value and the fourth loss value.

8. The method according to any one of claims 1 to 7, characterized in that: After generating text corresponding to the process image according to each of the process nodes, the node identifier of each of the process nodes, and the connection relationship between the node identifiers, the method further includes: Determine the user's session text; The conversation text and the text corresponding to the process image are input into a language model for processing to obtain a response text of the conversation text.

9. A flowchart processing device, characterized in that: include: A coding module, used for performing image coding on the process image to obtain image coding features; A decoding module, used for text decoding the image coding feature to obtain a process node set, a plurality of node identifiers and a connection relationship between the node identifiers of the process image; A determination module, used to determine the process node corresponding to each of the node identifiers according to the position information of each of the process nodes in the process node set and the position information contained in the node identifier; A generating module is used to generate text corresponding to the process image according to each of the process nodes, the node identifier of each of the process nodes and the connection relationship between the node identifiers.

10. A computer device, characterized in that: It comprises a processor and a memory, wherein the memory stores a plurality of instructions; the processor loads instructions from the memory to execute the steps of the flowchart processing method as claimed in any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor to execute the steps of the flowchart processing method according to any one of claims 1 to 8.

12. A computer program product, characterized in that The computer program product comprises a computer program, and the computer program is used by a processor to execute the steps of the flowchart processing method according to any one of claims 1 to 8.

Images