Structure diagram restoration method, device, electronic device, medium and program product

Abstract

The present invention discloses a method, apparatus, electronic device, medium and program product for restoring a structure diagram. The method includes: splitting an original hierarchical structure diagram to obtain m split hierarchical structure diagrams; extracting features from the m split hierarchical structure diagrams to obtain feature information of the m split hierarchical structure diagrams; performing feature fusion on the feature information of the m split hierarchical structure diagrams to obtain fusion information of the original hierarchical structure diagram; and performing restoration processing on the original hierarchical structure diagram according to the fusion information to obtain a corresponding restored hierarchical structure diagram. The above solution can improve the accuracy and convenience of structure diagram restoration.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method, device, electronic device, medium and program product for restoring a structure diagram. Background Art

[0002] A structure diagram is a hierarchical and clear nested structure diagram, which often appears in users' daily work and study life. Among them, taking a mind map as an example, it is an effective graphic thinking tool for expressing divergent thinking. For the convenience of file transmission and user reading, a mind map usually exists in the form of an image. When a mind map is stored and transmitted as a file in image format, there is a big defect - it only supports viewing and does not support editing.

[0003] Therefore, there is an urgent need to propose a structure diagram restoration solution. Summary of the Invention

[0004] Embodiments of the present invention provide a method, device, electronic device, medium and program product for restoring a structure diagram, which improves the convenience and accuracy of structure diagram restoration.

[0005] In a first aspect, embodiments of the present invention provide a method for restoring a structure diagram, including:

[0006] Segmenting an original hierarchical structure diagram to obtain m segmented hierarchical structure diagrams, where m is a positive integer;

[0007] Extracting features from the m segmented hierarchical structure diagrams to obtain feature information of the m segmented hierarchical structure diagrams;

[0008] Performing feature fusion on the feature information of the m segmented hierarchical structure diagrams to obtain fusion information of the original hierarchical structure diagram;

[0009] Performing restoration processing on the original hierarchical structure diagram according to the fusion information to obtain a corresponding restored hierarchical structure diagram.

[0010] Optionally, if there is an overlapping area between any two of the m segmented hierarchical structure diagrams, then the performing feature fusion on the feature information of the m segmented hierarchical structure diagrams to obtain fusion information of the original hierarchical structure diagram includes:

[0011] Performing feature fusion and duplicate removal processing on the feature information of the m segmented hierarchical structure diagrams to obtain fusion information of the original hierarchical structure diagram.

[0012] Optionally, the feature information includes the segmented text in the segmented hierarchical structure diagram, the fusion information includes the original text in the original hierarchical structure diagram, and the feature fusion of the feature information of the m segmented hierarchical structure diagrams to obtain the fusion information of the original hierarchical structure diagram includes:

[0013] According to the text position information of each segmented text in the m segmented hierarchical structure diagrams, each segmented text is converted into the original hierarchical structure diagram to obtain the converted text in the original hierarchical structure diagram;

[0014] The converted text is de-duplicated to obtain the original text.

[0015] Optionally, the feature information includes i segmented hierarchical nodes in the segmented hierarchical structure diagram, the fusion information includes n original hierarchical nodes in the original hierarchical structure diagram, and the feature fusion of the feature information of the m segmented hierarchical structure diagrams to obtain the fusion information of the original hierarchical structure diagram includes:

[0016] According to the node position information of each segmented hierarchical node in the m segmented hierarchical structure diagrams, each segmented hierarchical node is converted into the original hierarchical structure diagram to obtain j converted hierarchical nodes in the original hierarchical structure diagram, where both i and j are positive integers, and j is greater than i;

[0017] The j converted hierarchical nodes are de-duplicated to obtain n original hierarchical nodes, where n is a positive integer less than j.

[0018] Optionally, the feature information includes the segmented logical connection lines between the segmented hierarchical nodes in the segmented hierarchical structure diagram, the segmented logical connection lines include multiple feature points, the fusion information includes the original logical connection lines between the original hierarchical nodes in the original hierarchical structure diagram, and the feature fusion of the feature information of the m segmented hierarchical structure diagrams to obtain the fusion information of the original hierarchical structure diagram includes:

[0019] According to the pixel position information of each feature point, the feature points of the segmented logical connection lines between the segmented hierarchical nodes in the m segmented hierarchical structure diagrams are fused and de-duplicated to obtain the original logical connection lines.

[0020] Optionally, the fusion information includes the original text, n original hierarchical nodes and the original logical connection lines between the original hierarchical nodes in the original hierarchical structure diagram, where n is a positive integer. According to the fusion information, the reduction process of the original hierarchical structure diagram to obtain the corresponding reduced hierarchical structure diagram includes:

[0021] Perform structural reconstruction on the n original hierarchical nodes in the original hierarchical structure diagram and the original logical connections between the original hierarchical nodes to obtain a reconstructed hierarchical structure;

[0022] Associate and match the original text with the reconstructed hierarchical structure to obtain the restored hierarchical structure diagram.

[0023] Optionally, the original text includes at least one original text line and the position information of each original text line. The step of associating and matching the original text with the reconstructed hierarchical structure to obtain the restored hierarchical structure diagram includes:

[0024] According to the position information of each original text line, add each original text line to the reconstructed hierarchical structure correspondingly to obtain the restored hierarchical structure diagram.

[0025] Optionally, the method includes:

[0026] Receive an editing instruction for the restored hierarchical structure diagram. The editing instruction is used to modify the target information in the restored hierarchical structure diagram, and the target information includes at least one of the following: target text line, target hierarchical node, and target logical connection;

[0027] Respond to the editing instruction and perform corresponding modification processing on the target information in the restored hierarchical structure diagram.

[0028] In a second aspect, an embodiment of the present invention provides a structure diagram restoration device, including:

[0029] A splitting module, configured to split the original hierarchical structure diagram to obtain m split hierarchical structure diagrams, where m is a positive integer;

[0030] An extraction module, configured to extract feature information of the m split hierarchical structure diagrams to obtain the feature information of the m split hierarchical structure diagrams;

[0031] A fusion module, configured to perform feature fusion on the feature information of the m split hierarchical structure diagrams to obtain the fusion information of the original hierarchical structure diagram;

[0032] A restoration module, configured to perform restoration processing on the original hierarchical structure diagram according to the fusion information to obtain a corresponding restored hierarchical structure diagram.

[0033] Regarding the content not introduced or described in the embodiments of the present invention, reference can be made to the relevant introductions in the foregoing method embodiments correspondingly, and details are not described herein again.

[0034] In a third aspect, an embodiment of the present invention provides an electronic device, including a memory and one or more programs, where the one or more programs are stored in the memory and are configured to be executed by one or more processors to perform operation instructions included in the one or more programs for performing the structure diagram restoration method provided in the first aspect.

[0035] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, it implements the steps corresponding to the structure diagram restoration method provided in the first aspect.

[0036] One or more technical solutions provided by the embodiments of the present invention at least achieve the following technical effects or advantages:

[0037] The solution provided by the embodiment of the present invention divides the original hierarchical structure diagram to obtain m sliced hierarchical structure diagrams; extracts features from the m sliced hierarchical structure diagrams to obtain feature information of the m sliced hierarchical structure diagrams; performs feature fusion on the feature information of the m sliced hierarchical structure diagrams to obtain fusion information of the original hierarchical structure diagram; and performs restoration processing on the original hierarchical structure diagram according to the fusion information to obtain a corresponding restored hierarchical structure diagram. In the above solution, the present invention can divide the original hierarchical structure diagram into m sliced hierarchical structure diagrams, then extract and fuse features of the m sliced hierarchical structure diagrams to obtain fusion information of the original hierarchical structure diagram, and then perform restoration processing on the original hierarchical structure diagram based on the fusion information to obtain the final restored hierarchical structure diagram, so as to accurately and reliably achieve the restoration of the structure diagram, especially for large-scale original hierarchies, thereby improving the accuracy and convenience of the structure diagram restoration. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present invention, and for those of ordinary skill in the art, other drawings can be obtained according to the provided drawings without creative efforts.

[0039] Figure 1 It is a schematic flowchart of a structure diagram restoration method provided by an embodiment of the present invention.

[0040] Figure 2 It is a schematic diagram of partial feature information in a sliced hierarchical structure diagram provided by an embodiment of the present invention.

[0041] Figure 3 It is a schematic diagram of a reconstructed hierarchical structure provided by an embodiment of the present invention.

[0042] Figure 4 It is a schematic diagram of a reduction hierarchical structure diagram provided by an embodiment of the present invention.

[0043] Figure 5 It is a schematic structural diagram of a structure diagram reduction device provided by an embodiment of the present invention.

[0044] Figure 6 It is a schematic diagram of an electronic device provided by an embodiment of the present invention.

[0045] Figure 7 It is a schematic structural diagram of a server provided by an embodiment of the present invention. Detailed implementation manners

[0046] The present invention provides a structure diagram reduction method, device, electronic device and readable storage medium to solve the technical problem in the prior art that large-scale mind map reduction cannot be achieved due to the excessive size of images. The general idea is as follows:

[0047] The original hierarchical structure diagram is segmented to obtain m segmented hierarchical structure diagrams, where m is a positive integer; feature extraction is performed on the m segmented hierarchical structure diagrams to obtain the feature information of the m segmented hierarchical structure diagrams; feature fusion is performed on the feature information of the m segmented hierarchical structure diagrams to obtain the fusion information of the original hierarchical structure diagram; based on the fusion information, reduction processing is performed on the original hierarchical structure diagram to obtain the corresponding reduced hierarchical structure diagram.

[0048] Through the above technical solution, the present invention can segment the original hierarchical structure diagram into m segmented hierarchical structure diagrams, then perform feature extraction and fusion on the m segmented hierarchical structure diagrams to obtain the fusion information of the original hierarchical structure diagram, and further perform reduction processing on the original hierarchical structure diagram based on the fusion information to obtain the final reduced hierarchical structure diagram, so as to accurately and reliably achieve the reduction of the structure diagram, especially the large-scale original hierarchy, thereby improving the accuracy and convenience of the structure diagram reduction.

[0049] Please refer to Figure 1 , which is a schematic flowchart of a structure diagram reduction method provided by an embodiment of the present invention. The method can be applied to a terminal device, such as a smart phone or a tablet computer, and can also be applied to a server that has established data interaction with the terminal device, or can also be applied to a system composed of a terminal device and a server. The present invention does not make any limitations. As Figure 1 shown, the method includes the following steps:

[0050] S101. Segment the original hierarchical structure diagram to obtain m segmented hierarchical structure diagrams, where m is a positive integer.

[0051] The present invention can first obtain an original hierarchical structure diagram, and then perform segmentation processing on the original hierarchical structure diagram to obtain m segmented hierarchical structure diagrams. Among them, the original hierarchical structure diagram refers to a hierarchical structure diagram with a relatively large image size (for example, exceeding a preset first size), and can also be called a large-scale hierarchical structure diagram. The segmented hierarchical structure diagram is a hierarchical structure diagram with a relatively small image size (for example, the image size does not exceed a preset second size), and can also be called a small-scale hierarchical structure diagram. The preset first size and the preset second size are size thresholds set by the system itself, and they can be the same or different, which is not limited in the present invention.

[0052] The specific implementation manner of the segmentation / segmentation processing involved in the present invention can be overlapping segmentation or non-overlapping segmentation. The overlapping segmentation means that there is an overlapping area between two adjacent segmented hierarchical structure diagrams among the m segmented hierarchical structure diagrams obtained by segmentation. In other words, the present invention uses the overlapping segmentation method to segment the original hierarchical structure diagram, and can obtain m segmented hierarchical structure diagrams with overlapping areas. It should be noted that the overlapping segmentation ratio corresponding to the overlapping area is a ratio value set by the system itself, such as 20% or the like. The non-overlapping segmentation means that there is no overlap (or overlapping area) between any two of the m segmented hierarchical structure diagrams obtained by segmentation. In other words, the present invention uses the non-overlapping segmentation method to segment the original hierarchical structure diagram, and can obtain m non-overlapping segmented hierarchical structure diagrams.

[0053] For example, taking non-overlapping segmentation as an example, assuming that the size of the original hierarchical structure diagram is 5000*5000, since the conventional structure diagram restoration scheme only supports structure diagrams of ordinary size (1000*1000), the present invention can segment the 5000*5000 original hierarchical structure diagram into 25 1000*1000 segmented hierarchical structure diagrams, which can be respectively represented as segmented hierarchical structure Figure 1 ~ segmented hierarchical structure diagram 25.

[0054] The hierarchical structure diagram involved in the present invention refers to a structure diagram that can be used to represent hierarchical or level relationships. This structure diagram can clearly show the hierarchical relationships of each level, and may include, but is not limited to, mind maps, process structure diagrams, tree structure diagrams, or other custom structure diagrams with hierarchical or hierarchical relationships, etc.

[0055] S102. Extract features from the m segmented hierarchical structure diagrams to obtain the feature information of the m segmented hierarchical structure diagrams.

[0056] The characteristic information of the sliced hierarchical structure diagram described in the present invention includes, but is not limited to, the sliced text, multiple sliced hierarchical nodes, the sliced logical connection lines between the sliced hierarchical nodes, or other characteristic information of the custom sliced hierarchical structure diagram, etc. Taking one of the sliced hierarchical structure diagrams as an example, several specific implementation manners for extracting the characteristic information of each sliced hierarchical structure diagram are introduced below.

[0057] In one implementation manner, the present invention can use text recognition technology to perform text recognition on the sliced hierarchical structure diagram to obtain the sliced text in the sliced hierarchical structure diagram. The text recognition technology includes, but is not limited to, optical character recognition (OCR), geometric feature extraction technology, or other technologies for text or character recognition, etc. The sliced text refers to the text in the sliced hierarchical structure diagram, which can include custom text in any format, such as bmp image format text, jpg image format text, etc.

[0058] In another implementation manner, the present invention can use a pre-trained text recognition model to perform text recognition on the sliced hierarchical structure diagram to obtain the sliced text in the sliced hierarchical structure diagram. The text recognition model is a pre-trained model for recognizing text information in the hierarchical structure diagram, which can include, but is not limited to, a feedforward neural network model, a convolutional neural network model, a deep residual network model, a recurrent neural network model, a long short-term memory model, or other machine learning models, etc.

[0059] The sliced text involved in the present invention includes, but is not limited to, at least one sliced text line and the position information (also referred to as position coordinates) of each sliced text line. The sliced text line can refer to the text content in rows, or it can refer to the text content in rows and the text boxes owned by the text content. The present invention does not make a limitation. Optionally, in the following text of the present invention, the text line is taken as an example of the text content for the elaboration of relevant content, but it does not constitute a limitation.

[0060] In another implementation manner, the present invention can perform node extraction on the sliced hierarchical structure diagram to obtain i sliced hierarchical nodes in the sliced hierarchical structure diagram, where i is a positive integer. It should be noted that the number of sliced hierarchical nodes (i.e., i) each of the different sliced hierarchical structure diagrams has can be different or the same, which is specifically determined according to actual requirements. The present invention does not make a limitation.

[0061] In a specific implementation, the present invention can use a pre-trained key node detection model to detect and extract the hierarchical nodes in the sliced hierarchical structure diagram, so as to obtain i hierarchical nodes in the sliced hierarchical structure diagram. The number of the hierarchical nodes is not limited. Generally, the number of the hierarchical nodes in the sliced hierarchical structure diagram is multiple, that is, i is usually a positive integer greater than 1. The key node detection model includes but is not limited to a machine translation TensorFlow model, a finite element ANSYS extraction model, a feedforward neural network model, a convolutional neural network model, or other models for node extraction, etc.

[0062] In another embodiment, the present invention can perform connection extraction on the sliced hierarchical structure diagram to obtain the segmentation logic connections between the sliced hierarchical nodes in the sliced hierarchical structure diagram. It should be noted that the number of the segmentation logic connections included in different sliced hierarchical structure diagrams can be the same or different, which is specifically determined according to actual needs and is not limited in the present invention.

[0063] In a specific implementation, the present invention can use a pre-trained guiding line segmentation model to extract the segmentation logic connections between the hierarchical nodes in the sliced hierarchical structure diagram, so as to segment the segmentation logic connections from the image background of the sliced hierarchical structure diagram, thereby obtaining the segmentation logic connections between the hierarchical nodes in the sliced hierarchical structure diagram. The number of the segmentation logic connections is not limited and is usually multiple. The guiding line segmentation model includes but is not limited to a contour extraction model, a Gaussian line extraction model, a multi-source image line extraction model, or other models for extracting image connections, etc.

[0064] In practical applications, each of the segmentation logic connections is also composed of a series of feature points. Each feature point is correspondingly configured with a corresponding pixel classification, and the pixel classification is used to indicate whether the feature point is a point on the logic connection. Generally, the pixel classification refers to a binary classification at the pixel level, which can be specifically identified or represented by a specific character, a specific string, or a specific numerical value. For example, when "1" is used to represent the pixel classification of the feature point, it represents that the feature point is a point on the logic connection. On the contrary, when "0" is used to represent the pixel classification of the feature point, it represents that the feature point is an image background point, that is, it is not a point on the logic connection.

[0065] For example, please refer to Figure 2 a schematic diagram showing some feature information in a possible sliced hierarchical structure diagram. As Figure 2 shown, the partial feature information specifically includes: each hierarchical node in the sliced hierarchical structure diagram and the segmentation logic connections between the hierarchical nodes. For example, the label ① in the figure represents a hierarchical node, and the label ② in the figure represents a segmentation logic connection.

[0066] S103. Feature fusion is performed on the feature information of the m sliced hierarchical structure diagrams to obtain the fusion information of the original hierarchical structure diagram.

[0067] The present invention can perform feature fusion on the segmented text, i segmented hierarchical nodes, and the segmented logical connection lines between the segmented hierarchical nodes in each of the sliced hierarchical structure diagrams to obtain the fusion information of the original hierarchical structure diagram. Specifically, when there are overlapping regions between the sliced hierarchical structure diagrams, the present invention can specifically perform feature fusion and duplicate removal processing on the segmented text, i segmented hierarchical nodes, and the segmented logical connection lines between the segmented hierarchical nodes in each of the sliced hierarchical structure diagrams to obtain the fusion information. Among them, the fusion information of the original hierarchical structure diagram includes but is not limited to the original text of the original hierarchical structure diagram, n original hierarchical nodes, the original logical connection lines between the original hierarchical nodes, or other information, where n is a positive integer greater than i. Several specific implementation manners of step S103 are introduced below.

[0068] In one embodiment, the feature information of the sliced hierarchical structure diagram includes the segmented text in the sliced hierarchical structure diagram, and the fusion information of the original hierarchical structure diagram includes the original text in the original hierarchical structure diagram. At this time, the present invention can, according to the text position information of each segmented text in the m sliced hierarchical structure diagrams (specifically, it can refer to the text position coordinates of each segmented text line in the segmented text), and optionally in combination with the slicing order of the m sliced hierarchical structure diagrams, sequentially convert each segmented text to the original hierarchical structure diagram (specifically, it can be converted to the original coordinate system of the original hierarchical structure diagram), that is, convert the segmented text in all the sliced hierarchical structure diagrams to the same original coordinate system where the original hierarchical structure diagram is located, so as to obtain the converted text in the original hierarchical structure diagram.

[0069] When there are no overlapping regions between the sliced hierarchical structure diagrams, the present invention can directly use the converted text as the original text. On the contrary, when there are overlapping regions between the sliced hierarchical structure diagrams, the present invention can further use a preset text duplicate removal algorithm to perform duplicate removal processing on the converted text, so as to keep one copy of the converted text with the same text position coordinates and remove the redundant duplicate converted text, thereby obtaining the final original text. The text duplicate removal algorithm is an algorithm custom-set by the system for text duplicate removal, and it can include but is not limited to the hash simhash algorithm, the non-maximum suppression (NMS) duplicate removal algorithm, or other text duplicate removal algorithms, etc. Regarding the original text, reference can be made to the relevant introduction of the aforementioned segmented text, and details will not be elaborated here.

[0070] In another embodiment, the feature information of the sliced hierarchical structure diagram includes i sliced hierarchical nodes in the sliced hierarchical structure diagram, and the fusion information of the original hierarchical structure diagram includes n original hierarchical nodes in the original hierarchical structure diagram. At this time, the present invention can, according to the node position information (also referred to as node position coordinates) of each of the i sliced hierarchical nodes in the m sliced hierarchical structure diagrams, and optionally in combination with the slicing order of the m sliced hierarchical structure diagrams, sequentially convert each of the sliced hierarchical nodes into the original coordinate system of the original hierarchical structure diagram, that is, convert all the sliced hierarchical nodes in all the sliced hierarchical structure diagrams into the same original coordinate system where the original hierarchical structure diagram is located, so as to obtain j converted hierarchical nodes in the original hierarchical structure diagram, where j is a positive integer greater than i.

[0071] When there is no overlapping area between the sliced hierarchical structure diagrams, the present invention can directly use the j converted hierarchical nodes as the n original hierarchical nodes, and at this time n is equal to i. On the contrary, when there is an overlapping area between the sliced hierarchical structure diagrams, the present invention can further perform a duplicate removal process on the j converted hierarchical nodes, such as using a node data duplicate removal algorithm for duplicate removal, etc., to obtain the n original hierarchical nodes in the original hierarchical structure diagram, and at this time n is a positive integer less than j.

[0072] In another embodiment, the feature information of the sliced hierarchical structure diagram includes the sliced logical connection lines between the sliced hierarchical nodes in the sliced hierarchical structure diagram, and each sliced logical connection line includes multiple feature points; the fusion information of the original hierarchical structure diagram includes the original logical connection lines between the original hierarchical nodes in the original hierarchical structure. At this time, the present invention can perform feature point fusion and duplicate removal processing on each sliced logical connection line according to the pixel position information (i.e., pixel position coordinates) of each feature point that constitutes the sliced logical connection line in the m sliced hierarchical structure diagrams, so as to obtain the original logical connection line in the original hierarchical structure diagram. Specifically, the present invention can perform feature point fusion on a single sliced hierarchical structure diagram. The sliced hierarchical structure diagram includes multiple feature points at the pixel level, and each feature point is correspondingly configured with a pixel classification. The present invention can, according to the slicing order of the m sliced hierarchical structure diagrams, sequentially perform feature fusion according to the pixel classification of each feature point, and optionally perform duplicate removal processing, so as to obtain the original logical connection line in the original hierarchical structure diagram. It should be noted that the number of the original logical connection lines involved in the present invention is not limited, and generally the number of the original logical connection lines is multiple.

[0073] It should be noted that in practical applications, the above several embodiments can be implemented separately, or any one or several of the embodiments can be combined and implemented together, and the present invention does not make any limitations.

[0074] S104. According to the fusion information, perform a restoration process on the original hierarchical structure diagram to obtain a corresponding restored hierarchical structure diagram.

[0075] In a specific embodiment, the fusion information includes the original text in the original hierarchical structure diagram, n original hierarchical nodes, and the original logical connections between the original hierarchical nodes. The present invention can first perform a structure reconstruction on the n original hierarchical nodes in the original hierarchical structure diagram and the original logical connections between the original hierarchical nodes to obtain a reconstructed hierarchical structure. Specifically, the present invention can use a depth-first search algorithm based on point-line information to perform information fusion on the n original hierarchical nodes and the original logical connections between the original hierarchical nodes, and determine whether there is connectivity between any two of the original hierarchical nodes. If there is connectivity, connect the two original hierarchical nodes with connectivity through the corresponding original logical connection; otherwise, end the process, or then determine whether there is connectivity between other two original hierarchical nodes. Based on this principle, the reconstruction of the hierarchical information between nodes can be realized to obtain the reconstructed hierarchical structure.

[0076] For example, please refer to Figure 3 a schematic diagram showing a possible reconstructed hierarchical structure. Specifically, the present invention uses a depth-first search algorithm based on point-line information to perform a structure reconstruction on multiple original hierarchical nodes in the original hierarchical structure diagram and the original logical connections between the original hierarchical nodes to obtain Figure 3 the reconstructed hierarchical structure shown.

[0077] Next, the present invention can perform an associated matching between the original text and the reconstructed hierarchical structure to obtain the restored hierarchical structure diagram. In a specific embodiment, the original text includes at least one original text line and the position information (i.e., the text box position) of each original text line. Then, at this time, the present invention can add each original text to the corresponding position in the reconstructed hierarchical structure according to the position information of each original text line, such as the corresponding hierarchical node, etc., so as to obtain the restored hierarchical structure diagram. It can be understood that when the original text line includes the text content and the text box of the original text line, the present invention can specifically add the text content corresponding to each original text line to the corresponding position (such as a hierarchical node) in the reconstructed hierarchical structure, so as to realize the linkage matching between the text content and the hierarchical node to obtain the final restored hierarchical structure diagram.

[0078] In practical applications, the restored hierarchical structure diagram can be specifically stored in a file with a preset format, such as a json file, etc. The restored hierarchical structure diagram can also be displayed to the user in the form of a visual structure tree for viewing and editing.

[0079] For example, please refer to Figure 4 a schematic diagram showing a possible reduction hierarchical structure diagram. As Figure 4 shown in the reduction hierarchical structure diagram, it includes multiple editable text lines (which can also be referred to as text), multiple editable hierarchical nodes, and logical connections between the hierarchical nodes. The user can adaptively modify any information (such as hierarchical nodes, text, or logical connections, etc.) in the reduction hierarchical structure diagram according to their own needs.

[0080] In an alternative embodiment, after obtaining the reduction hierarchical structure diagram, the present invention can use some editing tools to perform information editing processing, that is, modification processing, on the reduction hierarchical structure diagram. Specifically, the present invention can receive an editing instruction from the user for the reduction hierarchical structure diagram, and the editing instruction is used to request modification processing of the target information in the reduction hierarchical structure diagram. The target information includes at least one of the following: any target text line, any target hierarchical node, and any target logical connection, etc. in the reduction hierarchical structure diagram. The quantities of the target text line, the target hierarchical node, and the target logical connection are not limited and can be determined according to the actual needs of the user.

[0081] Correspondingly, after receiving the editing instruction, the present invention can respond to the editing instruction and perform corresponding modification processing on the target information in the reduction hierarchical structure diagram, such as modifying the text line at a certain hierarchical node in the reduction hierarchical structure diagram, etc.

[0082] By implementing the embodiments of the present invention, the present invention slices the original hierarchical structure diagram to obtain m sliced hierarchical structure diagrams; extracts feature information of the m sliced hierarchical structure diagrams; performs feature fusion on the feature information of the m sliced hierarchical structure diagrams to obtain the fusion information of the original hierarchical structure diagram; and performs reduction processing on the original hierarchical structure diagram according to the fusion information to obtain the corresponding reduction hierarchical structure diagram. In the above solution, the present invention can slice the original hierarchical structure diagram into m sliced hierarchical structure diagrams, then extract and fuse the feature information of the m sliced hierarchical structure diagrams to obtain the fusion information of the original hierarchical structure diagram, and then perform reduction processing on the original hierarchical structure diagram based on the fusion information to obtain the final reduction hierarchical structure diagram, which can accurately and reliably implement the reduction of the structure diagram, especially for large-scale original hierarchies, thereby improving the accuracy and convenience of the structure diagram reduction.

[0083] Based on the same inventive concept, the embodiments of this specification also provide an apparatus and an electronic device corresponding to the structure diagram reduction method. Please refer to Figure 5 which is a schematic structural diagram of a structure diagram reduction apparatus provided by an embodiment of the present invention. As Figure 5The device shown includes: a splitting module 501, an extraction module 502, a fusion module 503, and a restoration module 504. Among them:

[0084] The splitting module 501 is used to split the original hierarchical structure diagram to obtain m split hierarchical structure diagrams, where m is a positive integer;

[0085] The extraction module 502 is used to extract features from the m split hierarchical structure diagrams to obtain the feature information of the m split hierarchical structure diagrams;

[0086] The fusion module 503 is used to perform feature fusion on the feature information of the m split hierarchical structure diagrams to obtain the fusion information of the original hierarchical structure diagram;

[0087] The restoration module 504 is used to perform a restoration process on the original hierarchical structure diagram according to the fusion information to obtain the corresponding restored hierarchical structure diagram.

[0088] Optionally, if there is an overlapping area between two of the m split hierarchical structure diagrams, the fusion module 503 is specifically used for:

[0089] Performing feature fusion and duplicate removal processing on the feature information of the m split hierarchical structure diagrams to obtain the fusion information of the original hierarchical structure diagram.

[0090] Optionally, the feature information includes the split text in the split hierarchical structure diagram, the fusion information includes the original text in the original hierarchical structure diagram, and the fusion module 503 is specifically used for:

[0091] According to the text position information of each split text in the m split hierarchical structure diagrams, converting each split text to the original hierarchical structure diagram to obtain the converted text in the original hierarchical structure diagram;

[0092] Performing duplicate removal processing on the converted text to obtain the original text.

[0093] Optionally, the feature information includes i split hierarchical nodes in the split hierarchical structure diagram, the fusion information includes n original hierarchical nodes in the original hierarchical structure diagram, and the fusion module 503 is specifically used for:

[0094] According to the node position information of each split hierarchical node in the m split hierarchical structure diagrams, converting each split hierarchical node to the original hierarchical structure diagram to obtain j converted hierarchical nodes in the original hierarchical structure diagram, where both i and j are positive integers, and j is greater than i;

[0095] Deduplicate the j conversion level nodes to obtain n original level nodes, where n is a positive integer less than j.

[0096] Optionally, the feature information includes the segmentation logic connections between the segmentation level nodes in the segmentation level structure diagram. The segmentation logic connections include multiple feature points. The fusion information includes the original logic connections between the original level nodes in the original level structure diagram. The fusion module 503 is specifically configured to:

[0097] Perform feature point fusion and deduplication on the segmentation logic connections between the segmentation level nodes in the m segmentation level structure diagrams according to the pixel position information of each feature point to obtain the original logic connections.

[0098] Optionally, the fusion information includes the original text, n original level nodes, and the original logic connections between the original level nodes in the original level structure diagram, where n is a positive integer. The restoration module 504 is used to:

[0099] Perform structure reconstruction on the n original level nodes and the original logic connections between the original level nodes in the original level structure diagram to obtain a reconstructed level structure;

[0100] Perform association matching on the original text and the reconstructed level structure to obtain the restored level structure diagram.

[0101] Optionally, the original text includes at least one original text line and the position information of each original text line. The restoration module 504 is specifically configured to:

[0102] Correspondingly add each original text line to the reconstructed level structure according to the position information of each original text line to obtain the restored level structure diagram.

[0103] Optionally, the device further includes a receiving module 505 and a processing module 506, where:

[0104] The receiving module 505 is configured to receive an editing instruction for the restored level structure diagram. The editing instruction is used to modify the target information in the restored level structure diagram. The target information includes at least one of the following: target text line, target level node, and target logic connection;

[0105] The processing module 506 is configured to respond to the editing instruction and perform corresponding modification processing on the target information in the restored level structure diagram.

[0106] It should be noted that the fusion module 503 in the device of the present invention includes a text recognition unit (which can also be referred to as an OCR recognition unit), a guide line segmentation unit, and a key node detection unit, which are not shown in the figure. The text recognition unit is used to recognize the text in the hierarchical structure diagram. For example, in the present invention, it recognizes the segmented text in each of the segmented hierarchical structure diagrams. The guide line segmentation unit is used to extract the logical connections between the hierarchical nodes in the hierarchical structure diagram. For example, in the present invention, a pre-trained guide line segmentation model is used to extract the segmented logical connections between the hierarchical nodes in the segmented hierarchical structure diagram. The key node detection unit is used to extract the hierarchical nodes in the hierarchical structure diagram. For example, in the present invention, a pre-trained key node detection model is used to detect and extract the hierarchical nodes in the segmented hierarchical structure diagram.

[0107] Regarding the device in the above embodiments, the specific manners in which each module or unit performs operations have been described in detail in the embodiments related to the method, and will not be elaborated herein.

[0108] Based on the same inventive concept, an embodiment of the present invention provides an electronic device 800. Figure 6 It is a block diagram of an electronic device 800 shown according to an exemplary embodiment. For example, the device 800 can be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0109] Referring to Figure 6 , the device 800 may include one or more of the following components: a processing component 802, a memory 804, a power component 806, a multimedia component 808, an audio component 810, an input / output (I / O) interface 812, a sensor component 814, and a communication component 816.

[0110] The processing component 802 generally controls the overall operation of the device 800, such as operations associated with display, telephone calls, data communication, camera operations, and recording operations. The processing element 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the above method. In addition, the processing component 802 may include one or more modules to facilitate the interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate the interaction between the multimedia component 808 and the processing component 802.

[0111] The memory 804 is configured to store various types of data to support the operation of the device 800. Examples of such data include instructions for any application or method operating on the device 800, contact data, phone book data, messages, pictures, videos, and the like. The memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0112] The power component 806 provides power to various components of the device 800. The power component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device 800.

[0113] The multimedia component 808 includes a screen that provides an output interface between the device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors can not only sense the boundaries of the touch or swipe actions, but also detect the duration and pressure associated with the touch or swipe operations. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the device 800 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each of the front camera and the rear camera can be a fixed optical lens system or have focal length and optical zoom capabilities.

[0114] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC) that is configured to receive external audio signals when the device 800 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals can be further stored in the memory 804 or transmitted via the communication component 816. In some embodiments, the audio component 810 further includes a speaker for outputting audio signals.

[0115] The I / O interface 812 provides an interface between the processing component 802 and a peripheral interface module, which can be a keyboard, a click wheel, buttons, etc. These buttons can include, but are not limited to: a home button, a volume button, a power button, and a lock button.

[0116] The sensor assembly 814 includes one or more sensors for providing a status assessment of various aspects of the device 800. For example, the sensor assembly 814 can detect the on / off state of the device 800, the relative positioning of components, such as the display and keypad of the device 800. The sensor assembly 814 can also detect a change in the position of the device 800 or a component of the device 800, the presence or absence of user contact with the device 800, the orientation or acceleration / deceleration of the device 800, and the temperature change of the device 800. The sensor assembly 814 can include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 814 can also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 can further include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0117] The communication component 816 is configured to facilitate communication between the device 800 and other devices in a wired or wireless manner. The device 800 can access a wireless network based on communication standards, such as WiFi, 2G, or 3G, or a combination thereof. In an exemplary embodiment, the communication component 816 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 further includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0118] In an exemplary embodiment, the device 800 can be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components for performing the above method.

[0119] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions, such as the memory 804 including instructions, is also provided. The above instructions can be executed by the processor 820 of the device 800 to complete the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device, etc.

[0120] Figure 7It is a schematic structural diagram of a server in some embodiments of the present invention. The server 1900 may vary greatly due to different configurations or performances, and may include one or more central processing units (CPUs) 1922 (for example, one or more processors) and a memory 1932, and one or more storage media 1930 (for example, one or more mass storage devices) for storing application programs 1942 or data 1944. Among them, the memory 1932 and the storage media 1930 may be transient storage or persistent storage. The programs stored in the storage media 1930 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations on the server. Further, the central processing unit 1922 may be configured to communicate with the storage media 1930 and execute a series of instruction operations in the storage media 1930 on the server 1900.

[0121] The server 1900 may further include one or more power supplies 1926, one or more wired or wireless network interfaces 1950, one or more input / output interfaces 1958, one or more keyboards 1956, and / or one or more operating systems 1941, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, and so on.

[0122] A non-transitory computer-readable storage medium, when the instructions in the storage medium are executed by a processor of a device (server or terminal), enables the device to execute the structure diagram restoration method of the foregoing embodiments.

[0123] A non-transitory computer-readable storage medium, when the instructions in the storage medium are executed by a processor of a device (server or terminal), enables the computer device to execute the foregoing Figure 1 description of the structure diagram restoration method in the corresponding embodiments. Therefore, it will not be elaborated here. In addition, the description of the beneficial effects of adopting the same method will not be elaborated either.

[0124] In addition, it should be noted that: The embodiments of the present application also provide a computer program product or a computer program for the technical details not disclosed in the embodiments of the computer program product or the computer program involved in the present application. Please refer to the description of the method embodiments of the present application. The computer program product or the computer program may include computer instructions, and the computer instructions may be stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor may execute the computer instructions, enabling the computer device to execute the foregoing Figure 1The description of the structure diagram restoration method in the corresponding embodiment will not be elaborated here. In addition, the description of the beneficial effects of using the same method will not be elaborated either.

[0125] One or more technical solutions provided by the embodiments of the present invention at least achieve the following technical effects or advantages: By splitting the original hierarchical structure diagram, the present invention obtains m split hierarchical structure diagrams; extracts features from the m split hierarchical structure diagrams to obtain the feature information of the m split hierarchical structure diagrams; performs feature fusion on the feature information of the m split hierarchical structure diagrams to obtain the fusion information of the original hierarchical structure diagram; and performs restoration processing on the original hierarchical structure diagram according to the fusion information to obtain the corresponding restored hierarchical structure diagram. In the above solution, the present invention can split the original hierarchical structure diagram into m split hierarchical structure diagrams, then extract and fuse features from the m split hierarchical structure diagrams to obtain the fusion information of the original hierarchical structure diagram, and further perform restoration processing on the original hierarchical structure diagram based on this fusion information to obtain the final restored hierarchical structure diagram, which can accurately and reliably realize the restoration of the structure diagram, especially the large-scale original hierarchy, thus improving the accuracy and convenience of the structure diagram restoration.

[0126] Those skilled in the art will readily think of other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses, or adaptations of the present invention that follow the general principles of the present invention and include common general knowledge or conventional technical means in the technical field not disclosed in this disclosure. The specification and examples are only regarded as exemplary, and the true scope and spirit of the present invention are pointed out by the following claims.

[0127] It should be understood that the present invention is not limited to the exact structures already described and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present invention is only limited by the appended claims. The above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for restoring a structure diagram, characterized in that, Including: Segmenting the original hierarchical structure diagram to obtain m segmented hierarchical structure diagrams, where m is a positive integer; Performing feature extraction on the m segmented hierarchical structure diagrams to obtain the feature information of the m segmented hierarchical structure diagrams; Performing feature fusion on the feature information of the m segmented hierarchical structure diagrams to obtain the fusion information of the original hierarchical structure diagram; Performing reduction processing on the original hierarchical structure diagram according to the fusion information to obtain the corresponding reduced hierarchical structure diagram; Wherein, the feature information includes at least two of the segmented text, i segmented hierarchical nodes, and segmented logical connections in the segmented hierarchical structure diagram. When the feature information includes the segmented text, the fusion information includes the original text in the original hierarchical structure diagram; when the feature information includes the i segmented hierarchical nodes, the fusion information includes n original hierarchical nodes in the original hierarchical structure diagram; when the feature information includes the segmented logical connection, the segmented logical connection includes multiple feature points, and the fusion information includes the original logical connection between the original hierarchical nodes in the original hierarchical structure diagram. Both i and n are positive integers.

2. The method according to claim 1, characterized in that If there is an overlapping area between any two of the m segmented hierarchical structure diagrams, then the performing feature fusion on the feature information of the m segmented hierarchical structure diagrams to obtain the fusion information of the original hierarchical structure diagram includes: Performing feature fusion and duplicate removal processing on the feature information of the m segmented hierarchical structure diagrams to obtain the fusion information of the original hierarchical structure diagram.

3. The method according to claim 1 or 2, characterized in that, The performing feature fusion processing on the feature information of the m segmented hierarchical structure diagrams to obtain the fusion information of the original hierarchical structure diagram includes: According to the text position information of each segmented text in the m segmented hierarchical structure diagrams, converting each segmented text to the original hierarchical structure diagram to obtain the converted text in the original hierarchical structure diagram; Performing duplicate removal processing on the converted text to obtain the original text.

4. The method according to claim 1 or 2, characterized in that, The performing feature fusion processing on the feature information of the m segmented hierarchical structure diagrams to obtain the fusion information of the original hierarchical structure diagram includes: According to the node position information of each segmented hierarchical node in the m segmented hierarchical structure diagrams, converting each segmented hierarchical node to the original hierarchical structure diagram to obtain j converted hierarchical nodes in the original hierarchical structure diagram, where j is a positive integer and j is greater than i; Performing duplicate removal processing on the j converted hierarchical nodes to obtain n original hierarchical nodes, where n is a positive integer less than j.

5. The method according to claim 1 or 2, characterized in that, The performing feature fusion processing on the feature information of the m segmented hierarchical structure diagrams to obtain the fusion information of the original hierarchical structure diagram includes: According to the pixel position information of each feature point, performing feature point fusion and duplicate removal processing on the segmented logical connections between the segmented hierarchical nodes in the m segmented hierarchical structure diagrams to obtain the original logical connection.

6. The method according to claim 1, characterized in that The fusion information includes the original text, n original hierarchical nodes, and the original logical connection between the original hierarchical nodes in the original hierarchical structure diagram; Restoring the original hierarchical structure diagram according to the fusion information to obtain a corresponding restored hierarchical structure diagram includes: Reconstructing the structure of n original hierarchical nodes and the original logical connections between the original hierarchical nodes in the original hierarchical structure diagram to obtain a reconstructed hierarchical structure; Associating and matching the original text with the reconstructed hierarchical structure to obtain the restored hierarchical structure diagram.

7. The method according to claim 6, wherein The original text includes at least one original text line and the position information of each original text line. Associating and matching the original text with the reconstructed hierarchical structure to obtain the restored hierarchical structure diagram includes: Adding each original text line to the reconstructed hierarchical structure according to the position information of each original text line to obtain the restored hierarchical structure diagram.

8. The method according to claim 7, wherein The method includes: Receiving an editing instruction for the restored hierarchical structure diagram, where the editing instruction is used to modify target information in the restored hierarchical structure diagram, and the target information includes at least one of the following: a target text line, a target hierarchical node, and a target logical connection; Responding to the editing instruction and performing corresponding modification processing on the target information in the restored hierarchical structure diagram.

9. A structure diagram restoration device, characterized in that, Includes: A splitting module for splitting the original hierarchical structure diagram to obtain m split hierarchical structure diagrams, where m is a positive integer; An extraction module for extracting feature information of the m split hierarchical structure diagrams to obtain the feature information of the m split hierarchical structure diagrams; A fusion module for fusing the feature information of the m split hierarchical structure diagrams to obtain the fusion information of the original hierarchical structure diagram; A restoration module for restoring the original hierarchical structure diagram according to the fusion information to obtain a corresponding restored hierarchical structure diagram; Wherein, the feature information includes at least two of the split text, i split hierarchical nodes, and split logical connections in the split hierarchical structure diagram. When the feature information includes the split text, the fusion information includes the original text in the original hierarchical structure diagram; when the feature information includes the i split hierarchical nodes, the fusion information includes the n original hierarchical nodes in the original hierarchical structure diagram; when the feature information includes the split logical connection, the split logical connection includes multiple feature points, and the fusion information includes the original logical connection between the original hierarchical nodes in the original hierarchical structure diagram. Both i and n are positive integers.

10. The device according to claim 9, characterized in that, If there is an overlapping area between any two of the m split hierarchical structure diagrams, the fusion module is specifically used for: Performing feature fusion and duplicate removal processing on the feature information of the m split hierarchical structure diagrams to obtain the fusion information of the original hierarchical structure diagram.

11. The device according to claim 9 or 10, characterized in that, The fusion module is specifically used for: Converting each split text to the original hierarchical structure diagram according to the text position information of each split text in the m split hierarchical structure diagrams to obtain the converted text in the original hierarchical structure diagram; Performing duplicate removal processing on the converted text to obtain the original text.

12. The device according to claim 9 or 10, characterized in that, The fusion module is specifically used for: According to the node position information of each of the m sliced hierarchical structure diagrams, each of the sliced hierarchical nodes is converted into the original hierarchical structure diagram to obtain j converted hierarchical nodes in the original hierarchical structure diagram, where j is a positive integer and j is greater than i; The j converted hierarchical nodes are processed to remove duplicates to obtain n original hierarchical nodes, where n is a positive integer less than j.

13. An electronic device, characterized in that, It includes a memory and one or more programs, where the one or more programs are stored in the memory and are configured to be executed by one or more processors to perform the operation instructions corresponding to the method according to any one of claims 1 to 8.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, the steps corresponding to the method according to any one of claims 1 to 8 are implemented.

15. A computer program product, characterized in that, The computer program product includes computer instructions, which are stored in a computer-readable storage medium and are adapted to be read and executed by a processor so that a computer device having the processor executes the method according to any one of claims 1-8.

Images