Structural image restoration method, device, electronic device, medium and program product

By identifying and classifying the text and logical connections in the hierarchical structure pictures, the restoration process is solved, and the problem that mind maps cannot be restored after image transposition and transformation is achieved, and efficient and accurate structural image restoration is achieved.

CN114154464BActive Publication Date: 2024-11-22BEIJING SOGOU TECHNOLOGY DEVELOPMENT CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202111275846.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-10-29
Publication Date
2024-11-22
Estimated Expiration
2041-10-29

AI Technical Summary

Technical Problem

The prior art cannot correctly realize the restoration of mind maps, especially after image transposition transformation, resulting in a decrease in accuracy and reliability of map restore.

Method used

By obtaining the transform hierarchical structure pictures, performing text recognition and image transformation classification, restoring the process according to the target transformation classification, reconstructing the hierarchical structure and logical connections, and generating a restore hierarchical structure picture that supports editable.

Benefits of technology

It improves the efficiency, accuracy and speed of structural image restoration, realizes accurate restoration of mind maps after image transposition and transformation, and improves the reliability of maps.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114154464B_ABST
    Figure CN114154464B_ABST
Patent Text Reader

Abstract

The present invention discloses a structural image restoration method, device, electronic device, medium and program product, the method comprising: obtaining a transformed hierarchical structural image; performing text recognition on the transformed hierarchical structural image to obtain recognized text in the transformed hierarchical structural image; performing image transformation classification on the recognized text to obtain a target transformation classification of the transformed hierarchical structural image; performing restoration processing on the transformed hierarchical structural image according to the target transformation classification to obtain a corresponding restored hierarchical structural image. The above scheme can improve the efficiency, accuracy and speed of structural image restoration.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a structural image restoration method, device, electronic equipment, medium and program product. Background Art

[0002] Hierarchical structure images are clearly nested structure images, which often appear in users' daily work and study life. Among them, mind mapping, for example, is an effective graphic thinking tool for expressing divergent thinking. In order to facilitate file transfer and user reading, mind maps usually exist in the form of images. However, when mind maps are stored and transmitted in image format, there is a big flaw - they can only support viewing but not re-editing.

[0003] Therefore, it is urgent to propose an efficient and reliable mind map restoration solution. Summary of the invention

[0004] The embodiments of the present invention solve the technical problem in the prior art that a mind map cannot be correctly restored after image transposition transformation by providing a structured image restoration method, device, electronic device and readable storage medium.

[0005] In a first aspect, an embodiment of the present invention provides a method for restoring a structured image, comprising:

[0006] Obtaining a transformed hierarchical structure picture, wherein the transformed hierarchical structure picture is a hierarchical structure picture after image transformation processing;

[0007] Performing text recognition on the transformed hierarchical structure picture to obtain recognized text in the transformed hierarchical structure picture;

[0008] Performing image transformation classification on the recognized text to obtain a target transformation classification of the transformation hierarchical structure picture;

[0009] The transformed hierarchical structure picture is restored according to the target transformation classification to obtain a corresponding restored hierarchical structure picture.

[0010] Optionally, when the transformed hierarchical structure picture is a mirrored hierarchical structure picture after image mirroring transformation, the recognized text in the transformed hierarchical structure picture is a mirrored text, and the target transformation classification is a target mirrored classification;

[0011] The restoring processing of the transformed hierarchical structure picture according to the target transformation classification to obtain the corresponding restored hierarchical structure picture includes:

[0012] According to the target image classification, the image hierarchical structure image is mirror-flipped accordingly to obtain a flipped hierarchical structure image, wherein the flipped hierarchical structure image includes a flipped text corresponding to the image text, a plurality of hierarchical nodes, and logical connections between the hierarchical nodes;

[0013] Reconstructing the structure of the plurality of hierarchical nodes and the logical connections between the hierarchical nodes in the flipped hierarchical structure image to obtain a reconstructed hierarchical structure;

[0014] The flipped text and the reconstructed hierarchical structure are associated and matched to obtain the restored hierarchical structure image.

[0015] Optionally, when the transformed hierarchical structure picture is a transposed hierarchical structure picture after image transposition transformation, the recognized text in the transformed hierarchical structure picture is a transposed text, and the target transformation classification is a target transposed classification; performing restoration processing on the transformed hierarchical structure picture according to the target transformation classification to obtain a corresponding restored hierarchical structure picture includes:

[0016] According to the target transposition classification, the transposed hierarchical structure image is subjected to inverse transposition processing to obtain an inverse transposition structure image, wherein the inverse transposition structure image includes an inverse transposed text corresponding to the transposed text, a plurality of hierarchical nodes, and logical connections between the hierarchical nodes;

[0017] Reconstructing the structure of the plurality of hierarchical nodes and the logical connections between the hierarchical nodes in the inverse transposed structure picture to obtain a reconstructed hierarchical structure;

[0018] The reverse transposed text and the reconstructed hierarchical structure are associated and matched to obtain the restored hierarchical structure image.

[0019] Optionally, the structural reconstruction of the plurality of hierarchical nodes and the logical connections between the hierarchical nodes in the inverse transposed structure picture to obtain the reconstructed hierarchical structure includes:

[0020] Performing node extraction and line extraction on the inverse transposed structure image to obtain the multiple hierarchical nodes in the inverse transposed structure image and the logical lines between the hierarchical nodes;

[0021] The plurality of hierarchical nodes and the logical connections between the hierarchical nodes are structurally reconstructed to obtain the reconstructed hierarchical structure.

[0022] Optionally, the reverse transposed text includes at least one reverse transposed text line and the position of each reverse transposed text line, and associating and matching the reverse transposed text with the reconstructed hierarchical structure to obtain the restored hierarchical structure image includes:

[0023] According to the position of each of the reverse transposed text lines, each of the reverse transposed text lines is added to the reconstructed hierarchical structure to obtain the restored hierarchical structure image.

[0024] Optionally, the method comprises:

[0025] receiving an editing instruction for the restored hierarchical structure picture, the editing instruction being used to edit target information in the restored hierarchical structure picture, the target information comprising at least one of the following: a target text line, a target hierarchical node, and a target logical connection;

[0026] In response to the editing instruction, corresponding editing processing is performed on the target information in the restored hierarchical structure picture.

[0027] Optionally, the restored hierarchical structure picture includes an editable reconstructed hierarchical structure picture, and the method further includes:

[0028] receiving an editing instruction for the reconstructed hierarchical structure diagram, wherein the editing instruction is used to edit the reconstructed hierarchical structure in the reconstructed hierarchical structure diagram;

[0029] In response to the editing instruction, corresponding editing processing is performed on the reconstructed hierarchical structure picture in the restored hierarchical structure picture.

[0030] Optionally, the transposed text includes at least one transposed text line, and performing image transposition classification on the transposed text to obtain a target transposition classification of the transposed hierarchical structure image includes:

[0031] Performing image transposition classification on each of the transposed text lines to obtain a text transposition classification of each of the transposed text lines;

[0032] When the number of transposed text lines of a preset transposition category exceeds a preset threshold, the preset transposition category is determined as a target transposition category for the transposed hierarchical structure picture.

[0033] In a second aspect, an embodiment of the present invention provides a structure restoration device, comprising:

[0034] An acquisition module, used to acquire a transformed hierarchical structure picture, wherein the transformed hierarchical structure picture is a hierarchical structure picture after image transformation processing;

[0035] A recognition module, used to perform text recognition on the transformed hierarchical structure picture to obtain recognized text in the transformed hierarchical structure picture;

[0036] A classification module, used for performing image transformation classification on the recognized text to obtain a target transformation classification of the transformation hierarchical structure image;

[0037] The restoration module is used to restore the transformed hierarchical structure picture according to the target transformation classification to obtain a corresponding restored hierarchical structure picture.

[0038] Optionally, when the transformed hierarchical structure picture is a transposed hierarchical structure picture after image transposition transformation, the recognized text in the transformed hierarchical structure picture is a transposed text, and the target transformation classification is a target transposed classification; the restoration module is specifically used for:

[0039] According to the target transposition classification, the transposed hierarchical structure image is subjected to inverse transposition processing to obtain an inverse transposition structure image, wherein the inverse transposition structure image includes an inverse transposed text corresponding to the transposed text, a plurality of hierarchical nodes, and logical connections between the hierarchical nodes;

[0040] Reconstructing the structure of the plurality of hierarchical nodes and the logical connections between the hierarchical nodes in the inverse transposed structure picture to obtain a reconstructed hierarchical structure;

[0041] The reverse transposed text and the reconstructed hierarchical structure are associated and matched to obtain the restored hierarchical structure image.

[0042] Optionally, the restoration module is specifically used for:

[0043] Performing node extraction and line extraction on the inverse transposed structure image to obtain the multiple hierarchical nodes in the inverse transposed structure image and the logical lines between the hierarchical nodes;

[0044] The plurality of hierarchical nodes and the logical connections between the hierarchical nodes are structurally reconstructed to obtain the reconstructed hierarchical structure.

[0045] Optionally, the reverse transposed text includes at least one reverse transposed text line and the position of each reverse transposed text line, and the restoration module is specifically used to:

[0046] According to the position of each of the reverse transposed text lines, each of the reverse transposed text lines is added to the reconstructed hierarchical structure to obtain the restored hierarchical structure image.

[0047] Optionally, the device further comprises:

[0048] A receiving module, configured to receive an editing instruction for the restored hierarchical structure image, wherein the editing instruction is used to edit target information in the restored hierarchical structure image, wherein the target information includes at least one of the following: a target text line, a target hierarchical node, and a target logical connection;

[0049] The processing module is used to respond to the editing instruction and perform corresponding editing processing on the target information in the restored hierarchical structure picture.

[0050] Optionally, the transposed text includes at least one transposed text line, and the classification module is specifically used for:

[0051] Performing image transposition classification on each of the transposed text lines to obtain a text transposition classification of each of the transposed text lines;

[0052] When the number of transposed text lines of a preset transposition category exceeds a preset threshold, the preset transposition category is determined as a target transposition category for the transposed hierarchical structure picture.

[0053] For the contents not introduced or described in the embodiments of the present application, please refer to the relevant introduction in the aforementioned method embodiments, which will not be repeated here.

[0054] In a third aspect, an embodiment of the present invention provides an electronic device comprising a memory and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by one or more processors to execute operation instructions corresponding to the one or more programs for performing the structural image restoration method provided in the first aspect.

[0055] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps corresponding to the structural image restoration method provided in the first aspect.

[0056] One or more technical solutions provided by the embodiments of the present invention achieve at least the following technical effects or advantages:

[0057] The solution provided by the embodiment of the present invention obtains a transformation hierarchical structure picture, performs text recognition on the transformation hierarchical structure picture to obtain the recognized text in the transformation hierarchical structure picture, then performs transformation classification on the recognized text to obtain the target transformation classification of the transformation hierarchical structure picture, and finally performs restoration processing on the transformation hierarchical structure picture according to the target transformation classification to obtain the restored hierarchical structure picture corresponding to the transformation hierarchical structure picture. In the above scheme, the target transformation classification of the entire transformation hierarchical structure picture can be identified based on the recognized text in the transformation hierarchical structure picture, and then the transformation hierarchical structure picture is restored according to the target transformation classification to obtain the final editable restored hierarchical structure picture, which can efficiently and accurately realize the rapid restoration of the transformation hierarchical structure picture, thereby improving the efficiency, accuracy and quickness of structural picture restoration. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0059] Figure 1 It is a flow chart of a structural image restoration method provided by an embodiment of the present invention.

[0060] Figure 2 It is a flow chart of a structural image restoration method provided by an embodiment of the present invention.

[0061] Figure 3 It is a schematic diagram of a transposed hierarchical structure picture provided by an embodiment of the present invention.

[0062] Figure 4 It is a schematic diagram of an inverse transposed structure picture provided by an embodiment of the present invention.

[0063] Figure 5 It is a schematic diagram of hierarchical structure information in a reverse transposed structure picture provided by an embodiment of the present invention.

[0064] Figure 6 It is a schematic diagram of a reconstruction hierarchical structure provided by an embodiment of the present invention.

[0065] Figure 7 It is a structural schematic diagram of a structural image restoration device provided by an embodiment of the present invention.

[0066] Figure 8 It is a schematic diagram of the structure of another structural image restoration device provided by an embodiment of the present invention.

[0067] Fig. 9 It is a schematic diagram of an electronic device provided by an embodiment of the present invention.

[0068] Fig.10 It is a structural diagram of a server provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0069] The present invention provides a structured image restoration method, device, electronic device and readable storage medium to solve the technical problem that the mind map restoration cannot be achieved in the prior art. The overall idea is as follows:

[0070] Acquire a transformed hierarchical structure picture, wherein the transformed hierarchical structure picture is a hierarchical structure picture that has been processed by image transformation; perform text recognition on the transformed hierarchical structure picture to obtain recognized text in the transformed hierarchical structure picture; perform image transformation classification on the recognized text to obtain a target transformation classification of the transformed hierarchical structure picture; perform restoration processing on the transformed hierarchical structure picture according to the target transformation classification to obtain a corresponding restored hierarchical structure picture.

[0071] Through the above technical scheme, the target transformation classification of the entire transformation hierarchical structure picture can be identified based on the recognition text in the transformation hierarchical structure picture, and then the transformation hierarchical structure picture can be restored according to the target transformation classification to obtain the final editable restored hierarchical structure picture. This can efficiently and accurately realize the rapid restoration of the transformation hierarchical structure picture, thereby improving the efficiency, accuracy and speed of structural picture restoration.

[0072] Please refer to Figure 1 , is a flow chart of a structural image restoration method provided by an embodiment of the present invention. The method can be applied to a terminal device, such as a smart phone, a tablet computer, or a server that has established data interaction with the terminal device, or a system composed of a terminal device and a server, which is not limited by the present invention. Figure 1 The method shown comprises the following steps:

[0073] S101. Obtain a transformed hierarchical structure picture, where the transformed hierarchical structure picture is a hierarchical structure picture that has undergone image transformation processing.

[0074] The transformed hierarchical structure picture described in the present invention refers to a hierarchical structure picture that has been processed by image transformation, such as image transposition transformation, image mirror transformation, etc. In a possible implementation, the transformed hierarchical structure picture may be a transposed hierarchical structure picture that has been transformed by image transposition; or, the transformed hierarchical structure picture may be a mirrored hierarchical structure picture that has been transformed by image mirror transformation, or may also be a hierarchical structure picture that has been processed by other image transformations, which is not limited in the present invention.

[0075] S102: Perform text recognition on the transformed hierarchical structure picture to obtain recognized text in the transformed hierarchical structure picture.

[0076] In a possible implementation manner, the transformed hierarchical structure picture is a transposed hierarchical structure picture. The present invention can perform text recognition on the transposed hierarchical structure picture to obtain the transposed text in the transposed hierarchical structure picture.

[0077] In another possible implementation, the transformed hierarchical structure picture is a mirrored hierarchical structure picture. The present invention can perform text recognition on the mirrored hierarchical structure picture to obtain the mirrored text in the mirrored hierarchical structure picture.

[0078] S103, performing image transformation classification on the recognized text to obtain a target transformation classification of the transformation hierarchical structure image.

[0079] In a possible implementation manner, the recognized text is the transposed text in the transposed hierarchical structure image. The present invention can perform image transposition classification on the transposed text to obtain a target transposition classification of the transposed hierarchical structure image.

[0080] In another possible implementation, the recognized text is a mirror text in the mirror hierarchical structure picture. The present invention can perform image mirror classification on the mirror text to obtain a target mirror classification of the mirror hierarchical structure picture.

[0081] S104. Perform restoration processing on the transformed hierarchical structure picture according to the target transformation classification to obtain a corresponding restored hierarchical structure picture.

[0082] In a possible implementation, the target transformation classification is a target transposition classification. The present invention can restore the transposed hierarchical structure picture according to the target transposition classification to obtain a corresponding restored hierarchical structure picture.

[0083] In another possible implementation, the target transformation classification is a target image classification. The present invention can restore the image hierarchy diagram according to the target image classification to obtain a corresponding restored hierarchy diagram.

[0084] It should be noted that the specific implementation methods of the present invention for restoring the transposed hierarchical structure image and the mirrored hierarchical structure image are very similar, and can be implemented with reference to each other. The present invention hereinafter takes the transformed hierarchical structure image as a transposed hierarchical structure image as an example to introduce the overall implementation process for restoring the transposed hierarchical structure image, but it does not constitute a limitation.

[0085] Please also see Figure 2 , is a flow chart of another structural image restoration method provided by an embodiment of the present application. Figure 2 The method shown includes the following implementation steps:

[0086] S201, obtaining a transposed hierarchical structure image.

[0087] The transposed hierarchical structure picture of the present invention is a hierarchical structure picture after image transposition transformation, and specifically refers to a hierarchical structure picture obtained by performing an arbitrary angle transposition / rotation process on the original hierarchical structure picture, such as performing an image transposition operation of 90°, 180° or 270°, etc. The original hierarchical structure picture refers to a conventional hierarchical structure picture that has not undergone any image transformation processing. For example, see Figure 3 A schematic diagram showing a possible transposed hierarchical structure picture is shown. Figure 3 The transposed hierarchical structure image shown is a hierarchical structure image after 90° transposition processing.

[0088] The hierarchical structure image involved in the present invention refers to a structural image that can be used to represent hierarchical or level relationships. The structural image can clearly show the hierarchical relationships of each level, which may include but is not limited to mind maps, process structure images, tree structure images, or other customized structural images with hierarchical or level relationships.

[0089] S202: Perform text recognition on the transposed hierarchical structure image to obtain transposed text in the transposed hierarchical structure image.

[0090] In one embodiment, the present invention can use text recognition technology to perform text recognition on the transposed hierarchical structure image to obtain the transposed text in the transposed hierarchical structure image. The text recognition technology includes but is not limited to optical character recognition (OCR), geometric feature extraction technology, or other technologies for text or word recognition. The transposed text refers to the text after the image transposition process, which may include custom text in any format, such as bmp image format text, jpg image format text, etc.

[0091] In another embodiment, the present invention can use a pre-trained text recognition model to perform text recognition on the transposed hierarchical structure image to obtain the transposed text in the transposed hierarchical structure image. The text recognition model is pre-trained for identifying text information in a hierarchical structure image, which may include but is not limited to a feedforward neural network model, a convolutional neural network model, a deep residual network model, a recurrent neural network model, a long short-term memory model, or other machine learning models.

[0092] The transposed text involved in the present invention includes but is not limited to at least one transposed text line and the position (also referred to as position coordinates) of each transposed text line. The transposed text line may refer to lines of text content, or may refer to lines of text content and text boxes owned by the text content, which is not limited in the present invention. Optionally, the present invention hereinafter takes the text line text content as an example to explain the relevant content, but does not constitute a limitation.

[0093] S203, performing image transposition classification on the transposed text to obtain a target transposition classification of the transposed hierarchical structure image.

[0094] In one embodiment, the present invention may use a pre-built variable-length text transposition classifier to perform image transposition classification on the transposed text to determine / obtain a target transposition classification of the transposed hierarchical structure image.

[0095] In another embodiment, the present invention can use a pre-trained transposition classification model to perform image transposition classification on the transposed text to obtain the target transposition classification of the transposed hierarchical structure image. The transposition classification model is a pre-trained model for text transposition classification, which may include but is not limited to a sequence to sequence (sequence to sequence, seq2seq) model, a bidirectional encoding representation model (encoder representations from transformers, BERT), a word to vector (Word2Vec) model, a text neural network model, or other models for text transposition classification.

[0096] In a specific embodiment, the transposed text includes at least one transposed text line. In the image transposition classification process, the present invention can specifically classify each transposed text line in the transposed text to obtain the text transposition classification of each transposed text line. When the number of transposed text lines corresponding to the preset transposition classification of the text transposition classification exceeds a preset threshold, it can be considered that the text transposition classification of most of the transposed text lines in the transposed text is the preset transposition classification, and then the preset transposition classification can be determined as the target transposition classification of the transposed hierarchical structure image. Otherwise, the process can be terminated.

[0097] The preset threshold is a threshold set by the system, which can be set according to the actual needs of the system, or can be an experience value set according to user experience, etc., which is not limited by the present invention. The preset transposition classification, the text transposition classification and the target transposition classification involved in the present invention are all pre-configured classifications in the transposition classifier or the transposition classification model, which may include but are not limited to normal transposition classification, 90° transposition classification, 180° transposition classification, 270° transposition classification, or other arbitrary or custom angle transposition classifications, etc.

[0098] S204 . Perform restoration processing on the transposed hierarchical structure picture according to the target transposition classification to obtain a corresponding restored hierarchical structure picture.

[0099] In a specific embodiment, the present invention can first perform an inverse transposition process corresponding to the target transposition classification on the transposed hierarchical structure picture according to the target transposition classification to obtain a corresponding inverse transposition structure picture, wherein the inverse transposition structure picture includes an inverse transposition text corresponding to the transposed text, a plurality of hierarchical nodes, and logical connections between the hierarchical nodes. The inverse transposition text specifically refers to the text obtained after the transposed text in the transposed hierarchical structure picture is subjected to a corresponding inverse transposition process according to the target transposition classification.

[0100] Specifically, taking the target transposition classification including four transposition classifications as an example, it specifically includes a normal transposition classification, a 90° transposition classification, a 180° transposition classification, and a 270° transposition classification. When the target transposition classification is a normal transposition classification, the present invention does not need to perform a reverse transposition process on the transposed hierarchical structure picture according to the indication of the normal transposition classification. At this time, the transposed hierarchical structure picture is regarded as an original hierarchical structure picture that has not undergone image transformation, and the subsequent process can be continued.

[0101] When the target transposition classification is a 90° transposition classification, the present invention can perform corresponding 90° inverse transposition processing on the transposed hierarchical structure picture according to the indication of the 90° transposition classification to obtain a corresponding inverse transposition hierarchical structure picture.

[0102] Correspondingly, when the target transposition classification is a 180° transposition classification, the present invention can perform corresponding 180° inverse transposition processing on the transposed hierarchical structure picture according to the indication of the 180° transposition classification to obtain a corresponding inverse transposition hierarchical structure picture.

[0103] Correspondingly, when the target transposition classification is a 270° transposition classification, the present invention can perform corresponding 270° inverse transposition processing on the transposed hierarchical structure picture according to the indication of the 270° transposition classification to obtain a corresponding inverse transposition hierarchical structure picture.

[0104] For example, citing Figure 3 The example of the transposed hierarchical structure picture shown in the figure, the present invention is based on the target transposition classification ( Figure 3 That is, 90° transposition classification) After the transposed hierarchical structure image is subjected to the corresponding 90° reverse transposition processing, the following Figure 4 The reverse transposed hierarchy picture shown.

[0105] Next, after obtaining the inverse transposed structure picture, the present invention can perform structural reconstruction on the inverse transposed structure picture to obtain a corresponding reconstructed hierarchical structure. Specifically, the present invention can perform structural information extraction (specifically, node extraction and line extraction) on the inverse transposed structure picture to obtain hierarchical structure information in the inverse transposed structure picture, and the hierarchical structure information includes multiple hierarchical nodes in the inverse transposed structure picture and logical connections between the hierarchical nodes. Then, the multiple hierarchical nodes and the logical connections between the hierarchical nodes are structurally reconstructed to obtain a corresponding reconstructed hierarchical structure.

[0106] In a specific implementation, the present invention can use a pre-trained guide line segmentation model to extract the logical connections between the hierarchical nodes in the inverse transposed structure image, so as to segment the logical connections from the image background of the inverse transposed structure image, thereby obtaining the logical connections between the hierarchical nodes in the inverse transposed structure image, and the number of the logical connections is not limited, and is usually multiple. The guide line segmentation model includes but is not limited to a contour extraction model, a Gaussian line extraction model, a multi-source image line extraction model, or other models for extracting image connections.

[0107] Accordingly, the present invention can also use a pre-trained key node detection model to detect and extract the hierarchical nodes in the inverse transposed structure image to obtain the hierarchical nodes in the inverse transposed structure image. The number of the hierarchical nodes is not limited, and the number of the hierarchical nodes of the inverse transposed structure image is usually multiple. The key node detection model includes but is not limited to a machine translation Tensorflow model, a finite element ANSYS extraction model, a feedforward neural network model, a convolutional neural network model, or other models for node extraction.

[0108] For example, see Figure 5 A schematic diagram showing a possible inverse transposition of the hierarchical structure information in the structure picture. Figure 5 As shown, the hierarchical structure information includes each hierarchical node in the inverse transposed structure picture and the logical connections between the hierarchical nodes. For example, the label ① in the diagram represents a hierarchical node, and the label ② in the diagram represents a logical connection.

[0109] After obtaining the hierarchical nodes and the logical connections between the hierarchical nodes in the inverse transposed structure image, the present invention can use a depth-first search algorithm based on point-line information to perform structural reconstruction on the hierarchical nodes and the logical connections between the hierarchical nodes to obtain the corresponding reconstructed hierarchical structure. Specifically, the present invention can use the above-mentioned depth-first search algorithm to perform information fusion on the hierarchical information and the logical connections between the hierarchical information, and determine whether any two of the hierarchical nodes have connectivity. If there is connectivity, the two hierarchical nodes with connectivity are connected together through corresponding logical connections; if there is no connectivity, the subsequent process continues, such as determining whether the other two hierarchical nodes have connectivity. Based on this principle, the reconstruction of the hierarchical structure information between nodes can be achieved to obtain the reconstructed hierarchical structure.

[0110] For example, see Figure 6 A schematic diagram of a possible reconstruction hierarchical structure is shown. Specifically, the present invention uses a depth-first search algorithm based on point-line information to Figure 5 The hierarchical structure information (specifically, multiple hierarchical nodes and logical connections between the hierarchical nodes) in the inverse transposed structure picture shown in FIG. 1 is reconstructed to obtain Figure 6 The reconstructed hierarchy is shown.

[0111] Finally, the present invention associates and matches the reverse transposed text with the reconstructed hierarchical structure, thereby obtaining the restored hierarchical structure image corresponding to the transposed hierarchical structure image. In a specific embodiment, the reverse transposed text includes at least one reverse transposed text line and the position of each reverse transposed text line (i.e., the text box position). At this time, the present invention can add each reverse transposed text to the corresponding position of the reconstructed hierarchical structure, such as the corresponding hierarchical node, etc., according to the position of each reverse transposed text line, so as to obtain the restored hierarchical structure image. It can be understood that when the reverse transposed text line includes the text content and text box of the reverse transposed text line, the present invention can specifically add the text content corresponding to each reverse transposed text line to the corresponding position (such as the hierarchical node) of the reconstructed hierarchical structure according to the position of each reverse transposed text line, so as to achieve the linkage matching between the text content and the hierarchical node, so as to obtain the final restored hierarchical structure image.

[0112] In practical applications, the restored hierarchical structure image can be stored in a file of a preset format, such as a json file, etc. The restored hierarchical structure image can also be displayed to the user for viewing and editing in the form of a visual structure tree.

[0113] In an optional embodiment, after the present invention obtains the restored hierarchical structure image, some editing tools may be used to perform information editing processing, such as modification processing, on the restored hierarchical structure image. Specifically, the present invention may receive an editing instruction from a user for the restored hierarchical structure image, and the editing instruction is used to request editing processing of target information in the restored hierarchical structure image, and the target information includes at least one of the following: any target text line, any target hierarchical node, and any target logical connection in the restored hierarchical structure image. The number of the target text lines, the target hierarchical nodes, and the target logical connections is not limited, and can be determined according to the actual needs of the user.

[0114] Accordingly, after receiving the editing instruction, the present invention can respond to the editing instruction and perform corresponding editing processing on the target information in the restored hierarchical structure image, such as modifying a text line at a certain level node in the restored hierarchical structure image, etc.

[0115] By implementing the embodiment of the present invention, the present invention obtains a transposed hierarchical structure picture, performs text recognition on the transposed hierarchical structure picture to obtain the transposed text in the transposed hierarchical structure picture, then performs image transposition classification on the transposed text to obtain the target transposition classification of the transposed hierarchical structure picture, and finally restores the transposed hierarchical structure picture according to the target transposition classification to obtain the restored hierarchical structure picture corresponding to the transposed hierarchical structure picture. In the above scheme, the target transposition classification of the entire transposed hierarchical structure picture can be identified based on the transposed text in the transposed hierarchical structure picture, and then the transposed hierarchical structure picture is restored according to the target transposition classification to obtain the final restored hierarchical structure picture, which can efficiently and accurately realize the rapid restoration of the transposed hierarchical structure picture, thereby improving the efficiency, accuracy and speed of structural picture restoration, and also solves the technical problems in the prior art that the correct restoration of the mind map after the image transposition transformation cannot be realized, thereby reducing the accuracy and reliability of the mind map restoration.

[0116] Based on the same inventive concept, the embodiments of this specification also provide a device and an electronic device corresponding to the structural image restoration method. Figure 7 , is a schematic diagram of a structural image restoration device provided by an embodiment of the present invention. Figure 7 The device 60 shown includes: a text recognition module 601, a transposition determination module 602, a structure restoration module 603 and an information fusion module 604. Among them:

[0117] The text recognition module 601 may also be referred to as an OCR recognition module, which is used to recognize text in a hierarchical structure image. For example, the text recognition module 601 in the present invention may be used to recognize transposed text in the transposed hierarchical structure image.

[0118] The transposition determination module 602 is used to determine the target transposition classification of the transposed hierarchical structure picture, and specifically can determine / determine the transposed text in the transposed hierarchical structure picture according to the transposed text in the transposed hierarchical structure picture.

[0119] The structure restoration module 603 is used to restore or reconstruct the hierarchical structure information in the transposed hierarchical structure image according to the target transposition classification, and may specifically include a guide line segmentation unit 6031, a key node detection unit 6032 and a structure search unit 6033.

[0120] The guide line segmentation unit 6031 is used to extract guide lines (i.e., connections) from the hierarchical structure image to obtain logical connections between hierarchical nodes in the hierarchical structure image. Specifically, for example, the guide line segmentation unit 6031 in the present invention can be used to extract connections from the inverse transposed structure image.

[0121] The key node detection unit 6032 is used to extract nodes from the hierarchical structure picture to obtain multiple hierarchical nodes in the hierarchical structure picture. Specifically, for example, the key node detection unit 6032 in the present invention can be used to extract nodes from the inverse transposed structure picture.

[0122] The structure search unit 6033 can be used to process the information extracted by the guide line segmentation unit 6031 and the key node detection unit 6032. For example, the structure search unit 6033 in the present invention can be used to reconstruct the structure of multiple hierarchical nodes in the inverse transposed structure image and the logical connections between the hierarchical nodes, such as connectivity judgment between nodes and hierarchical structure reconstruction between nodes, so as to obtain a reconstructed hierarchical structure.

[0123] The information fusion module 604 is used to associate and match the inverse transposed text corresponding to the transposed text with the reconstructed hierarchical structure to obtain the restored hierarchical structure image.

[0124] For the contents not introduced or described in the embodiments of the present invention, please refer to the relevant introduction in the aforementioned method embodiments, which will not be repeated here.

[0125] Please also see Figure 8 , is a schematic diagram of another structural image restoration device provided by an embodiment of the present invention. Figure 8 The device 70 shown includes: an acquisition module 701, an identification module 702, a classification module 703 and a restoration module 704. Among them:

[0126] The acquisition module 701 is used to acquire a transformed hierarchical structure picture, where the transformed hierarchical structure picture is a hierarchical structure picture after image transformation processing;

[0127] The recognition module 702 is used to perform text recognition on the transformed hierarchical structure picture to obtain recognized text in the transformed hierarchical structure picture;

[0128] The classification module 703 is used to perform image transformation classification on the recognized text to obtain the target transformation classification of the transformation hierarchical structure picture;

[0129] The restoration module 704 is used to restore the transformed hierarchical structure picture according to the target transformation classification to obtain a corresponding restored hierarchical structure picture.

[0130] Optionally, when the transformed hierarchical structure picture is a transposed hierarchical structure picture after image transposition transformation, the recognized text in the transformed hierarchical structure picture is a transposed text, and the target transformation classification is a target transposed classification; the restoration module 704 is specifically used to:

[0131] According to the target transposition classification, the transposed hierarchical structure image is subjected to inverse transposition processing to obtain an inverse transposition structure image, wherein the inverse transposition structure image includes an inverse transposed text corresponding to the transposed text, a plurality of hierarchical nodes, and logical connections between the hierarchical nodes;

[0132] Reconstructing the structure of the plurality of hierarchical nodes and the logical connections between the hierarchical nodes in the inverse transposed structure picture to obtain a reconstructed hierarchical structure;

[0133] The reverse transposed text and the reconstructed hierarchical structure are associated and matched to obtain the restored hierarchical structure image.

[0134] Optionally, the restoration module 704 is specifically used for:

[0135] Performing node extraction and line extraction on the inverse transposed structure image to obtain the multiple hierarchical nodes in the inverse transposed structure image and the logical lines between the hierarchical nodes;

[0136] The plurality of hierarchical nodes and the logical connections between the hierarchical nodes are structurally reconstructed to obtain the reconstructed hierarchical structure.

[0137] Optionally, the reverse transposed text includes at least one reverse transposed text line and the position of each reverse transposed text line, and the restoration module 704 is specifically used to:

[0138] According to the position of each of the reverse transposed text lines, each of the reverse transposed text lines is added to the reconstructed hierarchical structure to obtain the restored hierarchical structure image.

[0139] Optionally, the device further includes a receiving module 705 and a processing module 706, wherein:

[0140] The receiving module 705 is used to receive an editing instruction for the restored hierarchical structure picture, where the editing instruction is used to edit target information in the restored hierarchical structure picture, where the target information includes at least one of the following: a target text line, a target hierarchical node, and a target logical connection;

[0141] The processing module 706 is used to respond to the editing instruction and perform corresponding editing processing on the target information in the restored hierarchical structure picture.

[0142] Optionally, the restored hierarchical structure picture includes an editable reconstructed hierarchical structure picture.

[0143] The receiving module 705 is further configured to receive an editing instruction for the reconstructed hierarchical structure diagram, wherein the editing instruction is used to edit the reconstructed hierarchical structure in the reconstructed hierarchical structure diagram;

[0144] The processing module 706 is further configured to respond to the editing instruction and perform corresponding editing processing on the reconstructed hierarchical structure picture in the restored hierarchical structure picture.

[0145] Optionally, the transposed text includes at least one transposed text line, and the classification module 703 is specifically configured to:

[0146] Performing image transposition classification on each of the transposed text lines to obtain a text transposition classification of each of the transposed text lines;

[0147] When the number of transposed text lines of a preset transposition category exceeds a preset threshold, the preset transposition category is determined as a target transposition category for the transposed hierarchical structure picture.

[0148] It should be noted that the identification module 702 in the device of the present invention may be used correspondingly Figure 7 The text recognition module 601 in the device shown in the figure is replaced by the recognition module 702. In other words, the recognition module 702 and the text recognition module 601 are modules with the same function. Figure 7 The transposition determination module 602 in the device shown in the figure is replaced by the restoration module 704 in the device of the present invention. Figure 7 The structure restoration module 603 and the information fusion module 604 in the device shown in the figure are used as an alternative implementation. Regarding the device in the above embodiment, the specific way in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0149] Based on the same inventive concept, an embodiment of the present invention provides an electronic device 800, Fig. 9 8 is a block diagram of an electronic device 800 according to an exemplary embodiment. For example, the device 800 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0150] Reference Fig. 9 , device 800 may include one or more of the following components: a processing component 802 , a memory 804 , a power component 806 , a multimedia component 808 , an audio component 810 , an input / output (I / O) interface 812 , a sensor component 814 , and a communication component 816 .

[0151] The processing component 802 generally controls the overall operation of the device 800, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 802 may include one or more modules to facilitate the interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate the interaction between the multimedia component 808 and the processing component 802.

[0152] The memory 804 is configured to store various types of data to support operations on the device 800. Examples of such data include instructions for any application or method operating on the device 800, contact data, phone book data, messages, pictures, videos, etc. The memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0153] The power component 806 provides power to the various components of the device 800. The power component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device 800.

[0154] The multimedia component 808 includes a screen that provides an output interface between the device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the device 800 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0155] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC), and when the device 800 is in an operating mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 804 or sent via the communication component 816. In some embodiments, the audio component 810 also includes a speaker for outputting audio signals.

[0156] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: home button, volume button, start button, and lock button.

[0157] The sensor assembly 814 includes one or more sensors for providing various aspects of status assessment for the device 800. For example, the sensor assembly 814 can detect the open / closed state of the device 800, the relative positioning of components, such as the display and keypad of the device 800, and the sensor assembly 814 can also detect the position change of the device 800 or a component of the device 800, the presence or absence of user contact with the device 800, the orientation or acceleration / deceleration of the device 800, and the temperature change of the device 800. The sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 814 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0158] The communication component 816 is configured to facilitate wired or wireless communication between the device 800 and other devices. The device 800 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0159] In an exemplary embodiment, the device 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above methods.

[0160] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 804 including instructions, which can be executed by a processor 820 of the device 800 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0161] Fig.10 1 is a schematic diagram of the structure of the server in some embodiments of the present invention. The server 1900 may have relatively large differences due to different configurations or performances, and may include one or more central processing units (CPU) 1922 (for example, one or more processors) and memory 1932, and one or more storage media 1930 (for example, one or more mass storage devices) storing application programs 1942 or data 1944. Among them, the memory 1932 and the storage medium 1930 can be short-term storage or permanent storage. The program stored in the storage medium 1930 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the server. Furthermore, the central processing unit 1922 can be configured to communicate with the storage medium 1930 and execute a series of instruction operations in the storage medium 1930 on the server 1900.

[0162] The server 1900 may also include one or more power supplies 1926, one or more wired or wireless network interfaces 1950, one or more input and output interfaces 1958, one or more keyboards 1956, and / or, one or more operating systems 1941, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.

[0163] A non-temporary computer-readable storage medium, when the instructions in the storage medium are executed by a processor of a device (server or terminal), enables the device to execute the structural image restoration method of the aforementioned embodiments.

[0164] A non-temporary computer-readable storage medium, when the instructions in the storage medium are executed by the processor of the device (server or terminal), the computer device executes the above Figure 1 The description of the structural image restoration method in the corresponding embodiment will not be repeated here. In addition, the description of the beneficial effects of the same method will not be repeated here.

[0165] In addition, it should be noted that the embodiments of the present application also provide a computer program product or computer program embodiment involved in the present application for technical details not disclosed, please refer to the description of the method embodiment of the present application. The computer program product or computer program may include computer instructions, which may be stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor may execute the computer instructions, so that the computer device executes the above Figure 1 The description of the structural image restoration method in the corresponding embodiment will not be repeated here. In addition, the description of the beneficial effects of the same method will not be repeated here.

[0166] One or more technical solutions provided by the embodiments of the present invention achieve at least the following technical effects or advantages: the present invention obtains a transformation hierarchical structure picture, performs text recognition on the transformation hierarchical structure picture to obtain the recognized text in the transformation hierarchical structure picture, then performs transformation classification on the recognized text to obtain the target transformation classification of the transformation hierarchical structure picture, and finally performs restoration processing on the transformation hierarchical structure picture according to the target transformation classification to obtain the restored hierarchical structure picture corresponding to the transformation hierarchical structure picture. In the above scheme, the target transformation classification of the entire transformation hierarchical structure picture can be identified based on the recognized text in the transformation hierarchical structure picture, and then the transformation hierarchical structure picture is restored according to the target transformation classification to obtain the final editable restored hierarchical structure picture, which can efficiently and accurately realize the rapid restoration of the transformation hierarchical structure picture, thereby improving the efficiency, accuracy and speed of structural picture restoration.

[0167] Those skilled in the art will readily appreciate other embodiments of the invention after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses or adaptations of the present invention that follow the general principles of the present invention and include common knowledge or customary techniques in the art that are not disclosed in this disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present invention are indicated by the following claims.

[0168] It should be understood that the present invention is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from the scope thereof. The scope of the present invention is limited only by the appended claims. The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention shall be included in the scope of protection of the present invention.

Claims

1. A structural image restoration method, characterized in that: include: Obtaining a transformed hierarchical structure picture, wherein the transformed hierarchical structure picture is a hierarchical structure picture after image transformation processing; Performing text recognition on the transformed hierarchical structure picture to obtain recognized text in the transformed hierarchical structure picture; Performing image transformation classification on the recognized text to obtain a target transformation classification of the transformed hierarchical structure picture; when the transformed hierarchical structure picture is a transposed hierarchical structure picture after image transposition transformation, the recognized text in the transformed hierarchical structure picture is a transposed text, and the target transformation classification is a target transposed classification; Performing restoration processing on the transformed hierarchical structure picture according to the target transformation classification to obtain a corresponding restored hierarchical structure picture; The restoring processing of the transformed hierarchical structure picture according to the target transformation classification to obtain the corresponding restored hierarchical structure picture includes: According to the target transposition classification, the transposed hierarchical structure image is subjected to inverse transposition processing to obtain an inverse transposition structure image, wherein the inverse transposition structure image includes an inverse transposed text corresponding to the transposed text, a plurality of hierarchical nodes, and logical connections between the hierarchical nodes; Reconstructing the structure of the plurality of hierarchical nodes and the logical connections between the hierarchical nodes in the inverse transposed structure picture to obtain a reconstructed hierarchical structure; The reverse transposed text and the reconstructed hierarchical structure are associated and matched to obtain the restored hierarchical structure image.

2. The method according to claim 1, characterized in that When the transformed hierarchical structure picture is a mirrored hierarchical structure picture after image mirroring transformation, the recognized text in the transformed hierarchical structure picture is a mirrored text, and the target transformation classification is a target mirrored classification; The restoring processing of the transformed hierarchical structure picture according to the target transformation classification to obtain the corresponding restored hierarchical structure picture includes: According to the target image classification, the image hierarchical structure image is mirror-flipped accordingly to obtain a flipped hierarchical structure image, wherein the flipped hierarchical structure image includes a flipped text corresponding to the image text, a plurality of hierarchical nodes, and logical connections between the hierarchical nodes; Reconstructing the structure of the plurality of hierarchical nodes and the logical connections between the hierarchical nodes in the flipped hierarchical structure image to obtain a reconstructed hierarchical structure; The flipped text and the reconstructed hierarchical structure are associated and matched to obtain the restored hierarchical structure image.

3. The method according to claim 1, characterized in that The structural reconstruction of the plurality of hierarchical nodes and the logical connections between the hierarchical nodes in the inverse transposed structure picture to obtain a reconstructed hierarchical structure includes: Performing node extraction and line extraction on the inverse transposed structure picture to obtain the multiple hierarchical nodes in the inverse transposed structure picture and the logical lines between the hierarchical nodes; The plurality of hierarchical nodes and the logical connections between the hierarchical nodes are structurally reconstructed to obtain the reconstructed hierarchical structure.

4. The method according to claim 1, characterized in that: The reverse transposed text includes at least one reverse transposed text line and the position of each reverse transposed text line, and the associating and matching the reverse transposed text with the reconstructed hierarchical structure to obtain the restored hierarchical structure image includes: According to the position of each of the reverse transposed text lines, each of the reverse transposed text lines is added to the reconstructed hierarchical structure to obtain the restored hierarchical structure image.

5. The method according to claim 4, characterized in that The method further comprises: receiving an editing instruction for the restored hierarchical structure picture, the editing instruction being used to modify target information in the restored hierarchical structure picture, the target information comprising at least one of the following: a target text line, a target hierarchical node, and a target logical connection; In response to the editing instruction, corresponding modification processing is performed on the target information in the restored hierarchical structure picture.

6. The method according to claim 4, characterized in that The restored hierarchical structure picture includes an editable reconstructed hierarchical structure picture, and the method further includes: receiving an editing instruction for the reconstructed hierarchical structure diagram, wherein the editing instruction is used to edit the reconstructed hierarchical structure in the reconstructed hierarchical structure diagram; In response to the editing instruction, corresponding editing processing is performed on the reconstructed hierarchical structure picture in the restored hierarchical structure picture.

7. The method according to claim 1, characterized in that The transposed text includes at least one transposed text line, and the image transformation classification of the recognized text to obtain the target transformation classification of the transformation hierarchical structure picture includes: Performing image transposition classification on each of the transposed text lines to obtain a text transposition classification of each of the transposed text lines; When the number of transposed text lines of a preset transposition category exceeds a preset threshold, the preset transposition category is determined as a target transposition category for the transposed hierarchical structure picture.

8. A structural image restoration device, characterized in that: include: An acquisition module, used to acquire a transformed hierarchical structure picture, wherein the transformed hierarchical structure picture is a hierarchical structure picture after image transformation processing; A recognition module, used to perform text recognition on the transformed hierarchical structure picture to obtain recognized text in the transformed hierarchical structure picture; A classification module, used for performing image transformation classification on the recognized text to obtain a target transformation classification of the transformed hierarchical structure picture; when the transformed hierarchical structure picture is a transposed hierarchical structure picture after image transposition transformation, the recognized text in the transformed hierarchical structure picture is a transposed text, and the target transformation classification is a target transposition classification; A restoration module, used for performing restoration processing on the transformed hierarchical structure picture according to the target transformation classification to obtain a corresponding restored hierarchical structure picture; The restoration module is specifically used for: According to the target transposition classification, the transposed hierarchical structure image is subjected to inverse transposition processing to obtain an inverse transposition structure image, wherein the inverse transposition structure image includes an inverse transposed text corresponding to the transposed text, a plurality of hierarchical nodes, and logical connections between the hierarchical nodes; Reconstructing the structure of the plurality of hierarchical nodes and the logical connections between the hierarchical nodes in the inverse transposed structure picture to obtain a reconstructed hierarchical structure; The reverse transposed text and the reconstructed hierarchical structure are associated and matched to obtain the restored hierarchical structure image.

9. The device according to claim 8, characterized in that The restoration module is specifically used for: Performing node extraction and line extraction on the inverse transposed structure picture to obtain the multiple hierarchical nodes in the inverse transposed structure picture and the logical lines between the hierarchical nodes; The plurality of hierarchical nodes and the logical connections between the hierarchical nodes are structurally reconstructed to obtain the reconstructed hierarchical structure.

10. The device according to claim 8, characterized in that The reverse transposed text includes at least one reverse transposed text line and the position of each reverse transposed text line, and the restoration module is specifically used for: According to the position of each of the reverse transposed text lines, each of the reverse transposed text lines is added to the reconstructed hierarchical structure to obtain the restored hierarchical structure image.

11. An electronic device, characterized in that: The invention comprises a memory and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by one or more processors to execute the operation instructions corresponding to the method according to any one of claims 1 to 7 contained in the one or more programs.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps corresponding to the method according to any one of claims 1 to 7 are implemented.

13. A computer program product, characterized in that The computer program product includes computer instructions, which are stored in a computer-readable storage medium and are suitable for being read and executed by a processor, so that a computer device having the processor executes the method according to any one of claims 1 to 7.