Structural image restoration method, device, electronic device, medium and program product

Abstract

The present invention discloses a structural image restoration method, device, electronic device and readable storage medium, the method comprising: receiving a restoration request for an original structural image; responding to the restoration request, performing text recognition on the original structural image to obtain the original text; responding to the restoration request, performing node recognition and line recognition on the original structural image to obtain structural information, the structural information comprising multiple hierarchical nodes and logical lines between the hierarchical nodes; responding to the restoration request, determining the position information between the original text and the hierarchical nodes and the logical lines in the original structural image; restoring the original structural image according to the original text, the structural information and the determined position information to obtain a restored structural diagram. The above scheme can realize the rapid restoration of the original structural image.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a structural image restoration method, device, electronic equipment, medium and program product. Background Art

[0002] A structure diagram is a clearly nested structure diagram that often appears in users' daily work and study life. Taking mind mapping as an example, it is an effective graphic thinking tool for expressing divergent thinking. To facilitate file transmission and user reading, mind maps usually exist in the form of images. However, when mind maps are stored and transmitted in image format, there is a big flaw - they can only support viewing but not editing.

[0003] Therefore, in order to realize the re-editing operation of mind maps, it is urgent to propose a structural image restoration solution. Summary of the invention

[0004] The embodiments of the present invention provide a structure diagram restoration method, device, electronic device and readable storage medium, thereby solving the technical problem in the prior art that the mind map cannot support re-editing operations because it exists in the form of an image.

[0005] In a first aspect, an embodiment of the present invention provides a method for restoring a structured image, comprising:

[0006] Receiving a restoration request for the original structure image;

[0007] In response to the restoration request, performing text recognition on the original structure image to obtain the original text in the original structure image;

[0008] In response to the restoration request, perform node identification and line identification on the original structure image to obtain structure information in the original structure image, where the structure information includes a plurality of hierarchical nodes and logical lines between the hierarchical nodes;

[0009] In response to the restoration request, determining position information between the original text and the hierarchical nodes and the logical connections in the original structure image;

[0010] The original structure image is restored according to the original text, the structure information and the determined position information to obtain a restored structure image.

[0011] Optionally, the method further comprises:

[0012] Receiving an edit request for the restored structure diagram, the edit request being used to request a corresponding edit operation to be performed on target information in the restored structure diagram, the target information comprising at least one of the following: a target text line, a target hierarchical node, and a target logical connection;

[0013] In response to the editing request, a corresponding editing operation is performed on the target information in the restored structure diagram.

[0014] Optionally, in response to the editing request, performing a corresponding editing operation on the target information in the restored structure diagram includes at least one of the following:

[0015] If the editing request is a text editing request, responding to the text editing request, performing a corresponding editing operation on the target text line in the restoration structure diagram;

[0016] If the editing request is a node editing request, responding to the node editing request, performing a corresponding editing operation on the target level node in the restored structure diagram;

[0017] If the editing request is a connection editing request, then in response to the connection editing request, a corresponding editing operation is performed on the target logical connection in the restored structure diagram.

[0018] Optionally, the original text includes at least one original text line, and the responding to the restoration request and determining the position information between the original text and the hierarchical nodes and the logical connections in the original structure image includes:

[0019] In response to the restoration request, determining position information of each of the original text lines, where the position information of the original text line refers to position information between the original text line and the hierarchical node and / or the logical connection;

[0020] The restoring the original structure picture according to the original text, the structure information and the determined position information to obtain the restored structure picture includes:

[0021] Reconstructing the structure of multiple hierarchical nodes in the structural information and logical connections between the hierarchical nodes to obtain a reconstructed structure;

[0022] According to the position information of each original text line, each original text line is added to the reconstructed structure to obtain the restored structure diagram.

[0023] Optionally, before performing text recognition on the original structured image, the method further includes:

[0024] Get the initial structure picture;

[0025] The region picture to be processed is selected from the initial structure picture, thereby obtaining the original structure picture.

[0026] Optionally, the method further comprises:

[0027] Receive a rendering request, the rendering request being used to request a rendering operation to be performed on the information to be rendered in the restoration structure graph, the information to be rendered including at least one of the following: a text line to be rendered, a level node to be rendered, a logical connection to be rendered, and an image area to be rendered;

[0028] In response to the rendering request, a corresponding rendering operation is performed on the information to be rendered in the restored structure diagram.

[0029] Optionally, the original structural image is a structural image that complies with a preset image specification, or is a structural image that has not undergone any image transformation processing.

[0030] In a second aspect, an embodiment of the present invention provides a structured image restoration device, comprising:

[0031] A receiving module, used for receiving a restoration request for the original structure image;

[0032] A processing module, configured to respond to the restoration request, perform text recognition on the original structure image, and obtain the original text in the original structure image;

[0033] The processing module is further used to respond to the restoration request, perform node identification and line identification on the original structure image, and obtain structure information in the original structure image, wherein the structure information includes multiple hierarchical nodes and logical lines between the hierarchical nodes;

[0034] The processing module is further used to respond to the restoration request and determine the position information between the original text and the hierarchical nodes and the logical connections in the original structure image;

[0035] The restoration module is used to restore the original structure image according to the original text, the structure information and the determined position information to obtain a restored structure image.

[0036] For the contents not introduced or described in the embodiments of the present invention, please refer to the relevant introduction in the aforementioned method embodiments, which will not be repeated here.

[0037] In a third aspect, an embodiment of the present invention provides an electronic device comprising a memory and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by one or more processors to execute operation instructions corresponding to the one or more programs for performing the structural image restoration method provided in the first aspect.

[0038] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps corresponding to the structural image restoration method provided in the first aspect.

[0039] One or more technical solutions provided by the embodiments of the present invention achieve at least the following technical effects or advantages:

[0040] The solution provided by the embodiment of the present invention receives a restoration request for an original structural image, performs text recognition on the original structural image in response to the restoration request to obtain the corresponding original text, performs node recognition and line recognition on the original structural image in response to the restoration request to obtain the corresponding structural information, the structural information includes multiple hierarchical nodes in the original structural image and the logical lines between the hierarchical nodes, further responds to the restoration request and can also determine the position information between the original text and the hierarchical nodes and the logical lines in the original structural image, and finally restores the original structural image according to the original text, the structural information and the determined positional relationship to obtain the corresponding restored structural diagram. In the above scheme, the text, nodes, lines and positions of the original structural image can be directly recognized and determined in response to the restoration request for the original structural image, so as to restore the corresponding editable restored structural diagram, which not only realizes the rapid restoration of the original structural image, but also solves the technical problem in the prior art that the mind-guided image exists in the form of a picture image and cannot support re-editing operations.

[0041] Attached picture description

[0042] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0043] Figure 1 It is a flow chart of a structural image restoration method provided in an embodiment of the present application.

[0044] Figure 2 It is a schematic diagram of an original structure picture provided by an embodiment of the present invention.

[0045] Figure 3 It is a schematic diagram of structural information in an original structural image provided by an embodiment of the present invention.

[0046] Figure 4 It is a schematic diagram of a reconstruction structure provided by an embodiment of the present invention.

[0047] Figure 5 It is a structural schematic diagram of a structural image restoration device provided by an embodiment of the present invention.

[0048] Figure 6 It is a schematic diagram of the structure of another structural image restoration device provided by an embodiment of the present invention.

[0049] Figure 7 is a schematic diagram of an electronic device provided by an embodiment of the present invention.

[0050] Figure 8 It is a structural diagram of a server provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0051] The present invention provides a structured image restoration method, device, electronic device and readable storage medium to solve the technical problem in the prior art that the mind map exists in the form of an image and cannot support re-editing operations. The overall idea is as follows:

[0052] Receive a restoration request for an original structural image; in response to the restoration request, perform text recognition on the original structural image to obtain the original text in the original structural image; in response to the restoration request, perform node recognition and line recognition on the original structural image to obtain structural information in the original structural image, wherein the structural information includes multiple hierarchical nodes and logical lines between the hierarchical nodes; in response to the restoration request, determine the position information between the original text and the hierarchical nodes and the logical lines in the original structural image; restore the original structural image according to the original text, the structural information and the determined position information to obtain a restored structural diagram.

[0053] Through the above technical scheme, the original structure image can be restored in direct response to the image restoration request, and an editable restored structure diagram can be obtained. This not only realizes the rapid restoration of the original structure image, but also solves the technical problem in the prior art that the mind-guided image exists in the form of an image and cannot support re-editing operations.

[0054] Please refer to Figure 1 , is a flow chart of a structural image restoration method provided by an embodiment of the present invention. The method can be applied to a terminal device, such as a smart phone, a tablet computer, or a server that has established data interaction with the terminal device, or a system composed of a terminal device and a server, which is not limited by the present invention. Figure 1 The method shown comprises the following steps:

[0055] S101. Receive a restoration request for an original structure picture, where the restoration request is used to request restoration processing of the original structure picture.

[0056] The restoration request of the present invention may be generated when a user performs a restoration operation on the original structure image, or may be received from other devices (such as other terminal devices or servers), and the present invention does not limit this.

[0057] S102: In response to the restoration request, perform text recognition on the original structure image to obtain the original text in the original structure image.

[0058] In a specific implementation, the present invention can respond to the restoration request and first obtain the original structure image. The original structure image refers to a conventional structure image that has not undergone any image transformation processing, or refers to a structure image that meets a preset image specification. The preset image specification is a standard specification set by a system, for example, image information such as image size and image orientation meets a fixed image format.

[0059] For example, see Figure 2 A schematic diagram showing a possible original structure picture is shown below. Figure 2 The original structural image shown is a regular structural image input by a user, that is, a structural image that has not been subjected to any image transformation processing, such as image rotation, image transposition, or image mirror transformation.

[0060] The structural picture involved in the present invention refers to a structural picture that can be used to represent hierarchical or level relationships. The structural picture can clearly show the hierarchical relationships of each level, which may include but is not limited to mind-guided pictures, process structure pictures, tree structure pictures, or other customized structural pictures with hierarchical or level relationships.

[0061] In a specific embodiment, the present invention can first obtain an initial structural image, and then arbitrarily select a picture image area to be processed from the initial structural image as the original structural image. In other words, the original structural image is a part of the structural image in the initial structural image. Specifically, the present invention can select the original structural image from the initial structural image according to the actual needs of the system. For example, the user can circle / select the picture image area to be processed from the initial structural image according to his or her actual needs, so that the picture image area circled by the user is used as the original structural image.

[0062] After the original structural image is acquired, text recognition may be performed on the original structural image to obtain the original text in the original structural image.

[0063] In one embodiment, the present invention may use text recognition technology to perform text recognition on the original structure image to obtain the original text in the original structure image. The text recognition technology includes but is not limited to optical character recognition (OCR), geometric feature extraction technology, or other technologies for text or word recognition.

[0064] In another embodiment, the present invention can use a pre-trained text recognition model to perform text recognition on the original structure image to obtain the original text in the original structure image. The text recognition model is pre-trained for recognizing text information in the structure image, which may include but is not limited to a feedforward neural network model, a convolutional neural network model, a deep residual network model, a recurrent neural network model, a long short-term memory model, or other machine learning models.

[0065] The original text involved in the present invention includes but is not limited to at least one original text line and the position (also referred to as position coordinates) of each original text line. The original text line can refer to lines of text content, or lines of text content and text boxes owned by the text content, which is not limited in the present invention. Optionally, the present invention takes the text line as an example of text content to explain the relevant content below, but it does not constitute a limitation.

[0066] S103 . In response to the restoration request, extract structure information (ie, node identification and line identification) from the original structure image to obtain structure information in the original structure image.

[0067] The structural information of the present invention includes but is not limited to the hierarchical nodes in the original structural image, the logical connections between the hierarchical nodes, or other information. The number of the hierarchical nodes and the logical connections is not limited, and may be one or more. Usually in practical applications, the hierarchical nodes in the original structural image and the logical connections between the hierarchical nodes are both multiple.

[0068] In one embodiment, the present invention can perform node identification and extraction on the original structure image to obtain multiple hierarchical nodes in the original structure image. Specifically, the present invention can use a pre-trained key node detection model to detect and extract the hierarchical nodes in the original structure image to obtain the hierarchical nodes in the original structure image. The number of hierarchical nodes is not limited, and usually the number of hierarchical nodes in the original structure image is multiple. The key node detection model includes but is not limited to a machine translation Tensorflow model, a finite element ANSYS extraction model, a feedforward neural network model, a convolutional neural network model, or other models for node extraction.

[0069] In another embodiment, the present invention can identify and extract the connection lines of the original structure image to obtain the logical connection lines between the hierarchical nodes in the original structure image. Specifically, the present invention can use a pre-trained guide line segmentation model to extract the logical connection lines between the hierarchical nodes in the original structure image to segment the logical connection lines from the image background of the original structure image, thereby obtaining the logical connection lines between the hierarchical nodes in the original structure image, and the number of the logical connection lines is not limited, usually multiple. The guide line segmentation model includes but is not limited to a contour extraction model, a Gaussian line extraction model, a multi-source image line extraction model, or other models for extracting image lines.

[0070] For example, see Figure 3 A schematic diagram showing the structural information in a possible original structural image is shown in FIG. Figure 3 As shown, the structural information includes each hierarchical node in the original structural image and the logical connections between the hierarchical nodes. For example, the number ① in the diagram represents a hierarchical node, and the number ② in the diagram represents a logical connection. It should be noted that the hierarchical node involved in the present invention may refer to a wireframe or endpoint representing each level in the structural image, and the wireframe may refer to a wireframe of a regular shape or a wireframe of an irregular shape, which is not limited in the present invention.

[0071] S104. In response to the restoration request, determine the position information between the original text and the hierarchical nodes and the logical connections in the original structure image.

[0072] In a specific embodiment, the original text includes at least one original text line. The present invention can respond to the restoration request and determine the position information of each original text line. The position information of the original text line here refers to the position relationship between the original text line and the hierarchical node and / or the logical connection, in other words, the position information of the original text line in the structural information.

[0073] S104: restore the structural information according to the original text, the structural information and the determined position information to obtain a corresponding restored structural diagram.

[0074] In a specific embodiment, the present invention may first reconstruct the structure of the multiple layer nodes in the structural information and the logical connections between the hierarchical nodes to obtain the corresponding reconstructed structure. Specifically, the present invention may use a depth-first search algorithm based on point-line information to fuse the multiple hierarchical information and the logical connections between the hierarchical information, and determine whether any two of the hierarchical nodes have connectivity. If there is connectivity, the two hierarchical nodes with connectivity are connected together through corresponding logical connections; if there is no connectivity, the subsequent process continues, such as determining whether the other two hierarchical nodes have connectivity. Based on this principle, the reconstruction of the structural information between nodes can be achieved to obtain the reconstructed structure.

[0075] For example, see Figure 4 A schematic diagram of a possible reconstruction structure is shown. Specifically, the present invention uses a depth-first search algorithm based on point-line information to Figure 3 The structural information (specifically, multiple hierarchical nodes and logical connections between the hierarchical nodes) in the original structural image shown in FIG. Figure 4 The reconstructed structure is shown.

[0076] Next, the present invention associates and matches the original text with the reconstructed structure to obtain the corresponding restored structure diagram. Specifically, the present invention can add each of the original texts to the corresponding position of the reconstructed structure, such as the corresponding hierarchical node, etc., according to the position information of each of the original text lines (such as the text box position corresponding to the original text line, etc.), so as to obtain the restored structure diagram. It can be understood that when the original text line includes the text content and text box of the original text line, the present invention can specifically add the text content corresponding to each of the original text lines to the corresponding position (such as the hierarchical node) of the reconstructed structure according to the position information of each of the original text lines, so as to achieve the linkage matching between the text content and the hierarchical node to obtain the final restored structure diagram.

[0077] In practical applications, the restored structure diagram can be stored in a file of a preset format, such as a json file, etc. The restored structure diagram can also be displayed to the user for viewing and editing in the form of a visual structure tree.

[0078] In an optional embodiment, after the present invention obtains the restored structure diagram, some editing tools may be used to perform information editing processing, such as modification processing, on the restored structure diagram. Specifically, the present invention can receive an editing instruction from a user for the restored structure diagram, and the editing instruction is used to request modification processing of the target information in the restored structure diagram, and the target information includes at least one of the following: any target text line, any target hierarchical node, and any target logical connection in the restored structure diagram. The number of the target text lines, the target hierarchical nodes, and the target logical connections is not limited, and can be determined according to the actual needs of the user. Accordingly, after receiving the editing instruction, the present invention can respond to the editing instruction and perform corresponding editing operations on the target information in the restored structure diagram, such as modifying the text line at a certain hierarchical node in the restored structure diagram, etc. The following introduces several possible implementation methods.

[0079] In one embodiment, if the editing request is specifically a text editing request, and the text editing request is used to request an editing operation to be performed on a target text line in the restored structure diagram, the present invention can respond to the text editing request and perform a corresponding editing operation, such as a modification operation, on the target text line in the restored structure diagram.

[0080] In another embodiment, if the edit request is a node edit request, and the node edit request is used to request an edit operation on a target level node in the restored structure diagram, the present invention can respond to the node edit request and perform a corresponding edit operation on the target level node in the restored structure diagram.

[0081] In another embodiment, if the editing request is a connection editing request, and the connection editing request is used to request to edit the target logical connection in the restored structure diagram, the present invention can respond to the connection editing request and perform corresponding editing operations on the target logical connection in the restored structure diagram.

[0082] It should be noted that the above-mentioned possible implementation modes may be deployed and implemented separately, or any one or more of them may be combined for implementation, and the present invention is not limited thereto.

[0083] In an optional embodiment, the present invention may receive a rendering request for the restored structure diagram, the rendering request being used to request a rendering operation on the information to be rendered in the restored structure diagram, the rendering information including at least one of the following: a text line to be rendered, a hierarchical node to be rendered, a logical connection to be rendered, and a picture image area to be rendered. The text line to be rendered is any one or more restored text lines in the restored structure diagram. The hierarchical node to be rendered is any one or more restored hierarchical nodes in the restored structure diagram. The logical connection to be rendered is any one or more of the logical connections between the hierarchical nodes in the restored structure diagram. The picture image area to be rendered is any area in the restored structure diagram, such as the background area in the restored structure diagram, or a picture image area arbitrarily selected by the user, etc.

[0084] In response to the rendering request, a corresponding rendering operation is performed on the information to be rendered in the restored structure diagram, such as rendering the logical connections in the restored structure diagram in red.

[0085] By implementing the embodiment of the present invention, the present invention receives a restoration request for an original structural image, responds to the restoration request to perform text recognition on the original structural image to obtain the corresponding original text, responds to the restoration request to perform node recognition and line recognition on the original structural image to obtain the corresponding structural information, the structural information includes multiple hierarchical nodes in the original structural image and the logical connections between the hierarchical nodes, further responds to the restoration request to determine the position information between the original text and the hierarchical nodes and the logical connections in the original structural image, and finally restores the original structural image according to the original text, the structural information and the determined positional relationship to obtain the corresponding restored structural diagram. In the above scheme, the text, nodes, lines and positions of the original structural image can be directly recognized and determined in response to the restoration request for the original structural image, so as to restore the corresponding editable restored structural diagram, which not only realizes the rapid restoration of the original structural image, but also solves the technical problem in the prior art that the mind-guided image exists in the form of a picture image and cannot support re-editing operations.

[0086] Based on the same inventive concept, the embodiments of this specification also provide a device and an electronic device corresponding to the structural image restoration method. Figure 5 , is a schematic diagram of a structure diagram restoration device provided by an embodiment of the present invention. Figure 5 The device 50 shown includes: a text recognition module 501, a structure restoration module 502 and an information fusion module 503. Among them:

[0087] The text recognition module 501 may also be referred to as an OCR recognition module, which is used to recognize text in a structured image. For example, the text recognition module 501 in the present invention may be used to recognize original text in the original structured image.

[0088] The structure restoration module 502 is used to restore or reconstruct the structure information in the original structure image, and may specifically include a guide line segmentation unit 5021, a key node detection unit 5022, and a structure search unit 5023.

[0089] The guide line segmentation unit 5021 is used to extract guide lines (i.e., connections) from the structure image to obtain logical connections between hierarchical nodes in the structure image. Specifically, for example, the guide line segmentation unit 5021 in the present invention can be used to extract connections from the original structure image.

[0090] The key node detection unit 5022 is used to extract nodes from the structure picture to obtain multiple level nodes in the structure picture. Specifically, for example, the key node detection unit 5032 in the present invention can be used to extract nodes from the original structure picture.

[0091] The structure search unit 5023 can be used to process the information extracted by the guide line segmentation unit 5021 and the key node detection unit 5022. For example, the structure search unit 5023 in the present invention can be used to reconstruct the structure of multiple hierarchical nodes in the original structure image and the logical connections between the hierarchical nodes, such as connectivity judgment between nodes and hierarchical structure reconstruction between nodes, so as to obtain a reconstructed structure.

[0092] The information fusion module 503 is used to associate and match the original text with the reconstructed structure to obtain the restored structure image.

[0093] For the contents not introduced or described in the embodiments of the present invention, please refer to the relevant introduction in the aforementioned method embodiments, which will not be repeated here.

[0094] Please also see Figure 6 , is a schematic diagram of another structural image restoration device provided by an embodiment of the present invention. Figure 6 The device 60 shown includes: a receiving module 601, a processing module 602 and a restoring module 603, wherein:

[0095] The receiving module 601 is used to receive a restoration request for an original structured image;

[0096] The processing module 602 is used to respond to the restoration request, perform text recognition on the original structure picture, and obtain the original text in the original structure picture;

[0097] The processing module 602 is further configured to respond to the restoration request, perform node identification and line identification on the original structure image, and obtain structure information in the original structure image, wherein the structure information includes multiple hierarchical nodes and logical lines between the hierarchical nodes;

[0098] The processing module 602 is further configured to respond to the restoration request and determine the position information between the original text and the hierarchical nodes and the logical connections in the original structure image;

[0099] The restoration module 603 is used to restore the original structure image according to the original text, the structure information and the determined position information to obtain a restored structure image.

[0100] Optionally, the receiving module 601 is further used to receive an editing request for the restored structure diagram, the editing request being used to request a corresponding editing operation to be performed on target information in the restored structure diagram, the target information including at least one of the following: a target text line, a target hierarchical node, and a target logical connection;

[0101] The processing module 602 is further configured to respond to the editing request and perform corresponding editing operations on the target information in the restored structure diagram.

[0102] Optionally, the processing module 602 is specifically configured to perform any one of the following:

[0103] If the editing request is a text editing request, responding to the text editing request, performing a corresponding editing operation on the target text line in the restoration structure diagram;

[0104] If the editing request is a node editing request, responding to the node editing request, performing a corresponding editing operation on the target level node in the restored structure diagram;

[0105] If the editing request is a connection editing request, then in response to the connection editing request, a corresponding editing operation is performed on the target logical connection in the restored structure diagram.

[0106] Optionally, the original text includes at least one original text line, and the processing module 602 is specifically configured to:

[0107] In response to the restoration request, determining position information of each of the original text lines, where the position information of the original text line refers to position information between the original text line and the hierarchical node and / or the logical connection;

[0108] Reconstructing the structure of multiple hierarchical nodes in the structural information and logical connections between the hierarchical nodes to obtain a reconstructed structure;

[0109] According to the position information of each original text line, each original text line is added to the reconstructed structure to obtain the restored structure diagram.

[0110] Optionally, the processing module 602 is further configured to:

[0111] Get the initial structure picture;

[0112] The region picture to be processed is selected from the initial structure picture, thereby obtaining the original structure picture.

[0113] Optionally, the receiving module 601 is further used to receive a rendering request, wherein the rendering request is used to request a rendering operation to be performed on the information to be rendered in the restored structure diagram, wherein the information to be rendered includes at least one of the following: a text line to be rendered, a level node to be rendered, a logical connection to be rendered, and a picture area to be rendered;

[0114] The processing module 602 is further configured to respond to the rendering request and perform corresponding rendering operations on the information to be rendered in the restored structure diagram.

[0115] Optionally, the original structural image is a structural image that complies with a preset image specification, or is a structural image that has not undergone any image transformation processing.

[0116] It should be noted that the processing module 602 in the device of the present invention may be used correspondingly Figure 5 The text recognition module 501, the structure restoration module 503 and the information fusion module 504 in the device shown are replaced by the text recognition module 501, the structure restoration module 503 and the information fusion module 504. In other words, the module functions integrated by the text recognition module 501, the text recognition module 501, the structure restoration module 503 and the information fusion module 504 are the same as the processing module 602. Regarding the device in the above embodiment, the specific manner in which each module and each unit performs the operation has been described in detail in the embodiment of the method, and will not be elaborated here.

[0117] Based on the same inventive concept, an embodiment of the present invention provides an electronic device 800, Figure 7 8 is a block diagram of an electronic device 800 according to an exemplary embodiment. For example, the device 800 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0118] Reference Figure 7, device 800 may include one or more of the following components: a processing component 802 , a memory 804 , a power component 806 , a multimedia component 808 , an audio component 810 , an input / output (I / O) interface 812 , a sensor component 814 , and a communication component 816 .

[0119] The processing component 802 generally controls the overall operation of the device 800, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 802 may include one or more modules to facilitate the interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate the interaction between the multimedia component 808 and the processing component 802.

[0120] The memory 804 is configured to store various types of data to support operations on the device 800. Examples of such data include instructions for any application or method operating on the device 800, contact data, phone book data, messages, pictures, videos, etc. The memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0121] The power component 806 provides power to the various components of the device 800. The power component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device 800.

[0122] The multimedia component 808 includes a screen that provides an output interface between the device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the device 800 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0123] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC), and when the device 800 is in an operating mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 804 or sent via the communication component 816. In some embodiments, the audio component 810 also includes a speaker for outputting audio signals.

[0124] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0125] The sensor assembly 814 includes one or more sensors for providing various aspects of status assessment for the device 800. For example, the sensor assembly 814 can detect the open / closed state of the device 800, the relative positioning of components, such as the display and keypad of the device 800, and the sensor assembly 814 can also detect the position change of the device 800 or a component of the device 800, the presence or absence of user contact with the device 800, the orientation or acceleration / deceleration of the device 800, and the temperature change of the device 800. The sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 814 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0126] The communication component 816 is configured to facilitate wired or wireless communication between the device 800 and other devices. The device 800 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0127] In an exemplary embodiment, the device 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above methods.

[0128] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 804 including instructions, which can be executed by a processor 820 of the device 800 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0129] Figure 8 1 is a schematic diagram of the structure of the server in some embodiments of the present invention. The server 1900 may have relatively large differences due to different configurations or performances, and may include one or more central processing units (CPU) 1922 (for example, one or more processors) and memory 1932, and one or more storage media 1930 (for example, one or more mass storage devices) storing application programs 1942 or data 1944. Among them, the memory 1932 and the storage medium 1930 can be short-term storage or permanent storage. The program stored in the storage medium 1930 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the server. Furthermore, the central processing unit 1922 can be configured to communicate with the storage medium 1930 and execute a series of instruction operations in the storage medium 1930 on the server 1900.

[0130] The server 1900 may also include one or more power supplies 1926, one or more wired or wireless network interfaces 1950, one or more input and output interfaces 1958, one or more keyboards 1956, and / or, one or more operating systems 1941, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.

[0131] A non-temporary computer-readable storage medium, when the instructions in the storage medium are executed by a processor of a device (server or terminal), enables the device to execute the structural image restoration method of the aforementioned embodiments.

[0132] A non-temporary computer-readable storage medium, when the instructions in the storage medium are executed by the processor of the device (server or terminal), the computer device executes the above Figure 1 The description of the structural image restoration method in the corresponding embodiment will not be repeated here. In addition, the description of the beneficial effects of the same method will not be repeated here.

[0133] In addition, it should be noted that the embodiments of the present application also provide a computer program product or computer program embodiment involved in the present application for technical details not disclosed, please refer to the description of the method embodiment of the present application. The computer program product or computer program may include computer instructions, which may be stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor may execute the computer instructions, so that the computer device executes the above Figure 1 The description of the structural image restoration method in the corresponding embodiment will not be repeated here. In addition, the description of the beneficial effects of the same method will not be repeated here.

[0134] One or more technical solutions provided by the embodiments of the present invention at least achieve the following technical effects or advantages: the present invention receives a restoration request for an original structural image, performs text recognition on the original structural image in response to the restoration request to obtain the corresponding original text, performs node recognition and line recognition on the original structural image in response to the restoration request to obtain the corresponding structural information, the structural information includes multiple hierarchical nodes in the original structural image and the logical lines between the hierarchical nodes, further responds to the restoration request and can also determine the position information between the original text and the hierarchical nodes and the logical lines in the original structural image, and finally restores the original structural image according to the original text, the structural information and the determined position relationship to obtain the corresponding restored structural diagram. In the above scheme, the text, nodes, lines and positions of the original structural image can be directly recognized and determined in response to the restoration request for the original structural image, so as to restore the corresponding editable restored structural diagram, which not only realizes the rapid restoration of the original structural image, but also solves the technical problem in the prior art that the mind guide image exists in the form of a picture image and cannot support re-editing operations. .

[0135] Those skilled in the art will readily appreciate other embodiments of the invention after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses or adaptations of the present invention that follow the general principles of the present invention and include common knowledge or customary techniques in the art that are not disclosed in this disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present invention are indicated by the following claims.

[0136] It should be understood that the present invention is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from the scope thereof. The scope of the present invention is limited only by the appended claims. The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention shall be included in the scope of protection of the present invention.

Claims

1. A structural image restoration method, characterized in that: include: Receiving a restoration request for the original structure image; In response to the restoration request, performing text recognition on the original structure image to obtain original text in the original structure image; wherein the original text includes at least one original text line; In response to the restoration request, perform node identification and line identification on the original structure image to obtain structure information in the original structure image, where the structure information includes a plurality of hierarchical nodes and logical lines between the hierarchical nodes; In response to the restoration request, determining position information between the original text and the hierarchical nodes and the logical connections in the original structure image; In response to the restoration request, determining the position information between the original text and the hierarchical nodes and the logical connections in the original structure picture includes: In response to the restoration request, determining position information of each of the original text lines, wherein the position information of the original text line refers to position information between the original text line and the hierarchical node and the logical connection; Restoring the original structure image according to the original text, the structure information and the determined position information to obtain a restored structure image; The restoring the original structure picture according to the original text, the structure information and the determined position information to obtain the restored structure picture includes: Using a depth-first search algorithm based on point-line information to reconstruct the structure of multiple hierarchical nodes in the structural information and the logical connections between the hierarchical nodes to obtain a reconstructed structure; According to the position information of each original text line, each original text line is added to the reconstructed structure to obtain the restored structure diagram.

2. The method according to claim 1, characterized in that The method further comprises: Receiving an edit request for the restored structure diagram, the edit request being used to request a corresponding edit operation to be performed on target information in the restored structure diagram, the target information comprising at least one of the following: a target text line, a target hierarchical node, and a target logical connection; In response to the editing request, a corresponding editing operation is performed on the target information in the restored structure diagram.

3. The method according to claim 2, characterized in that In response to the editing request, performing a corresponding editing operation on the target information in the restored structure diagram includes at least one of the following: If the editing request is a text editing request, responding to the text editing request, performing a corresponding editing operation on the target text line in the restoration structure diagram; If the editing request is a node editing request, responding to the node editing request, performing a corresponding editing operation on the target level node in the restored structure diagram; If the editing request is a connection editing request, then in response to the connection editing request, a corresponding editing operation is performed on the target logical connection in the restored structure diagram.

4. The method according to claim 1, characterized in that Before performing text recognition on the original structure image, the method further includes: Get the initial structure picture; The region picture to be processed is selected from the initial structure picture, thereby obtaining the original structure picture.

5. The method according to claim 1, characterized in that The method further comprises: Receive a rendering request, the rendering request is used to request a rendering operation on the information to be rendered in the restored structure graph, the information to be rendered includes at least one of the following: a text line to be rendered, a level node to be rendered, a logical connection to be rendered, and a picture area to be rendered; In response to the rendering request, a corresponding rendering operation is performed on the information to be rendered in the restored structure diagram.

6. The method according to any one of claims 1 to 5, characterized in that The original structural image is a structural image that meets a preset image specification, or is a structural image that has not undergone any image transformation processing.

7. A structural image restoration device, characterized in that: include: A receiving module, used for receiving a restoration request for the original structure image; A processing module, configured to respond to the restoration request, perform text recognition on the original structure image, and obtain the original text in the original structure image; The processing module is further used to respond to the restoration request, perform node identification and line identification on the original structure image, and obtain structure information in the original structure image, wherein the structure information includes multiple hierarchical nodes and logical lines between the hierarchical nodes; The processing module is further used to respond to the restoration request and determine the position information between the original text and the hierarchical nodes and the logical connections in the original structure image; The original text includes at least one original text line, and the processing module is specifically used for: In response to the restoration request, determining position information of each of the original text lines, wherein the position information of the original text line refers to position information between the original text line and the hierarchical node and the logical connection; A restoration module, used to restore the original structure picture according to the original text, the structure information and the determined position information to obtain a restored structure picture; The restoration module is specifically used to reconstruct the structure of the multiple hierarchical nodes in the structural information and the logical connections between the hierarchical nodes by using a depth-first search algorithm based on point-line information to obtain a reconstructed structure; According to the position information of each original text line, each original text line is added to the reconstructed structure to obtain the restored structure diagram.

8. The device according to claim 7, characterized in that The receiving module is further used to receive an editing request for the restored structure diagram, wherein the editing request is used to request a corresponding editing operation to be performed on target information in the restored structure diagram, wherein the target information includes at least one of the following: a target text line, a target hierarchical node, and a target logical connection; The processing module is further used to respond to the editing request and perform corresponding editing operations on the target information in the restored structure diagram.

9. The device according to claim 8, characterized in that The processing module is specifically configured to perform any one of the following: If the editing request is a text editing request, responding to the text editing request, performing a corresponding editing operation on the target text line in the restoration structure diagram; If the editing request is a node editing request, responding to the node editing request, performing a corresponding editing operation on the target level node in the restored structure diagram; If the editing request is a connection editing request, then in response to the connection editing request, a corresponding editing operation is performed on the target logical connection in the restored structure diagram.

10. The device according to claim 7, characterized in that The receiving module is further used to receive a rendering request, wherein the rendering request is used to request a rendering operation to be performed on the information to be rendered in the restored structure diagram, wherein the information to be rendered includes at least one of the following: a text line to be rendered, a level node to be rendered, a logical connection to be rendered, and an image area to be rendered; The processing module is further used to respond to the rendering request and perform corresponding rendering operations on the information to be rendered in the restored structure diagram.

11. An electronic device, characterized in that: The invention comprises a memory and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by one or more processors to execute the operation instructions corresponding to the method according to any one of claims 1 to 6 contained in the one or more programs.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps corresponding to the method according to any one of claims 1 to 6 are implemented.

13. A computer program product, characterized in that The computer program product includes computer instructions, which are stored in a computer-readable storage medium and are suitable for being read and executed by a processor, so that a computer device having the processor executes the method according to any one of claims 1 to 6.

Images