Paper information digitization method, device and equipment and storage medium

Abstract

The invention discloses a paper information digitization method and device, equipment and a storage medium. According to the method, a target image containing paper content is determined, the paper content is terminal strip wiring content, the target image comprises a plurality of independent content areas, and each independent content area is identified and divided based on area features; key primitive retrieval is carried out on each independent content area in the target image according to a set wiring drawing analysis rule and analysis prompt information, and primitive positions and connection information of each key primitive are obtained; performing character recognition on the corresponding key primitives according to the positions of the primitives to obtain recognition results of the key primitives; and performing structured processing on each identification result, and generating a set format digital file corresponding to the target image based on the structured processing result and each piece of connection information. By utilizing the method, the self-adaptive analysis of the paper contents of different formats is realized by relying on the wiring drawing analysis rule and the analysis prompt information, and the accuracy of paper content identification is improved.

Description

Technical Field

[0001] This invention relates to the field of visual processing technology, and in particular to a method, apparatus, device, and storage medium for digitizing paper information. Background Technology

[0002] Terminal block wiring diagrams are an important basis for the construction and maintenance of substation secondary cabinets. As the level of digitalization and intelligence of power systems continues to improve, substation wiring construction diagrams also need to be digitalized to provide a reference for subsequent intelligent and digital operation and maintenance.

[0003] Currently, AI-based visual recognition technologies (such as the YOLO object detection algorithm) are developing rapidly. When applied to the recognition of substation wiring diagrams, they can automatically extract electrical component information, connection relationships, and text annotations from the drawings, providing an efficient intelligent auxiliary tool for the digitization of wiring diagrams.

[0004] However, existing visual recognition algorithms such as YOLO have a shallow structure and relatively simple layers, lacking almost any image understanding and reasoning capabilities. When recognizing drawings using these algorithms, their performance is highly dependent on the drawing format of the training dataset. Furthermore, different design institutes often have different design habits when developing terminal block wiring diagrams, resulting in stylistically diverse drawings even under the same drafting standard. Therefore, when faced with a variety of stylistically different drawings, existing visual recognition algorithms are limited by the fixed format of the training dataset, leading to low recognition efficiency and accuracy, and weak versatility. Summary of the Invention

[0005] This invention provides a method, apparatus, device, and storage medium for digitizing paper information, in order to solve the problems of poor versatility, low efficiency, and low accuracy of existing visual recognition algorithms when faced with drawings of different styles.

[0006] In a first aspect, embodiments of the present invention provide a method for digitizing paper information, the method comprising:

[0007] A target image containing paper content is identified, wherein the paper content is terminal block wiring content, and the target image includes multiple independent content areas, each of which is divided based on region feature identification;

[0008] Based on the set wiring diagram parsing rules and parsing prompts, key graphic elements are retrieved for each independent content area in the target image to obtain the graphic element position and connection information of each key graphic element.

[0009] Based on the position of each graphic element, character recognition is performed on the corresponding key graphic elements to obtain the recognition results of each key graphic element;

[0010] The recognition results are processed in a structured manner, and a digital file in a set format corresponding to the target image is generated based on the structured processing results and the connection information.

[0011] Secondly, embodiments of the present invention provide a device for digitizing paper information, the device comprising:

[0012] The target image determination module is used to determine a target image containing paper content, wherein the paper content is terminal block wiring content, and the target image includes multiple independent content areas, each of which is divided based on region feature recognition.

[0013] The key graphic element retrieval module is used to retrieve key graphic elements in each independent content area of ​​the target image according to the set wiring diagram parsing rules and parsing prompt information, and obtain the graphic element position and connection information of each key graphic element;

[0014] The character recognition module is used to perform character recognition on the corresponding key graphic elements according to the position of each graphic element, and obtain the recognition result of each key graphic element;

[0015] The structured processing module is used to perform structured processing on each of the recognition results, and generate a digital file in a set format corresponding to the target image based on the structured processing results and the connection information.

[0016] Thirdly, embodiments of the present invention provide an electronic device, the electronic device comprising:

[0017] At least one processor;

[0018] and a memory communicatively connected to the at least one processor;

[0019] The memory stores a computer program that can be executed by the at least one processor, which enables the at least one processor to perform the method for digitizing paper content according to any embodiment of the present invention.

[0020] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing computer instructions that, when executed by a processor, implement the method for digitizing paper content as described in any embodiment of the present invention.

[0021] The technical solution of this invention involves determining a target image containing paper-based content, specifically terminal block wiring information. The target image comprises multiple independent content areas, each defined based on regional feature recognition. According to predefined wiring diagram parsing rules and parsing prompts, key graphic elements are retrieved from each independent content area of ​​the target image to obtain the position and connection information of each key graphic element. Based on the position of each graphic element, character recognition is performed on the corresponding key graphic element to obtain the recognition result. The recognition results are then structured, and a digital file in a predefined format corresponding to the target image is generated based on the structured processing results and the connection information. This method improves the efficiency and accuracy of subsequent operations such as key element retrieval and character recognition by identifying target images comprising multiple independent content regions. By retrieving key elements according to wiring diagram parsing rules and parsing prompts, it achieves adaptive parsing of diverse paper content of different styles and formats without large-scale pre-training, and enhances the efficiency and accuracy of key element location. Character recognition of corresponding key elements based on their positions further improves the accuracy, reliability, and efficiency of paper content recognition. By structuring the recognition results and generating a digital file in a specified format corresponding to the target image based on the structuring results and connection information, the accuracy and efficiency of drawing digitization are improved, providing strong support for subsequent data analysis and applications.

[0022] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 A flowchart illustrating a method for digitizing paper content provided in an embodiment of the present invention;

[0025] Figure 2 A schematic diagram of the structure of a device for digitizing paper content provided in an embodiment of the present invention;

[0026] Figure 3 A schematic diagram of an electronic device that can be used to implement embodiments of the present invention is shown. Detailed Implementation

[0027] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0028] It should be noted that the terms "original," "target," etc., used in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0029] This invention provides a method for digitizing paper content. Figure 1 This is a flowchart of a method for digitizing paper content provided by an embodiment of the present invention. The embodiment of the present invention is applicable to scenarios where drawing content is digitized. The method can be executed by a paper content digitization device, which can be implemented in the form of software and / or hardware. Optionally, it can be implemented by an electronic device, preferably a mobile terminal, desktop computer, laptop computer, or server.

[0030] like Figure 1 As shown, the method for digitizing paper content provided in this embodiment of the invention may specifically include:

[0031] S101. Determine the target image containing paper content, where the paper content is terminal block wiring content. The target image includes multiple independent content areas, and each independent content area is divided based on area feature recognition.

[0032] The target image can be understood as an image consisting of multiple independent content regions obtained after preprocessing and content segmentation of the original image.

[0033] In this embodiment, the original image containing paper content as terminal block wiring can be preprocessed first. The preprocessing can include image enhancement, noise removal, and format conversion to ensure that the drawing quality is suitable for subsequent processing. Then, the preprocessed image is subjected to region feature recognition. Based on the recognition results, the original image is divided into multiple independent content regions, and the image containing multiple independent content regions is determined as the target image.

[0034] S102. Based on the set wiring diagram parsing rules and parsing prompts, perform key element retrieval on each independent content area in the target image to obtain the element position and connection information of each key element.

[0035] In this context, key primitives can be understood as crucial elements in the drawing, such as cables and terminal blocks. Parsing prompts can be understood as information used to guide the model through natural language, helping it better understand and perform the parsing task. Primitive location can be considered the position of the primitive within the target image.

[0036] In this embodiment, the parsing prompt information may include information such as drawing style, drawing specifications, and element feature descriptions. The parsing prompt information can be determined based on the editing operation detected when an editing operation is performed on the editing area of ​​the current interface. The editing operation can be user-inputted parsing prompt information, or it can be a preset parsing prompt information selected in a drop-down box, etc. This embodiment of the invention does not impose any limitations on this.

[0037] It is understandable that the location and symbolic features of elements such as terminal blocks and cables are subject to specific drawing specifications, naming requirements, and design conventions.

[0038] Therefore, in this embodiment, a knowledge base can be pre-built to provide pre-defined wiring diagram parsing rules, which can elaborate on the location and symbol features of key graphic elements. Visual features are extracted from each independent content area in the target image using a visual language model combined with the wiring diagram parsing rules, thereby obtaining content marker features. Then, the position, state, and drafting style of each independent content area can be parsed using a visual language model combined with pre-defined parsing prompts to obtain corresponding parsing results. Finally, the parsing results are semantically aligned and matched with the corresponding content marker features using a visual language model to obtain the located key graphic elements and their positions. Based on the connection logic formed by the cables in the key graphic elements, the connection information of the key graphic elements can be determined.

[0039] S103. Based on the position of each graphic element, perform character recognition on the corresponding key graphic elements to obtain the recognition results of each key graphic element.

[0040] The recognition result can be considered as the text annotation content corresponding to the identified key graphic elements. The text annotation content can be understood as the textual annotation content used to explain the meaning, function and connection relationship of the graphic elements.

[0041] In this embodiment, character recognition can be performed on the electrical symbols and label text of key graphic elements at each graphic element location based on a character recognition method or character recognition model, thereby obtaining the recognition result of the key graphic elements.

[0042] S104. Perform structured processing on each recognition result, and generate a digital file in the set format corresponding to the target image based on the structured processing result and each connection information.

[0043] In this embodiment, the recognition results can be structured using a model with cross-modal generation capabilities. Based on the structured processing results and the connection information of each key graphic element, a digital file in a specific format corresponding to the target image is generated. The digital file in the specified format can be a JSON file, an XML file, a YAML file, or a CSV file.

[0044] The above-described technical solution in this embodiment involves identifying a target image containing paper-based content, specifically terminal block wiring information. The target image comprises multiple independent content areas, each defined based on regional feature recognition. According to predefined wiring diagram parsing rules and parsing prompts, key graphic elements are retrieved from each independent content area of ​​the target image to obtain the position and connection information of each key graphic element. Based on the position of each graphic element, character recognition is performed on the corresponding key graphic element to obtain the recognition result. The recognition results are then structured, and a digital file in a predefined format corresponding to the target image is generated based on the structured processing results and the connection information. This method improves the efficiency and accuracy of subsequent operations such as key element retrieval and character recognition by identifying target images comprising multiple independent content regions. By retrieving key elements according to wiring diagram parsing rules and parsing prompts, it achieves adaptive parsing of diverse paper content of different styles and formats without large-scale pre-training, and enhances the efficiency and accuracy of key element location. Character recognition of corresponding key elements based on their positions further improves the accuracy, reliability, and efficiency of paper content recognition. By structuring the recognition results and generating a digital file in a specified format corresponding to the target image based on the structuring results and connection information, the accuracy and efficiency of drawing digitization are improved, providing strong support for subsequent data analysis and applications.

[0045] As a first optional embodiment of the present invention, based on the above embodiments, the steps of determining the target image containing paper content can be optimized as follows:

[0046] a1) Obtain the original image containing paper content as terminal block wiring.

[0047] The original image can be understood as an image that has not undergone segmentation processing. The original image can be in PDF, JPEG, or PNG format, etc.

[0048] In this embodiment, the method for acquiring the original image containing the paper content as terminal block wiring can be as follows: if an editing operation on the editing area of ​​the current interface is detected, the original image containing the paper content as terminal block wiring is determined based on the editing operation. The editing operation can be a user input operation or a selection operation in a drop-down box, etc. Alternatively, the method for acquiring the original image containing the paper content as terminal block wiring can be as follows: if a triggering of the camera control on the current interface is detected and camera permission is obtained, the image acquisition device is controlled to acquire the original image containing the paper content as terminal block wiring in real time.

[0049] b1) Perform grayscale and binarization processing on the original image to obtain the processed binarized image.

[0050] In this embodiment, the original image is subjected to grayscale and binarization conversion to obtain a processed binarized image, which provides support for subsequent targeted enhancement analysis based on the primitive features in the wiring diagram.

[0051] c1) The binarized image is subjected to region feature recognition by a threshold conversion algorithm, the binarized image is divided into multiple independent content regions, and the binarized image containing each of the independent content regions is determined as the target image.

[0052] In this embodiment, a threshold conversion algorithm can be used to identify connected regions and boundary contours in the binarized image to detect independent regions and extract region features such as location, area, and shape. Based on the region feature recognition results, the binarized image is divided into multiple non-overlapping independent content regions, which can be understood as the smallest unit that can be processed independently. The binarized image containing each independent content region is then determined as the target image.

[0053] The above-described technical solution in this embodiment performs grayscale and binarization processing on the original image containing paper-based terminal block wiring to obtain a processed binarized image. A threshold conversion algorithm is then used to achieve regional feature recognition, thereby dividing the binarized image into multiple independent content regions. This reduces background interference and provides data support for subsequent parallel processing. At the same time, the features within the divided independent content regions are relatively simple, resulting in higher accuracy in feature extraction and analysis. Furthermore, this segmentation method does not rely on instance training and can adapt to diverse original images.

[0054] As a second optional embodiment of the present invention, the step of performing key element retrieval on each independent content region in the target image according to the set wiring diagram parsing rules and parsing prompt information to obtain the element position and connection information of each key element can be specified as the following steps:

[0055] a2) Using the encoder of the visual language model and the wiring diagram parsing rules provided in the knowledge base, visual features are extracted from each independent content region in the target image to obtain the corresponding content marking features.

[0056] The wiring diagram parsing rules can include standardized element feature descriptions, such as terminal block graphic topology and cable numbering rules.

[0057] In this embodiment, the encoder embedded in the visual language model can be a visual feature encoder. Combining the wiring diagram parsing rules provided in the knowledge base, the visual feature encoder tracks pixel features (such as line segment endpoint coordinates and symbol outlines) on the wiring path, identifies pixel change areas, and thus locates and extracts the content marking features of each independent content area in the target image. The content marking features may include cable labels, terminal block numbers, circuit numbers, terminal numbers, short-circuit marks, and jumper marks.

[0058] b2) Through the feature analysis layer of the visual language model, combined with the preset parsing prompts, the element position, state, and drawing style of each independent content area are parsed to obtain the corresponding parsing results.

[0059] It should be noted that when the terminal blocks within an independent content area have a multi-layered structure, the feature analysis layer of the visual language model can identify and parse the connection relationships of the multi-layered terminal blocks layer by layer. During this process, layer omissions can easily occur. To avoid this problem, layer-by-layer identification guidance information can be pre-designed in the parsing prompts. Furthermore, guidance information related to feature localization logic can also be preset in the parsing prompts to more accurately locate the features of elements within the independent content area when dealing with paper content of different styles.

[0060] In this embodiment, by combining the feature analysis layer of the visual language model with preset parsing prompts, the feature matching strategy of the corresponding drawing specification can be dynamically activated through the parsing prompts. The feature analysis layer detects and locates the graphic elements in each independent content area according to the feature matching strategy, and parses the position, state and drawing specification style of the corresponding graphic elements, thereby obtaining the corresponding parsing results of each graphic element.

[0061] c2) Through the feature integration layer in the visual language model, the parsing results are semantically aligned and matched with the corresponding content tag features to obtain the located terminal block elements and cables.

[0062] In this embodiment, in order to ensure the accuracy of locating key graphic elements, the feature integration layer in the visual language model can be used to semantically align and match each parsing result with the corresponding content tag features, thereby obtaining the accurately located terminal block elements and cables.

[0063] d2) Identify each of the terminal block elements and cables as key graphic elements and obtain the corresponding graphic element positions, and obtain the connection information of each of the key graphic elements based on the connection logic of the cable structure.

[0064] In this embodiment, terminal block elements and cables are identified as key graphic elements, and the graphic element positions of the key graphic elements are obtained. At the same time, the connection information of each key graphic element is obtained based on the connection logic formed by the cables.

[0065] The above-described technical solution in this embodiment does not require large-scale pre-training of the visual language model. When processing multi-style paper content, the feature matching strategy of the corresponding drawing specification can be dynamically activated by parsing the prompt information. Combined with the wiring diagram parsing rules provided in the knowledge base, key graphic elements can be accurately located, reducing the dependence on the training dataset. Furthermore, through the feature integration layer in the visual language model, the parsing results are semantically aligned and matched with the corresponding content label features, achieving fully automated parsing and matching, reducing manual intervention, and further improving the efficiency of locating key graphic element features.

[0066] As a third optional embodiment of the present invention, the step of performing character recognition on the corresponding key graphic elements according to the position of each graphic element to obtain the recognition result of each key graphic element can be specified as the following steps:

[0067] a3) Using a character recognition model, a cross-modal attention mechanism and an optical character recognition mechanism are employed to perform character recognition on key primitives at each of the said primitive positions, obtain the text annotation content corresponding to each of the said key primitives, and determine the text annotation content as the corresponding recognition result.

[0068] Among them, the character recognition model can be understood as a trained model with character recognition capabilities, which can be a model based on a multimodal large language model.

[0069] In this embodiment, a character recognition model is used to perform character recognition on key primitives at each primitive location using a cross-modal attention mechanism and an optical character recognition mechanism. The recognized electrical symbol graphic features can be jointly encoded with text labels to obtain the text annotation content corresponding to each key primitive, and each text annotation content is determined as the recognition result of the corresponding key primitive.

[0070] The above-described technical solution in this embodiment uses a character recognition model, employing a cross-modal attention mechanism and an optical character recognition mechanism, to perform character recognition on key graphic elements at each of the graphic element locations. This accurately and efficiently clarifies the text annotation content of the key graphic elements, providing support for subsequent digitization of paper content.

[0071] As a fourth optional embodiment of the present invention, the method further includes:

[0072] a4) Based on the industry design standards for terminal block wiring and character recognition knowledge, the recognition results of each key graphic element are identified and corrected.

[0073] In this embodiment, the industry design standard information may include industry design standard specifications and design practices of various design institutes, and the character recognition knowledge may include easily confused characters. The industry design standard information for terminal block wiring is used to verify the rationality of the text annotations in the recognition results of each key graphic element. Combined with easily confused characters from the character recognition knowledge, and based on the established correction rules, the content of the recognition deviation is corrected.

[0074] As one implementation method, the recognition results of each key graphic element can be identified and corrected based on the industry design standard information of terminal block wiring and character recognition knowledge, which can be further specified as follows:

[0075] a41) Extract the naming information of each key graphic element from the industry design standard information of terminal block wiring, and use the naming information to correct the naming of each text annotation content as the recognition result.

[0076] In this embodiment, one possible approach is to use a fuzzy matching algorithm to calculate the matching degree between the naming information of each key graphic element extracted from the industry design standard information of the terminal block wiring and the text annotation content in each recognition result. If the matching degree falls within the first threshold range (e.g., [90%, 100%]), no naming correction is required for the text annotation content; if the matching degree falls within the second threshold range (e.g., [50%, 90%)), semantic completion and correction are performed on the text annotation content based on the corresponding information in the industry design standard information, such as terminal block naming rules and circuit number naming conventions; if the matching degree falls within the third threshold range (e.g., [0%, 50%)), an alarm message for the matching is triggered, prompting manual verification. The alarm message can be in the form of text, voice, ringing, or light illumination, and can be displayed via SMS, email, or pop-up window.

[0077] a42) Using easily confused character information from character recognition knowledge, determine the recognition deviation content in the recognition result.

[0078] It is understandable that some characters are easily misidentified due to similar shapes or printing quality issues; these characters can be considered easily confused characters. Easily confused character information can be understood as information containing easily confused characters, which can be pre-summarized based on historical data and experience. For example, easily confused character information may include the English character O and the number 0, the English character B and the number 8, etc.

[0079] In this embodiment, each character in the recognition result can be compared with the easily confused character information. If a character in the recognition result matches a character in the easily confused character pair, that character can be marked as a possible deviation.

[0080] For example, the recognition result is "1234S", where "S" and "5" are easily confused characters, and "S" can be marked as deviation content.

[0081] a43) Based on the set correction rules, the identification deviation content is corrected to obtain the identification results after naming correction and deviation correction.

[0082] For example, the set correction rule may include if the recognition result is "1234S", "S" is marked as deviation content, and the context indicates that it should be the number "5", then "S" is corrected to "5" to obtain the deviation-corrected recognition result "12345".

[0083] The above-described technical solution in this embodiment corrects the naming of each text label by using the naming information in the industry design standard information of terminal block wiring, and determines the recognition deviation content in the recognition result by using easily confused character information in character recognition knowledge. Based on the set correction rules, the recognition deviation content is corrected, ensuring the accuracy and reliability of the key graphic element recognition results. This further improves the adaptability and universality of paper content with different design styles, and provides strong support for the subsequent promotion of paper content digitization.

[0084] b4) Generate a traceability log based on the identification and correction results, and update the identification results of each key graphic element after correction to the identification results.

[0085] In this embodiment, a traceability log can be generated for the identification and / or correction part based on the identification and correction results to prompt manual verification, and the identification results of each key graphic element after automatic or manual correction can be updated to the identification results.

[0086] The above-described technical solution in this embodiment corrects the recognition results of each key graphic element based on industry design standards for terminal block wiring and character recognition knowledge. It generates a traceability log based on the recognition correction results and updates the recognition results using the corrected recognition results of each key graphic element. This ensures the accuracy and reliability of the recognition results and further guarantees that the digitization method for paper content provided in this embodiment of the invention is universally applicable to paper content with different design styles.

[0087] As a fifth optional embodiment of the present invention, the set format digital file is a JSON format file. The JSON format file presents a three-dimensional data structure, including the logical relationship between the terminal blocks and cables contained in the paper content, as well as the equipment level of the involved equipment and metadata information.

[0088] Understandably, JSON is a lightweight data exchange format that is easy to read and write, as well as easy for machines to parse and generate. It can be used to store and transmit data and supports nested structures, making it suitable for representing hierarchical data.

[0089] For example, the logical relationship between terminal blocks and cables can be used to describe the connection relationship between terminal blocks and cables, including terminal block number, cable label, loop number, and terminal number, etc.; device hierarchy can be used to describe the hierarchical relationship of devices, such as system, subsystem, and component, etc.; metadata information can be used to describe additional information about the identification process, such as identification time, algorithm, confidence level, etc.

[0090] The above-described technical solution in this embodiment improves the readability and operability of the data by setting the formatted digital file to a JSON format file, and presents a three-dimensional data structure, including the logical relationship between the terminal blocks and cables contained in the paper content, as well as the equipment level of the involved equipment and metadata information, which facilitates subsequent data processing, analysis and application logic execution.

[0091] In one optional embodiment, the paper content digitization method provided by the present invention can be applied to a multi-agent system, which may include a preprocessing agent, a drawing design knowledge base agent, an element detection and positioning agent, a character recognition agent, an image reasoning and verification agent, and a digitization agent.

[0092] The preprocessing agent can be used to identify target images containing paper-based content. Specifically, the original image containing terminal block wiring is subjected to grayscale and binarization processing to obtain a processed binarized image. The binarized image is then divided into multiple independent content regions, and the image containing multiple independent content regions is identified as the target image.

[0093] The element detection and localization agent can be used to retrieve key graphic elements from each independent content region in the target image according to the set wiring diagram parsing rules and parsing prompts, thereby obtaining the graphic element position and connection information of each key graphic element. Specifically, firstly, based on the understanding of the wiring diagram parsing rules by the drawing design knowledge base agent, an encoder based on the large visual language model embedding performs visual feature extraction on each independent content region in the target image. Specifically, the encoder can locate elements such as cable labels, terminal block numbers, circuit numbers, terminal numbers, short-circuit marks, and jumper marks by tracking path pixel features (such as line segment endpoint coordinates and symbol contours), and extract features from the located elements to obtain the corresponding content marking features.

[0094] Next, the feature analysis layer of the visual language model, with the help of preset parsing prompts provided by the prompt word project, can analyze the element position, state, and graphic style of different elements in each independent content area to obtain the corresponding parsing results. Then, through the feature integration layer in the visual language model, the parsing results are semantically aligned and matched with the corresponding content marker features to obtain the located terminal block elements and cables. Each terminal block element and cable is identified as a key graphic element, and its corresponding graphic element position is obtained. Based on the connection logic of the cable structure, the connection information of each key graphic element is obtained. Combining feature extraction and pixel tracking, the intelligent agent can analyze the connection relationship of multiple terminal blocks layer by layer against interference, avoiding layer omissions and improving positioning accuracy. It achieves fully automated parsing and matching, reduces manual intervention, and improves the positioning efficiency of terminal block features and cable features.

[0095] The character recognition agent can be used to perform character recognition on corresponding key graphic elements based on their positions, and obtain the recognition results for each key graphic element. Based on a multimodal large language pre-trained model, cross-modal attention mechanism and optical character recognition mechanism are used to perform character recognition on key graphic elements at their respective positions. The graphic features of electrical symbols are jointly encoded with text labels to initially clarify the text annotation content of key graphic elements.

[0096] The image reasoning verification agent can correct the recognition results of each key graphic element based on industry design standard information and character recognition knowledge provided by the drawing design knowledge base agent. This method uses industry design standard information as a benchmark to verify the rationality of the recognition results. By combining easily confused characters from character recognition knowledge and based on set correction rules, it corrects recognition deviations (such as characters) in the recognition results, obtaining both naming-corrected and deviation-corrected recognition results. This method further ensures the universality of the proposed digitization method for paper content, applicable to paper content including terminal block wiring in different design styles.

[0097] The digital intelligent agent performs structured processing on the recognition results and generates a digital file in a predetermined format corresponding to the target image based on the structured processing results and the connection information. This intelligent agent, through the cross-modal generation capability of a large model, converts the recognition results processed by the agent into a digital file in a predetermined format, such as a JSON file conforming to a specific standard. The JSON file presents a three-dimensional data structure, including the logical relationships between terminal blocks and cables contained in the paper content, as well as the equipment hierarchy and metadata information of the involved devices. The digital intelligent agent can also generate traceable log records for the verification and error correction parts, prompting manual verification.

[0098] In this embodiment, through the cooperation of multiple intelligent agents, the naming logic of terminal blocks and cables in paper content containing terminal block wiring is parsed, which can quickly adapt to changes in paper content, improve the recognition ability of key elements in drawings, and accelerate the digitization process of paper content.

[0099] Figure 2 This is a schematic diagram of the structure of a device for digitizing paper content provided in an embodiment of the present invention. Figure 2 As shown, the device includes: a target image determination module 21, a key primitive retrieval module 22, a character recognition module 23, and a structured processing module 24, wherein...

[0100] The target image determination module 21 is used to determine a target image containing paper content, wherein the paper content is terminal block wiring content, and the target image includes multiple independent content areas, each of which is divided based on region feature recognition.

[0101] The key graphic element retrieval module 22 is used to perform key graphic element retrieval on each independent content area in the target image according to the set wiring diagram parsing rules and parsing prompt information, and obtain the graphic element position and connection information of each key graphic element;

[0102] The character recognition module 23 is used to perform character recognition on the corresponding key graphic elements according to the position of each graphic element, and obtain the recognition result of each key graphic element;

[0103] The structured processing module 24 is used to perform structured processing on each of the recognition results, and generate a digital file in a set format corresponding to the target image based on the structured processing results and the connection information.

[0104] The above-described technical solution in this embodiment involves identifying a target image containing paper-based content, specifically terminal block wiring information. The target image comprises multiple independent content areas, each defined based on regional feature recognition. According to predefined wiring diagram parsing rules and parsing prompts, key graphic elements are retrieved from each independent content area of ​​the target image to obtain the position and connection information of each key graphic element. Based on the position of each graphic element, character recognition is performed on the corresponding key graphic element to obtain the recognition result. The recognition results are then structured, and a digital file in a predefined format corresponding to the target image is generated based on the structured processing results and the connection information. This method improves the efficiency and accuracy of subsequent operations such as key element retrieval and character recognition by identifying target images comprising multiple independent content regions. By retrieving key elements according to wiring diagram parsing rules and parsing prompts, it achieves adaptive parsing of diverse paper content of different styles and formats without large-scale pre-training, and enhances the efficiency and accuracy of key element location. Character recognition of corresponding key elements based on their positions further improves the accuracy, reliability, and efficiency of paper content recognition. By structuring the recognition results and generating a digital file in a specified format corresponding to the target image based on the structuring results and connection information, the accuracy and efficiency of drawing digitization are improved, providing strong support for subsequent data analysis and applications.

[0105] Furthermore, the target image determination module 21 can specifically be used for:

[0106] Obtain the original image containing paper-based information about terminal block wiring;

[0107] The original image is subjected to grayscale and binarization processing to obtain a processed binarized image;

[0108] The binarized image is divided into multiple independent content regions by using a threshold conversion algorithm to identify regional features. The binarized image containing each of the independent content regions is then identified as the target image.

[0109] Furthermore, the key primitive retrieval module 22 can be specifically used for:

[0110] By using the encoder of the visual language model and combining the wiring diagram parsing rules provided in the knowledge base, visual features are extracted from each independent content region in the target image to obtain the corresponding content marking features.

[0111] By using the feature analysis layer of the visual language model and combining it with preset parsing prompts, the element position, state, and drawing style of each independent content area are parsed to obtain the corresponding parsing results.

[0112] By using the feature integration layer in the visual language model, the parsing results are semantically aligned and matched with the corresponding content tag features to obtain the located terminal block elements and cables;

[0113] Each terminal block element and cable is identified as a key graphic element and its corresponding position is obtained. Connection information of each key graphic element is obtained based on the connection logic formed by the cable.

[0114] Furthermore, the character recognition module 23 can be specifically used for:

[0115] Using a character recognition model, a cross-modal attention mechanism and an optical character recognition mechanism are employed to perform character recognition on key primitives at each of the stated primitive positions, obtain the text annotation content corresponding to each of the stated key primitives, and determine the text annotation content as the corresponding recognition result.

[0116] Furthermore, the device also includes a calibration module, which specifically may include:

[0117] The identification and correction unit is used to identify and correct the identification results of each key graphic element based on industry design standard information for terminal block wiring and character recognition knowledge.

[0118] The log generation and result update unit is used to generate a traceability log based on the identification and correction results, and to update the identification results of each key graphic element after correction to the identification results.

[0119] Furthermore, the identification and correction unit can specifically be used for:

[0120] The naming information of each key graphic element is extracted from the industry design standard information of terminal block wiring, and the naming information is used to correct the naming of each text annotation content as the recognition result.

[0121] By using easily confused character information from character recognition knowledge, the recognition deviation content in the recognition result is determined;

[0122] Based on the established correction rules, the identification deviations are corrected to obtain the identification results after naming correction and deviation correction.

[0123] Furthermore, the set format digital file is a JSON format file. The JSON format file presents a three-dimensional data structure, which includes the logical relationship between the terminal blocks and cables contained in the paper content, as well as the equipment level of the involved equipment and metadata information.

[0124] The paper content digitization device provided in the embodiments of the present invention can execute the paper content digitization method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of executing the method.

[0125] Figure 3 A schematic diagram of an electronic device 30 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0126] like Figure 3 As shown, the electronic device 30 includes at least one processor 31 and a memory, such as a read-only memory (ROM) 32 or a random access memory (RAM) 33, communicatively connected to the at least one processor 31. The memory stores computer programs executable by the at least one processor. The processor 31 can perform various appropriate actions and processes based on the computer program stored in the ROM 32 or loaded from storage unit 38 into the RAM 33. The RAM 33 can also store various programs and data required for the operation of the electronic device 30. The processor 31, ROM 32, and RAM 33 are interconnected via a bus 34. An input / output (I / O) interface 35 is also connected to the bus 34.

[0127] Multiple components in electronic device 30 are connected to I / O interface 35, including: input unit 36, such as keyboard, mouse, etc.; output unit 37, such as various types of monitors, speakers, etc.; storage unit 38, such as disk, optical disk, etc.; and communication unit 39, such as network card, modem, wireless transceiver, etc. Communication unit 39 allows electronic device 30 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0128] Processor 31 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 31 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 31 performs the various methods and processes described above, such as methods for digitizing paper content.

[0129] In some embodiments, the method for digitizing paper content may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 38. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 30 via ROM 32 and / or communication unit 39. When the computer program is loaded into RAM 33 and executed by processor 31, one or more steps of the method for digitizing paper content described above may be performed. Alternatively, in other embodiments, processor 31 may be configured to perform the method for digitizing paper content by any other suitable means (e.g., by means of firmware).

[0130] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0131] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0132] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0133] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0134] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0135] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through a communication network. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0136] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0137] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for digitizing paper content, characterized in that, include: A target image containing paper content is identified, wherein the paper content is terminal block wiring content, and the target image includes multiple independent content areas, each of which is divided based on region feature identification; Based on the set wiring diagram parsing rules and parsing prompts, key graphic elements are retrieved for each independent content area in the target image to obtain the graphic element position and connection information of each key graphic element. Based on the position of each graphic element, character recognition is performed on the corresponding key graphic elements to obtain the recognition results of each key graphic element; The recognition results are processed in a structured manner, and a digital file in a set format corresponding to the target image is generated based on the structured processing results and the connection information.

2. The method according to claim 1, characterized in that, The determination of the target image containing paper content includes: Obtain the original image containing paper-based information about terminal block wiring; The original image is subjected to grayscale and binarization processing to obtain a processed binarized image; The binarized image is divided into multiple independent content regions by using a threshold conversion algorithm to identify regional features. The binarized image containing each of the independent content regions is then identified as the target image.

3. The method according to claim 1, characterized in that, The step of retrieving key graphic elements from each independent content region in the target image according to the set wiring diagram parsing rules and parsing prompts, and obtaining the graphic element position and connection information of each key graphic element, includes: By using the encoder of the visual language model and combining the wiring diagram parsing rules provided in the knowledge base, visual features are extracted from each independent content region in the target image to obtain the corresponding content marking features. By using the feature analysis layer of the visual language model and combining it with preset parsing prompts, the element position, state, and drawing style of each independent content area are parsed to obtain the corresponding parsing results. By using the feature integration layer in the visual language model, the parsing results are semantically aligned and matched with the corresponding content tag features to obtain the located terminal block elements and cables; Each terminal block element and cable is identified as a key graphic element and its corresponding position is obtained. Connection information of each key graphic element is obtained based on the connection logic formed by the cable.

4. The method according to claim 1, characterized in that, The step of performing character recognition on corresponding key graphic elements based on the positions of each graphic element to obtain the recognition results of each key graphic element includes: Using a character recognition model, a cross-modal attention mechanism and an optical character recognition mechanism are employed to perform character recognition on key primitives at each of the stated primitive positions, obtain the text annotation content corresponding to each of the stated key primitives, and determine the text annotation content as the corresponding recognition result.

5. The method according to claim 1, characterized in that, Also includes: Based on industry design standards for terminal block wiring and character recognition knowledge, the recognition results of each key graphic element are identified and corrected. A traceability log is generated based on the identification and correction results, and the identification results of each key graphic element after correction are updated to the identification results.

6. The method according to claim 5, characterized in that, The process of identifying and correcting the recognition results of each key graphic element based on industry design standards for terminal block wiring and character recognition knowledge includes: The naming information of each key graphic element is extracted from the industry design standard information of terminal block wiring, and the naming information is used to correct the naming of each text annotation content as the recognition result. By using easily confused character information from character recognition knowledge, the recognition deviation content in the recognition result is determined; Based on the established correction rules, the identification deviations are corrected to obtain the identification results after naming correction and deviation correction.

7. The method according to claim 1, characterized in that, The specified format digital file is a JSON format file. The JSON format file presents a three-dimensional data structure, which includes the logical relationship between the terminal blocks and cables contained in the paper content, as well as the equipment level of the involved equipment and metadata information.

8. A device for digitizing paper content, characterized in that, include: The target image determination module is used to determine a target image containing paper content, wherein the paper content is terminal block wiring content, and the target image includes multiple independent content areas, each of which is divided based on region feature recognition. The key graphic element retrieval module is used to retrieve key graphic elements in each independent content area of ​​the target image according to the set wiring diagram parsing rules and parsing prompt information, and obtain the graphic element position and connection information of each key graphic element; The character recognition module is used to perform character recognition on the corresponding key graphic elements according to the position of each graphic element, and obtain the recognition result of each key graphic element; The structured processing module is used to perform structured processing on each of the recognition results, and generate a digital file in a set format corresponding to the target image based on the structured processing results and the connection information.

9. An electronic device, characterized in that, The electronic device includes: At least one processor; and a memory communicatively connected to the at least one processor; The memory stores a computer program that can be executed by the at least one processor, which enables the at least one processor to perform the method for digitizing paper content according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed by a processor, implement the method for digitizing paper content as described in any one of claims 1-7.

Images