Image information extraction method and device, electronic equipment and storage medium

Abstract

The invention discloses an image information extraction method and device, electronic equipment and a storage medium. The method comprises the steps that target layer segmentation is conducted on a drawing image, at least one target layer is obtained, and the target layers comprise entities of the same type; for each target layer, extracting a target entity in the target layer; performing semantic segmentation on each target entity to obtain at least one material entity in the drawing image; and determining at least one circuit device and corresponding device parameters according to the position of each material entity in the drawing image. According to the technical scheme provided by the embodiment of the invention, the drawing recognition accuracy and efficiency can be improved, and the recognized information is complete.

Description

Technical Field

[0001] The present invention relates to the field of data processing, and in particular to an image information extraction method, device, electronic equipment and storage medium. Background Art

[0002] The power grid industry needs to process a large number of drawings during design and operation and maintenance, such as substation electrical wiring diagrams and equipment layout diagrams. These drawings contain key equipment parameters, such as circuit breaker models and insulator rated voltages.

[0003] Currently, the task of identifying information in drawings primarily relies on manual interpretation of drawings and the manual entry of parameters into the system. However, this approach presents the following problems: Parsing a single complex drawing takes a long time, and manual entry has a high error rate; unstructured drawings contain a variety of annotation formats, such as text annotations, symbol codes, and dimension line annotations, making them difficult to parse uniformly; and changes to equipment parameters require revising drawings and records, resulting in inefficient collaborative adjustments. Summary of the Invention

[0004] The present invention provides an image information extraction method, device, electronic equipment and storage medium, which can improve the accuracy and efficiency of drawing recognition and make the recognized information complete.

[0005] According to one aspect of the present invention, a method for extracting image information is provided, the method comprising:

[0006] Performing target layer segmentation on the drawing image to obtain at least one target layer, wherein the target layer includes entities of the same type;

[0007] For each target layer, extract the target entity in the target layer;

[0008] Performing semantic segmentation on each of the target entities to obtain at least one material entity in the drawing image;

[0009] At least one circuit device and corresponding device parameters are determined according to the position of each material entity in the drawing image.

[0010] According to another aspect of the present invention, there is provided an image information extraction device, the device comprising:

[0011] A target layer segmentation module is used to segment the drawing image into target layers to obtain at least one target layer, wherein the target layer includes entities of the same type;

[0012] An entity extraction module, configured to extract target entities from each target layer;

[0013] a semantic segmentation module, configured to perform semantic segmentation on each target entity to obtain at least one material entity in the drawing image;

[0014] The parameter identification module is used to determine at least one circuit device and corresponding device parameters according to the position of each material entity in the drawing image.

[0015] According to another aspect of the present invention, an electronic device is provided, comprising:

[0016] at least one processor; and

[0017] a memory communicatively connected to the at least one processor; wherein,

[0018] The memory stores a computer program that can be executed by the at least one processor. The computer program is executed by the at least one processor to enable the at least one processor to perform the image information extraction method described in any embodiment of the present invention.

[0019] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the image information extraction method according to any embodiment of the present invention when executed.

[0020] The technical solution of the embodiment of the present invention classifies entities in the drawing image, divides entities of the same type into the same target layer, and then performs semantic segmentation of entities on each target layer to obtain fine-grained material entities. Finally, based on the position of the material entity in the drawing image, the circuit device and the device parameters of each circuit device are determined. This can reduce the labor cost of image entity recognition, improve the efficiency of image recognition, and realize multi-stage and multi-level entity recognition, improve the accuracy of image entity recognition. By determining the device parameters associated with the entity based on the position of the entity in the image, the information of the circuit device can be further supplemented, the richness and completeness of the information of the circuit device can be increased, the accuracy of image analysis can be improved, and the information of the identified device can be updated in real time when the device parameters change, thereby improving the efficiency of collaborative adjustment. This solves the problems of low efficiency, poor accuracy, high difficulty in analysis and low efficiency of collaborative adjustment in manual drawing analysis in the prior art, can improve the efficiency and accuracy of device identification in the drawing, simplify the complexity of device information identification, and improve the real-time updating of device information.

[0021] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present invention, nor is it intended to limit the scope of the present invention. Other features of the present invention will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0023] Figure 1 is a flowchart of an image information extraction method provided according to an embodiment of the present invention;

[0024] Figure 2 is a flowchart of an image information extraction method provided according to an embodiment of the present invention;

[0025] Figure 3 is a structural diagram of an image information extraction device provided according to an embodiment of the present invention;

[0026] Figure 4 It is a structural diagram of an electronic device for implementing the image information extraction method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0027] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0028] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0029] Figure 1This is a flowchart of an image information extraction method provided by an embodiment of the present invention. This embodiment of the present invention is applicable to obtaining equipment and related information from image analysis of power grid drawings. The method can be performed by an image information extraction device, which can be implemented in hardware and / or software. The image information extraction device can be configured in an electronic device that carries the image information extraction function, such as a server or client device. Client devices may include mobile phones, tablet computers, laptops, and computers.

[0030] See also Figure 1 The image information extraction method shown includes:

[0031] S101: Segment a drawing image into target layers to obtain at least one target layer, where the target layers include entities of the same type.

[0032] The drawing image may refer to an image obtained by image acquisition of drawings generated in various life cycle stages of a power grid. In some embodiments, drawings may be generated during the power grid design stage and the power grid operation and maintenance stage. In some embodiments, the drawings may include substation electrical wiring diagrams and equipment layout drawings, etc. In some embodiments, material parameters may be extracted from the material model included in the drawing image. The material model may be a device and other devices mounted on the device, wherein the drawing image includes a cross-sectional view of the device, and the cross-sectional view includes the outline of the device. The drawing image may be in the format of a CAD (Computer Aided Design) image.

[0033] The target layer is used to classify different types of entities. Entities of the same type belong to the same target layer. Target layer segmentation can be understood as the preliminary identification of device areas within an image drawing. This identification step can be a coarse-grained recognition operation. Entities within a drawing image can refer to objects of a certain volume. In some embodiments, entities can include power grid equipment, connecting lines, and text. The drawing image can be parsed to separate different types of entities and form at least one target layer.

[0034] In one example, a drawing image can be segmented, for example, into devices and text. The text can be the title bar area in the lower right corner of the image drawing. The title bar and device areas can be divided into a circuit layer and an annotation layer. The circuit layer includes circuit devices and connecting wires, etc. The annotation layer can include text, etc. The circuit layer can be further subdivided into a device layer and a wire layer. The device layer can include circuit devices, and the wire layer can include connecting wires, etc. The device layer can be further subdivided, with different device types belonging to different device layers.

[0035] S102: For each target layer, extract the target entity in the target layer.

[0036] Extracting an entity from a drawing image may involve determining the area corresponding to the entity within the drawing image. For each target layer, each target entity is identified separately within the target layer. For text in an annotation layer, text within the same area may be identified as a single target entity. For example, text within a cell may be a single target entity. Extracting an entity from a target layer may involve device recognition, which may be a medium-granularity recognition operation.

[0037] In one example, the drawing image may be vectorized, and the target entity graphics in each target layer of the drawing image may be split and classified to obtain the target entities in each target layer.

[0038] S103: Perform semantic segmentation on each target entity to obtain at least one material entity in the drawing image.

[0039] Semantic segmentation of the target entity can include identifying the outline and type of the target entity. Furthermore, the target entity can be further subdivided within the region corresponding to the target entity. For example, connection points and lines within the target entity can be identified. A material entity can be the region corresponding to the target entity. The region of the target entity extracted in S102 is a coarse-grained region, while the region corresponding to the material entity obtained by segmentation in S103 is a fine-grained region. The region boundary of the material entity is the outline of the material entity.

[0040] In some embodiments, a pre-trained semantic segmentation model may be used to perform semantic segmentation on the target entity to obtain at least one material entity.

[0041] In an optional embodiment, semantic segmentation is performed on each of the target entities to obtain at least one material entity in the drawing image, including: screening each of the target entities, eliminating redundant entities, and obtaining at least one key entity; and semantic segmentation is performed on each of the key entities to obtain at least one material entity in the drawing image.

[0042] The target entities are screened to obtain redundant entities and key entities. Key entities are retained and redundant entities are eliminated. Redundant entities may refer to entities unrelated to the device. Key entities may refer to entities related to the device. Redundant entities may include reflective spots, creases in the drawing, and stains on the drawing, etc. generated during the image capture process of the drawing. In some embodiments, a drawing may be scanned to obtain a drawing image. During the scanning process, scattered light from other light sources may appear on the drawing. This scattered light interferes with the captured light, generating reflections and spots in the captured image, i.e., reflective spots. In some embodiments, drawings may be folded for storage or transportation, resulting in creases. Drawings may be stored or transported in adverse environments, causing contamination and stains. In other cases, elements that reduce the quality of the drawing image may be generated on the drawing image. These elements may interfere with the identification of material entities. These elements can be identified first and eliminated as redundant entities, improving the quality and effectiveness of the remaining key entities.

[0043] In some embodiments, key entities may include: device symbols (BLOCK), annotation text (MTEXT), and connecting lines (LINE), etc.

[0044] In one example, redundant entities can be identified based on rules, for example, by obtaining redundant features of preset redundant entities, screening entities corresponding to the redundant features from target entities as redundant entities, and determining target entities other than redundant entities as key entities.

[0045] It can be seen that by prioritizing redundant screening of entities before semantic segmentation, eliminating redundant entities, and performing semantic segmentation only on the retained key entities to obtain material entities, scanning stains, drawing creases and reflective spots can be eliminated, and irrelevant elements can be excluded, thereby reducing invalid data for semantic segmentation, reducing the amount of redundant processing data, improving entity recognition accuracy, and improving the efficiency of entity semantic segmentation.

[0046] In an optional embodiment, semantic segmentation is performed on each of the key entities to obtain at least one material entity in the drawing image, including: performing data enhancement on each of the key entities, the data enhancement including at least one of the following: geometric transformation, image quality enhancement and entity completion; and semantic segmentation is performed on the key entities after data enhancement to obtain at least one material entity in the drawing image.

[0047] Among them, geometric transformation may include rotation, scaling and shearing. Among them, rotation is used to adjust the angle of the key entity to a standard direction, reducing the entity recognition error introduced by the non-standard direction. Scaling is used to adjust the size of the key entity, increase the details of the key entity or normalize the key entity to an entity of the same size, reducing the error introduced by the size. Shearing is used to achieve the tilt deformation of the image by stretching or compressing the pixel distribution along a specific direction in the cross-section or profile view, so as to correct the tilt of the key entity angle. Image quality enhancement is used to improve the image quality of the key entity. In some embodiments, image quality enhancement may include enhancing resolution, optimizing contrast, optimizing brightness and suppressing noise, etc., wherein the noise suppression here may be non-significant noise, and the significant noise may be obvious creases, stains and spots (in the above example). Entity completion may refer to completing the missing content of the key entity, such as completing the outline and connecting lines of the entity. Entity completion can be achieved by using a difference method.

[0048] It can be seen that by pre-enhancing the data of key entities before semantic segmentation, the geometric figures of key entities can be adjusted so that the geometric figures of key entities are close to the standard figures. There is no need to establish corresponding model features and corresponding learning features for the same device with different shapes, different angles and different sizes, which reduces the device recognition cost, thereby improving the recognition accuracy, reducing the missed detection rate of key entities, enhancing image quality, optimizing entity edges and details, assisting in improving the accuracy of entity recognition edges, and reducing interference information unrelated to the entity, thereby improving the accuracy of entity semantic segmentation.

[0049] In an optional embodiment, semantic segmentation is performed on each of the target entities to obtain at least one material entity in the drawing image. Data enhancement can also be performed on each of the target entities, and the data enhancement includes at least one of the following: geometric transformation, image quality enhancement, and entity completion; semantic segmentation is performed on the target entity after data enhancement to obtain at least one material entity in the drawing image.

[0050] In an optional embodiment, the data enhancement includes entity completion; the data enhancement of each key entity includes: obtaining the connection lines within the preset spatial range of the key entity; in the case where the connection lines of the key entity within the preset spatial range are incomplete, obtaining the entity type corresponding to the key entity and the spatial distribution relationship between the key entity and the incomplete connection lines; completing the incomplete connection lines according to the entity type and the spatial distribution relationship.

[0051] Among them, entity completion may include connection line completion. The connection line may be a connection line connecting a key entity, for example, a connection line may connect a key entity with other key entities, or a connection line may connect a terminal in a key entity with another terminal, that is, a connection line connects two parts of the key entity itself. The preset spatial range may refer to the spatial range in which the outline of the key entity extends outward by a preset distance. For example, the preset distance is 1 cm. The existence of an incomplete connection line may mean that the connection line is discontinuous and there is a broken position. The entity type is used to determine the connection specifications of the key entity, specifically to determine whether there is a connection line for the key entity, and the connection object of the connection line. The spatial distribution relationship is used to determine whether the incomplete connection line is a connection line connecting the key entity. If it is determined that the incomplete connection line is connected to the key entity based on the entity type and the spatial distribution relationship, the incomplete connection line can be interpolated between the connection objects corresponding to the incomplete connection line according to the connection specifications of the connection line corresponding to the entity type to complete the incomplete connection line.

[0052] In some embodiments, it is possible to detect whether there are incomplete connection lines in the search area (e.g., a neighborhood of 5 mm) corresponding to the key entity based on the geometric characteristics of the connection line, such as the continuity of the line segment of the connection line, and the characteristics of the power grid, such as the circuit topology connection rules. In one example, based on the detection of the proximity of the endpoints, if the distance between the endpoints of the line segment is lower than a threshold, it is determined that there is an incomplete connection line between the two terminals, which can be completed by extending the line segment or interpolating. For example, based on the direction consistency check, the line segment slope difference of the two segments is calculated. If the difference is less than the set angle range (e.g., ±5°), it is considered to be a line segment with the same direction. The two line segments are incomplete connection lines and can be extended along the slope direction to connect the two line segments to complete the incomplete line segment. For example, based on the device type constraints in the power grid and the power grid topology logic, it is checked whether the line segment endpoints are connected to compliant devices. For example, the bus is only allowed to be connected to high-level devices such as circuit breakers or transformers. If there is no connection to a compliant device, it is determined that there is an incomplete connection line between the endpoint and the corresponding device, and a connection between the terminal and the corresponding device is established to complete the incomplete connection line. For example, if the endpoint of a broken line segment points to a load node but should actually connect to a transformer, there is an incomplete connection line between the transformer and the endpoint. Another example is loop integrity verification, which detects whether line segments form a closed loop or a radial structure. Broken line segments that cannot form a valid loop are identified as incomplete connection lines, and the corresponding equipment can be connected to complete the incomplete connection line. For example, a breadth-first search (BFS) is used to traverse the connection relationship and identify isolated nodes or broken branches.

[0053] In some embodiments, line segment logical connections are completed using an interpolation algorithm or a topology rule base. For example, if a discontinuous line segment is detected between two terminals of a circuit breaker, a connection path is automatically generated based on a standard spacing.

[0054] It can be seen that by detecting the incomplete connection lines of the key entities within the preset spatial range and based on the spatial distribution relationship between the entity type of the key entity and the incomplete connection lines, the potential correlation between the incomplete line segments of the key entities can be captured, and the incomplete line segments that conform to the power grid topology logic can be identified and completed, thereby repairing the topological relationship of the key entities in the image and improving the recognition accuracy of the connection relationship of the key entities.

[0055] S104: Determine at least one circuit device and corresponding device parameters according to the position of each material entity in the drawing image.

[0056] In this example, a material entity with a device type is determined to be a circuit device. Text information surrounding the circuit device can be determined as annotation information. The topological relationship of the circuit device can be determined based on the connecting lines connecting the circuit device. The annotation information and topological relationship can be determined as device parameters of the circuit device. In some embodiments, text within a preset radius of the material entity's physical area can be searched as the device parameters of the circuit device corresponding to the material entity. The preset radius can be 5mm.

[0057] In an optional embodiment, determining at least one circuit device and corresponding device parameters based on the position of each material entity in the drawing image includes: obtaining at least one circuit device in each material entity; obtaining the material entities of each circuit device within a preset adjacent range based on the position of each circuit device in the drawing image; and for each circuit device, determining the text content of the text entity in the material entity within the adjacent range as the device parameter corresponding to the circuit device.

[0058] Among them, the adjacent range can be the same as the aforementioned spatial range or different. The adjacent range is used as the regional range for searching text related to circuit equipment. Both the adjacent range and the spatial range can be determined based on experiments. Usually the annotation text of the circuit equipment is located in the area near the circuit equipment. Accordingly, the adjacent range can be preset, and the text within the adjacent range of the circuit equipment can be determined as the device parameters related to the circuit equipment. In fact, some circuit equipment lines are adjacent, and the annotation text of a circuit equipment in these circuit equipment may be located within the adjacent ranges of multiple circuit equipment. Therefore, the content of the annotation text can also be matched with the entity type of each circuit equipment located in the adjacent range, and the annotation text can be determined as the matching result as the device parameter of the corresponding circuit equipment. Among them, the equipment parameters can include basic ledgers, technical parameters, topological relationships, etc.

[0059] In practice, the circuit device in the material entity obtained through semantic segmentation includes the device region and the device region category. The text entity in the material entity includes the text location and text content. The connection line in the material entity includes the connection line location and the connected object. Based on the semantically segmented material entity, the circuit device and its device parameters can be determined, and JSON format content can be generated based on the circuit device and its device parameters as the final output provided to the user. In one example, the JSON format content includes:

[0060] {"equipment":[{

[0061] "Identifier":"DEV_001",

[0062] "Type":"transformer",

[0063] "Coordinates": [[x1,y1],[x2,y2],...], / / contour coordinates

[0064] "parameter":{

[0065] "Capacity":"1000kVA",

[0066] "voltage": "10kV / 0.4kV"},

[0067] "Connection relationship":[

[0068] {"Connection object":"DEV_002","via_line":"LINE_005"},

[0069] {"Connection object":"BUS_001","via_line":"LINE_006"}

[0070] ]}],

[0071] "Connecting line":[{

[0072] "Identification":"LINE_005",

[0073] "Type":"busbar",

[0074] "Path": [[x1,y1],[x2,y2],...]}]}

[0075] It can be seen that by determining the text content in the adjacent range of the material entity as the equipment parameters, the relevant annotation content of the circuit equipment in the drawing can be identified, the parameter association recognition in the drawing can be realized, the association relationship between the hand-drawn annotation text and the unstructured annotation and the circuit equipment can be correctly analyzed, and the recognition accuracy of the equipment annotation text can be improved.

[0076] The technical solution of the embodiment of the present invention classifies entities in the drawing image, divides entities of the same type into the same target layer, and then performs semantic segmentation of entities on each target layer to obtain fine-grained material entities. Finally, based on the position of the material entity in the drawing image, the circuit device and the device parameters of each circuit device are determined. This can reduce the labor cost of image entity recognition, improve the efficiency of image recognition, and realize multi-stage and multi-level entity recognition, improve the accuracy of image entity recognition. By determining the device parameters associated with the entity based on the position of the entity in the image, the information of the circuit device can be further supplemented, the richness and completeness of the information of the circuit device can be increased, the accuracy of image analysis can be improved, and the information of the identified device can be updated in real time when the device parameters change, thereby improving the efficiency of collaborative adjustment. This solves the problems of low efficiency, poor accuracy, high difficulty in analysis and low efficiency of collaborative adjustment in manual drawing analysis in the prior art, can improve the efficiency and accuracy of device identification in the drawing, simplify the complexity of device information identification, and improve the real-time updating of device information.

[0077] Figure 2 A flowchart of an image information extraction method provided in an embodiment of the present invention. Building on the above-described embodiment, this embodiment further refines the process of "performing semantic segmentation on each target entity to obtain at least one material entity in the drawing image" to include: performing semantic segmentation on each target entity using a semantic segmentation model to obtain at least one material entity in the drawing image. It should be noted that for portions not described in detail in this embodiment of the present invention, reference can be made to the descriptions in other embodiments.

[0078] See also Figure 2 The image information extraction method shown includes:

[0079] S201: Segment the drawing image into target layers to obtain at least one target layer, where the target layer includes entities of the same type.

[0080] S202: For each target layer, extract the target entity in the target layer.

[0081] S203 : Using a semantic segmentation model to perform semantic segmentation on each of the target entities to obtain at least one material entity in the drawing image.

[0082] The semantic segmentation model is used to semantically segment the target entity. The input of the semantic segmentation model may include a drawing image, a target layer, and the target entities included in each target layer. The output of the semantic segmentation model may include a material entity, a detection box (area) of the material entity, and the type of the material entity. In some embodiments, the input data is encoded to obtain a feature vector, and the feature vector is decoded to obtain the detection box and type of the material entity.

[0083] S204: Determine at least one circuit device and corresponding device parameters according to the position of each material entity in the drawing image.

[0084] In an optional embodiment, the penalty weight for recognition errors of error-prone samples (such as blurred text and rotated symbols) can be doubled during the training of the semantic segmentation model. Specifically, in the loss function, the weight of the pixel-level error in the error-prone area is increased. The error-prone area can be the text overlapping area, and the weight can be tripled. By doubling the penalty weight for error-prone samples, the semantic segmentation model can increase its attention to error-prone samples, correct the optimization direction during the training process, enhance the generalization performance of the model, and improve the accuracy of semantic segmentation.

[0085] In an optional embodiment, during the training of the semantic segmentation model, difficult examples can be mined and the sampling frequency of difficult examples can be increased, thereby dynamically adjusting the dynamic training data distribution. Specifically, difficult examples are obtained from the following dimensions: 1. Misidentification of overlapping text: "10kV" is recognized as "10KV" or "1OkV"; 2. Missed detection of rotated symbols: A disconnector symbol rotated 45° is classified as an unknown category; 3. Missed detection of small symbols: Ground symbols with a diameter of less than 5px are not detected.

[0086] For the mined difficult samples, we can perform a. Data augmentation: applying stronger rotations (±180°) and scaling (0.1-5x) to erroneous samples; b. Loss weight adjustment: assigning a higher weight to small objects in FocalLoss (γ=3); and c. Model structure optimization: adding a multi-scale attention module to the semantic segmentation model to improve sensitivity to small objects. By acquiring and processing difficult samples, we can focus on training difficult scenes, reduce false positives, and improve the semantic segmentation accuracy of the semantic segmentation model for difficult samples.

[0087] In an optional embodiment, adversarial training samples can be added to the training of the semantic segmentation model. Low-quality processing can be performed on the drawing image to simulate low quality, resulting in a low-quality image. The low-quality drawing image can then be input into the semantic segmentation model for training. For example, the effects of drawing aging, such as blurring and / or fading, can be simulated. More specifically, blurry text and / or faded symbols can be simulated, and the drawing image can be processed to form a low-quality image to force the semantic segmentation model to learn robust features under low-quality input.

[0088] In an optional embodiment, the learning rhythm can be adaptively adjusted as the training progresses. For example, in the initial stage, the learning rate is set to one ten-thousandth of the standard value, such as the standard learning rate 1e -3 , correspondingly, the initial learning rate is 1e -7) to prevent the model from converging to the local optimum too early. Iterative adjustment: After every 10 rounds of training (Epoch), the learning rate is reduced by 50% (such as 1e -7 Reduced to 5e -8 ), while dynamically adjusting based on the validation set accuracy: if the accuracy improvement is less than 1%, early stopping is triggered; if the accuracy drops, the model is rolled back to the previous optimal model and learning rate decay is restarted. The AdamW (Adam Adaptive Moment Estimation with Weight Decay) optimizer is used, with momentum parameters β1 = 0.9 and β2 = 0.999 to prevent drastic fluctuations in the learning rate. Initially accelerating convergence, a higher learning rate is used in training to quickly approach the optimal solution region, shortening convergence time. Fine-tuning is then performed in the later stages, with the learning rate automatically reduced near convergence, minimizing parameter update oscillations and improving the model's final performance.

[0089] The embodiment of the present invention uses a semantic segmentation model to perform semantic segmentation on the target entity, which can perform more fine-grained recognition of material entities, cover a more comprehensive type recognition range, improve the recognition accuracy and speed of material entities, and extract device-associated parameter information, especially with higher recognition accuracy for the recognition of unstructured text. It binds the device and associated parameters, and can quickly and collaboratively modify the device's associated parameters in real time when the device parameters change.

[0090] In an optional embodiment, the semantic segmentation model includes: a shallow feature extraction layer, a deep feature extraction layer and a decoding layer; the semantic segmentation model is trained by: obtaining a first sample set, wherein the first sample in the first sample set includes a drawing sample, a layer sample and a standard detection result of the drawing sample, and the standard detection result includes: at least one standard entity area and a standard category corresponding to each standard entity area; freezing the deep feature extraction layer in the initial model; using the drawing samples and layer samples in the first sample set as input to the initial model to obtain a first prediction result output by the initial model, the first prediction result includes: a predicted entity area of ​​at least one device entity and a predicted category corresponding to each predicted entity area; and training the initial model according to the standard detection result and the first prediction result included in the first sample. The model is adjusted to obtain an intermediate model; a second sample set is obtained, wherein the second sample in the second sample set includes: each of the predicted entity areas, the extended area of ​​each of the predicted entity areas, and the standard regression result of each of the predicted entity areas, and the standard regression result includes the standard text and standard relationship entity of each of the standard entity areas and the extended area of ​​each of the standard entity areas; each of the predicted entity areas and the standard extended area of ​​each of the predicted entity areas in the second sample set is input into the initial model to obtain a second prediction result output by the initial model, and the second prediction result includes: the rotation regression area of ​​each of the predicted entity areas, the predicted text and the predicted relationship entity of each of the extended areas; the intermediate model is adjusted according to the standard regression result and the second prediction result included in the second sample to obtain a semantic segmentation model.

[0091] The semantic segmentation model consists of an encoder and a decoder. The encoder includes a shallow feature extraction layer and a deep feature extraction layer. The decoder includes a decoding layer. The encoder extracts multi-scale features from the image and consists of multiple convolutional blocks and downsampling layers (such as MaxPooling and Strided Conv). The encoder outputs multiple layers of feature maps. The decoder generates a mask map, restores the resolution of the multi-layer feature maps, and classifies the pixels in the image. Specifically, the decoder upsamples the low-resolution feature map to the input image size and outputs a high-resolution mask map. The difference between the deep and shallow feature extraction layers is that the feature maps extracted by the deep feature extraction layer include global contextual information, while the feature maps extracted by the shallow feature extraction layer only include local details. For example, the deep feature extraction layer can determine that a material entity is a circuit breaker based on contextual semantics, while the shallow feature extraction layer can only determine that the material entity has lines and color blocks.

[0092] In which, the drawing sample may refer to a drawing image used as a training sample. The layer sample may refer to an image of a layer divided by the drawing sample, wherein the target layer and the layer sample in the aforementioned embodiment may be a mask image. The standard entity region may be represented by a binary mask. The predicted category may include the category of the circuit device. The intersection-over-union ratio between each predicted entity region and the corresponding standard entity region may be calculated, and the difference between the predicted category of the predicted entity region and the standard category of the corresponding standard entity region may be calculated, and the intersection-over-union ratio and the difference may be normalized and summed to obtain a loss value. With reducing the loss value as the optimization goal, the shallow feature extraction layer and the decoding layer in the initial model are adjusted to obtain an intermediate model.

[0093] The extended area may refer to the area obtained by expanding outward with the entity area as the center. Standard text and predicted text may refer to text content. Relational entities may refer to the entity area and category of entities that express the relationship of circuit devices. The rotated regression area may be an entity area that adjusts the predicted entity area to a preset positive direction. At this time, the standard entity area is used to provide the direction of the true value. In fact, in the drawing image, the circuit device may be an obliquely drawn device. Usually, the circuit device will be in a fixed direction as the positive direction. It is necessary to rotate the extracted circuit device image to the preset positive direction to facilitate the user's subsequent processing. The text differences between each predicted text and the corresponding standard text, the intersection-over-union (IoU) and category differences between each predicted entity and the corresponding relational entity, and the angular differences between the rotated regression entity area and the corresponding standard entity area can be calculated. The IoU and the differences can be normalized and summed to obtain the loss value. With reducing the loss value as the optimization goal, the encoding layer and the decoding layer in the intermediate model are adjusted to obtain a semantic segmentation model. The encoding layer includes a deep feature extraction layer and a shallow feature extraction layer. Since the second stage of training requires the model to learn the ability to recognize details, the deep feature extraction layer used to extract more comprehensive global features is involved in the parameter adjustment, that is, the deep feature extraction layer is unfrozen.

[0094] The training process of the semantic segmentation model can include two stages:

[0095] Phase 1 training (coarse-grained segmentation): The training objective is to detect the outlines of circuit devices (not the physical outlines of actual circuit devices). Input data includes: drawing samples and layers (images); output data includes: the physical area of ​​the circuit device and the category of the circuit device.

[0096] In some embodiments, circuit devices can be device symbols. These are graphical representations of devices in drawings, such as a rectangle representing a transformer or a circle representing a circuit breaker. While real devices have complex shapes, they are simplified into standard shapes using symbols in drawings. Layers can include circuit layers and annotation layers. Circuit device categories can include transformers and circuit breakers, among others. A solid area is an image region within a drawing image.

[0097] The initial model is a semantic segmentation model in the first stage. At this stage, the deep feature extraction layer is used to extract more detailed text and relational entities. To improve training accuracy and efficiency, the parameters of the deep feature extraction layer can be frozen in the first stage, and only the parameters of the shallow feature extraction layer and the decoding layer can be adjusted. The output of the encoding layer is a fusion of the features of the deep and shallow feature extraction layers. Thus, even if the parameters of the deep feature extraction layer are frozen, the encoding layer can still output the encoding result. The drawing sample and layer sample in each first sample are input into the initial model, and the predicted entity area and predicted category for each first sample are output. The predicted entity area and predicted category are compared with the standard entity area and standard category in the first sample, and the difference is calculated. The initial model parameters are adjusted based on the difference. The layer sample can be a layer mask, and the standard entity area and predicted entity area can be binary masks based on device symbols. The initial model of the first stage is completed and determined as the intermediate model.

[0098] Second-stage training (fine-grained segmentation): The training goal is to further segment text regions and connector detail entities within the circuit device entity region. Input data includes entity regions and extended regions; output data includes text, relationship entities, and positive entity regions. Furthermore, the rotation angle of the entity region can be output. This rotation angle is used to adjust the circuit device region to a standard angle with a fixed orientation.

[0099] In some embodiments, the relationship entity may include: arrows and terminals (connection points). The extended area is an area formed by extending the boundary of the entity area outward by 5-10 mm, and there is no overlapping area between the extended area and the entity area.

[0100] The intermediate model is the semantic segmentation model in the second stage. The second stage adjusts the parameters of both the encoding and decoding layers, meaning that all model parameters are adjusted. The intermediate module that completes the second stage is identified as the semantic segmentation model. The angular difference between the rotated regression region and the standard entity region is calculated. The textual difference between the standard text and the predicted text is calculated, for example, by calculating the ratio between the number of misrecognized characters and the total number of characters, i.e., the textual accuracy, which is used as the textual difference. The intersection-over-union ratio and category difference between the standard relational entity and the predicted relational entity are calculated to obtain the relational entity accuracy, which is used as the relational entity difference. The angular difference, textual difference, and relational entity difference are weighted to obtain the loss value for the second stage. With minimizing the loss value as the optimization goal, the parameters of the shallow feature extraction layer and decoding layer in the intermediate model are adjusted to obtain the semantic segmentation model.

[0101] Optionally, the first training sample set and the second training sample set can use the same dataset. In the process of calculating the loss function, different weights can be used for different loss items. In the loss value of the loss function of the first stage, the weight of the intersection-over-union ratio is 70%, while in the loss value of the loss function of the second stage, the weight of the recognition accuracy of text and relational entities is 30%. The total loss value of the two stages is summarized as:

[0102] Total loss = 0.7 × IoU + 0.3 × (text accuracy + relation entity accuracy)

[0103] It can be seen that through the two-stage training of the semantic segmentation model, the first stage trains the semantic segmentation model to understand the contours of graphics, and the second stage trains the semantic segmentation model to comprehensively learn to identify detailed features. This allows the semantic segmentation model to prioritize learning coarse-grained target detection capabilities before learning fine-grained target detection capabilities. The first stage focuses on contour recognition to block redundant details, which can avoid feature confusion caused by premature exposure to complex information and enhance sensitivity to the main shape. This lays the foundation for subsequent precise detail extraction and adapts to feature changes from macro to micro. The first stage of contour recognition ensures the stability of the basic form, while the second stage of detailed feature learning can specifically enhance the ability to distinguish similar samples and improve anti-interference performance, thereby gradually improving the accuracy of the semantic segmentation model.

[0104] In some optional embodiments, the standard extended area in the second sample is subjected to contrast enhancement and / or random noise addition. Real-world images contain interference such as blur, stains, and reflections. Adding noise enables the semantic segmentation model to learn to eliminate interference. The specific steps are as follows: generating random noise of the same size as the drawing image (with a mean of 0 and a standard deviation of 25); superimposing the noise on the original image; limiting pixel values ​​to between 0 and 255 (to avoid overflow) and converting back to unsigned 8-bit integers (OpenCV (Open Source Computer Vision Library) standard format).

[0105] In an optional embodiment, the semantic segmentation model determines the parameter weight corresponding to each sub-server based on the amount of data provided by multiple sub-servers, and adjusts the parameters based on the parameter weight corresponding to each sub-server and the provided parameter adjustment amount; wherein the parameter adjustment amount is the adjustment amount obtained by the sub-server adjusting the parameters of the semantic segmentation model.

[0106] The semantic segmentation model can be obtained through joint training. The sub-server is a server that trains, applies, and deploys the semantic segmentation model. The image information extraction method provided in the embodiments of the present invention can be executed by the sub-server. The sub-server can train the semantic segmentation model and send the training results to the central server. The central server summarizes the training results and distributes them to each sub-server, enabling parameter adjustment of the semantic segmentation model based on a large amount of sample data.

[0107] Among them, the parameter weight corresponding to the sub-server may refer to the degree of influence of the training results of the sub-server on the aggregated training results. The parameter weight can be determined based on factors such as the location, function, performance and amount of training data provided by the sub-server. In some embodiments, the parameter weight is determined based on the amount of local data of the sub-server. In one example, the parameter weight is the ratio of the local data volume of the sub-server to the sum of the local data volumes of each sub-server. For example, the data volume of sub-server A is 50,000 images, the data volume of sub-server B is 30,000 images, and the data volume of sub-server C is 20,000 images. The sum of the local volumes of the three sub-servers is 100,000 images. The corresponding parameter weights are 5 / 10=0.5, 3 / 10=0.3 and 2 / 10=0.2 respectively.

[0108] The parameter adjustment amount is the difference between the parameters obtained by tuning the locally deployed semantic segmentation model on the subserver and the parameters obtained in the previous tuning. The central server periodically summarizes the parameter adjustments of each subserver, for example, once a week. The central server obtains the parameter adjustments of the subservers within a week and performs a weighted summation based on the parameter weights to obtain a target adjustment amount. This is then distributed to each subserver as the tuning result. Each subserver receives the target adjustment amount issued by the central server and adds it to the parameters of the local semantic segmentation model to tune the local semantic segmentation model. The parameter adjustments corresponding to each subserver can be weighted according to their corresponding parameter weights to obtain a summarized parameter adjustment amount, which is then used to tune the semantic segmentation model. After issuing the target adjustment amount, the central server receives the parameter adjustment amount obtained from the next round of training from each subserver. After receiving the target adjustment amount, the subserver adds the target adjustment amount to the current parameter value to update the current parameter value. At this point, the updated current parameter value of each subserver is the same, meaning that the updated current parameter value is the global parameter value.

[0109] In one example, each sub-server uploads a parameter adjustment (Δθi), and the central server calculates a weighted target adjustment θglobal based on the following formula:

[0110]

[0111] Δθi=θi k local-θ k-1 global

[0112] Where wi is the parameter weight of the i-th sub-server. θi k local is the parameter value obtained by the k-th round of training for the i-th sub-server, θ k-1 Global is the global parameter value obtained from the k-1th round of training. It is also the updated parameter value obtained by the i-th sub-server by adding the target adjustment amount of the central server in the k-1th round to the local parameter value. The updated parameter values ​​after the k-1th round of training for different sub-servers are the same.

[0113] During the first round of training, the central server pre-trains and obtains the first-round parameter values, which serve as the target adjustment values. The local parameter values ​​of each sub-server are initially set to 0. The target adjustment values ​​are then added to the local parameter values, and the updated parameter values ​​are the same as the central server's parameter values. In other words, the central server's parameter values ​​are updated to all sub-servers and become the global parameter values. In subsequent training rounds, the parameter values ​​are continuously updated, the target adjustment values ​​are calculated, and these are distributed to each sub-server. The local parameter values ​​of each sub-server are also updated to ensure that the parameter values ​​of each sub-server are consistent with the global parameter values ​​updated by the central server.

[0114] It can be seen that by collaboratively training the semantic segmentation model through sub-servers, the model's training samples can be enriched, complex processing tasks can be adapted, and the model's performance can be improved. In addition, by distributing the training model and aggregating the training results of each sub-server, the model's training speed can be increased.

[0115] Figure 3 This is a schematic diagram of the structure of an image information extraction device provided by an embodiment of the present invention. This embodiment of the present invention is applicable to obtaining equipment and related information from image analysis of power grid drawings. The device can execute an image information extraction method, can be implemented in hardware and / or software, and can be configured in an electronic device that carries the image information extraction function.

[0116] See also Figure 3 The image information extraction device shown includes:

[0117] The target layer segmentation module 301 is used to segment the drawing image into target layers to obtain at least one target layer, wherein the target layer includes entities of the same type;

[0118] An entity extraction module 302 is configured to extract target entities from each target layer.

[0119] A semantic segmentation module 303 is configured to perform semantic segmentation on each target entity to obtain at least one material entity in the drawing image;

[0120] The parameter identification module 304 is configured to determine at least one circuit device and corresponding device parameters according to the position of each material entity in the drawing image.

[0121] The technical solution of the embodiment of the present invention classifies entities in the drawing image, divides entities of the same type into the same target layer, and then performs semantic segmentation of entities on each target layer to obtain fine-grained material entities. Finally, based on the position of the material entity in the drawing image, the circuit device and the device parameters of each circuit device are determined. This can reduce the labor cost of image entity recognition, improve the efficiency of image recognition, and realize multi-stage and multi-level entity recognition, improve the accuracy of image entity recognition. By determining the device parameters associated with the entity based on the position of the entity in the image, the information of the circuit device can be further supplemented, the richness and completeness of the information of the circuit device can be increased, the accuracy of image analysis can be improved, and the information of the identified device can be updated in real time when the device parameters change, thereby improving the efficiency of collaborative adjustment. This solves the problems of low efficiency, poor accuracy, high difficulty in analysis and low efficiency of collaborative adjustment in manual drawing analysis in the prior art, can improve the efficiency and accuracy of device identification in the drawing, simplify the complexity of device information identification, and improve the real-time updating of device information.

[0122] Optionally, the semantic segmentation module 303 is specifically configured to:

[0123] Screening each of the target entities, eliminating redundant entities, and obtaining at least one key entity;

[0124] Semantic segmentation is performed on each of the key entities to obtain at least one material entity in the drawing image.

[0125] Optionally, the semantic segmentation module 303 is specifically configured to:

[0126] Performing data enhancement on each of the key entities, wherein the data enhancement includes at least one of the following: geometric transformation, image quality enhancement, and entity completion;

[0127] Semantic segmentation is performed on the key entities after data enhancement to obtain at least one material entity in the drawing image.

[0128] Optionally, the data enhancement includes entity completion;

[0129] The semantic segmentation module 303 is specifically used to:

[0130] Obtaining connection lines within a preset spatial range of the key entity;

[0131] When a connection line of the key entity within a preset spatial range is incomplete, obtaining an entity type corresponding to the key entity and a spatial distribution relationship between the key entity and the incomplete connection line;

[0132] The incomplete connection lines are completed according to the entity types and the spatial distribution relationships.

[0133] Optionally, the semantic segmentation module 303 is specifically configured to:

[0134] A semantic segmentation model is used to perform semantic segmentation on each of the target entities to obtain at least one material entity in the drawing image.

[0135] Optionally, the semantic segmentation model includes: a shallow feature extraction layer, a deep feature extraction layer and a decoding layer;

[0136] The image information extraction device further includes a model training module for:

[0137] Acquire a first sample set, wherein a first sample in the first sample set includes a drawing sample, a layer sample, and a standard detection result of the drawing sample, wherein the standard detection result includes: at least one standard entity area and a standard category corresponding to each standard entity area;

[0138] Freeze the deep feature extraction layers in the initial model;

[0139] Inputting drawing samples and layer samples in the first sample set into the initial model to obtain a first prediction result output by the initial model, the first prediction result including: a predicted entity area of ​​at least one device entity and a prediction category corresponding to each predicted entity area;

[0140] Adjusting parameters of the initial model according to the standard test result included in the first sample and the first prediction result to obtain an intermediate model;

[0141] Obtaining a second sample set, wherein the second samples in the second sample set include: each of the predicted entity regions, an extended region of each of the predicted entity regions, and a standard regression result of each of the predicted entity regions, wherein the standard regression result includes a standard text and a standard relationship entity in each of the standard entity regions and the extended region of each of the standard entity regions;

[0142] Using each of the predicted entity regions and the standard extended region of each of the predicted entity regions in the second sample set as input into the initial model, obtaining a second prediction result output by the initial model, the second prediction result including: a rotation regression region of each of the predicted entity regions, a predicted text of each of the extended regions, and a predicted relationship entity;

[0143] According to the standard regression result and the second prediction result included in the second sample, the intermediate model is adjusted to obtain a semantic segmentation model.

[0144] Optionally, the semantic segmentation model determines the parameter weight corresponding to each sub-server based on the amount of data provided by multiple sub-servers, and adjusts the parameters based on the parameter weight corresponding to each sub-server and the provided parameter adjustment amount; wherein the parameter adjustment amount is the adjustment amount obtained by the sub-server adjusting the parameters of the semantic segmentation model.

[0145] The image information extraction device provided in the embodiment of the present invention can execute the image information extraction method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0146] In the technical solutions of the embodiments of the present invention, the acquisition, storage and application of the data involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0147] Figure 4 A schematic diagram of the structure of an electronic device 400 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0148] like Figure 4 As shown, the electronic device 400 includes at least one processor 401, and a memory connected to the at least one processor 401 in communication, such as a read-only memory (ROM) 402, a random access memory (RAM) 403, etc., wherein the memory stores a computer program that can be executed by the at least one processor, and the processor 401 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 402 or the computer program loaded from the storage unit 408 into the random access memory (RAM) 403. In the RAM 403, various programs and data required for the operation of the electronic device 400 can also be stored. The processor 401, ROM 402 and RAM 403 are connected to each other via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0149] Multiple components in the electronic device 400 are connected to the I / O interface 405, including an input unit 406, such as a keyboard, a mouse, etc.; an output unit 407, such as various types of displays, speakers, etc.; a storage unit 408, such as a magnetic disk, an optical disk, etc.; and a communication unit 409, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 409 allows the electronic device 400 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0150] Processor 401 can be various general-purpose and / or specialized processing components with processing and computing capabilities. Some examples of processor 401 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. Processor 401 performs the various methods and processes described above, such as the image information extraction method.

[0151] In some embodiments, the image information extraction method can be implemented as a computer program tangibly embodied in a computer-readable storage medium, such as storage unit 408. In some embodiments, part or all of the computer program can be loaded and / or installed on electronic device 400 via ROM 402 and / or communication unit 409. When the computer program is loaded into RAM 403 and executed by processor 401, one or more steps of the image information extraction method described above can be performed. Alternatively, in other embodiments, processor 401 can be configured to perform the image information extraction method in any other appropriate manner (e.g., via firmware).

[0152] Various embodiments of the systems and techniques described above can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0153] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the computer program is executed by the processor, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The computer program may be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0154] In the context of the present invention, computer-readable storage media can be tangible media that can contain or store a computer program for use with an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. Computer-readable storage media can include but are not limited to electronic, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, computer-readable storage media can be machine-readable signal media. More specific examples of machine-readable storage media can include electrical connections based on one or more lines, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0155] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0156] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0157] A computing system may include clients and servers. The clients and servers are generally remote from each other and typically interact via a communication network. This client-server relationship arises through computer programs running on the respective computers, creating a client-server relationship. The server may be a cloud server, also known as a cloud computing server or cloud host. This server is a hosting product within a cloud computing service ecosystem that addresses the management difficulties and limited scalability of traditional physical hosting and VPS (Virtual Private Server) services.

[0158] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in the present invention can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of the present invention can be achieved. This is not limited herein.

[0159] The above specific embodiments do not limit the scope of protection of the present invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

Claims

1. A method for extracting image information, characterized in that: The method comprises: Performing target layer segmentation on the drawing image to obtain at least one target layer, wherein the target layer includes entities of the same type; For each target layer, extract the target entity in the target layer; Performing semantic segmentation on each of the target entities to obtain at least one material entity in the drawing image; At least one circuit device and corresponding device parameters are determined according to the position of each material entity in the drawing image.

2. The method according to claim 1, characterized in that Performing semantic segmentation on each of the target entities to obtain at least one material entity in the drawing image includes: Screening each of the target entities, eliminating redundant entities, and obtaining at least one key entity; Semantic segmentation is performed on each of the key entities to obtain at least one material entity in the drawing image.

3. The method according to claim 2, characterized in that Performing semantic segmentation on each of the key entities to obtain at least one material entity in the drawing image includes: Performing data enhancement on each of the key entities, wherein the data enhancement includes at least one of the following: geometric transformation, image quality enhancement, and entity completion; Semantic segmentation is performed on the key entities after data enhancement to obtain at least one material entity in the drawing image.

4. The method according to claim 3, characterized in that The data enhancement includes entity completion; The data enhancement for each of the key entities includes: Obtaining connection lines within a preset spatial range of the key entity; When a connection line of the key entity within a preset spatial range is incomplete, obtaining an entity type corresponding to the key entity and a spatial distribution relationship between the key entity and the incomplete connection line; The incomplete connection lines are completed according to the entity types and the spatial distribution relationships.

5. The method according to claim 1, wherein The performing semantic segmentation on each target entity to obtain at least one material entity in the drawing image includes: A semantic segmentation model is used to perform semantic segmentation on each of the target entities to obtain at least one material entity in the drawing image.

6. The method according to claim 5, characterized in that The semantic segmentation model includes: a shallow feature extraction layer, a deep feature extraction layer and a decoding layer; The semantic segmentation model is trained in the following way: Acquire a first sample set, wherein a first sample in the first sample set includes a drawing sample, a layer sample, and a standard detection result of the drawing sample, wherein the standard detection result includes: at least one standard entity area and a standard category corresponding to each standard entity area; Freeze the deep feature extraction layers in the initial model; Inputting drawing samples and layer samples in the first sample set into the initial model to obtain a first prediction result output by the initial model, the first prediction result including: a predicted entity area of ​​at least one device entity and a prediction category corresponding to each predicted entity area; Adjusting parameters of the initial model according to the standard test result included in the first sample and the first prediction result to obtain an intermediate model; Obtaining a second sample set, wherein the second samples in the second sample set include: each of the predicted entity regions, an extended region of each of the predicted entity regions, and a standard regression result of each of the predicted entity regions, wherein the standard regression result includes a standard text and a standard relationship entity in each of the standard entity regions and the extended region of each of the standard entity regions; Using each of the predicted entity regions and the standard extended region of each of the predicted entity regions in the second sample set as input into the initial model, obtaining a second prediction result output by the initial model, the second prediction result including: a rotation regression region of each of the predicted entity regions, a predicted text of each of the extended regions, and a predicted relationship entity; According to the standard regression result and the second prediction result included in the second sample, the intermediate model is adjusted to obtain a semantic segmentation model.

7. The method according to claim 5 or 6, characterized in that in, The semantic segmentation model determines the parameter weight corresponding to each sub-server based on the amount of data provided by multiple sub-servers, and adjusts the parameters based on the parameter weight corresponding to each sub-server and the provided parameter adjustment amount; wherein the parameter adjustment amount is the adjustment amount obtained by the sub-server adjusting the parameters of the semantic segmentation model.

8. An image information extraction device, characterized in that: The device comprises: A target layer segmentation module is used to segment the drawing image into target layers to obtain at least one target layer, wherein the target layer includes entities of the same type; An entity extraction module, configured to extract target entities from each target layer; a semantic segmentation module, configured to perform semantic segmentation on each target entity to obtain at least one material entity in the drawing image; The parameter identification module is used to determine at least one circuit device and corresponding device parameters according to the position of each material entity in the drawing image.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor. The computer program is executed by the at least one processor to enable the at least one processor to perform the image information extraction method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the image information extraction method according to any one of claims 1 to 7 when executed.

Images