Data processing method, computer device and readable storage medium

Abstract

The application relates to the technical field of drawing analysis, and particularly discloses a data processing method, computer equipment and a readable storage medium. The method comprises the following steps: converting an original format of an architectural drawing into a vector drawing, the vector drawing having the same graphic features and layer distribution as the architectural drawing, and maintaining the mapping relationship between the architectural drawing entity and the vector drawing graphic element; performing sliding window slicing on the vector drawing to obtain a plurality of vector sub-drawings and raster images corresponding to the vector sub-drawings; obtaining vector embedding features based on the vector sub-drawings, and obtaining image embedding features based on the raster images; performing cross-modal fusion on the vector embedding features and the image embedding features to obtain a joint feature representation; performing decoding processing on the joint feature representation based on an attention mechanism, and generating a panoramic segmentation result of the architectural drawing based on the decoding processing result. Thus, the geometric precision of the vector information and the texture semantics of the image information are more fully utilized, and the accuracy of component recognition and panoramic segmentation in the architectural drawing is improved.

Description

Technical Field

[0001] This application relates to the field of drawing analysis technology, and in particular to a data processing method, computer equipment, and readable storage medium. Background Technology

[0002] In the processes of building electromechanical design, fire protection drawing review, engineering quantity calculation, and digital twin modeling, a large amount of information is still stored in the form of CAD drawings. The components in CAD drawings include equipment objects such as cabinets, detectors, buttons, and lighting fixtures, as well as linear components such as wires, pipelines, and loop boundaries. These objects usually span multiple layers, are densely arranged, have large scale differences, and have complex occlusion and overlap, which poses a significant challenge to automatic identification and structured analysis.

[0003] Existing methods for recognizing CAD drawings can be broadly categorized into two types: one type directly renders the drawing as an image and then uses convolutional neural networks for semantic segmentation or object detection; the other type utilizes only the geometric attributes of vector primitives for rule matching or graph neural network modeling. While the former can leverage image texture and local context, it easily loses the precise geometric structure, layer organization, and instance association information in the vector drawing. The latter can preserve vector information but often ignores texture cues, symbol appearance differences, and cross-scale context within the image modality. Therefore, both methods have their shortcomings, and further optimization of drawing recognition methods is an urgent task in this field. Summary of the Invention

[0004] Therefore, it is necessary to provide a data processing method, a computer device, and a readable storage medium to address the above problems.

[0005] According to a first aspect of the embodiments of this application, a data processing method is provided, comprising: Convert the original architectural drawings into vector graphics, which have the same graphic features and layer distribution as the architectural drawings, and maintain the mapping relationship between the entities in the architectural drawings and the primitives in the vector graphics. The vector paper is sliced ​​using a sliding window to obtain multiple vector sub-images and the raster image corresponding to each vector sub-image; Vector embedding features are obtained based on each of the vector sub-graphs, and image embedding features are obtained based on each of the raster images; The vector embedding features and the image embedding features are fused across modally to obtain a joint feature representation; The joint feature representation is decoded using an attention mechanism, and a panoramic segmentation result of the architectural drawing is generated based on the decoding result.

[0006] In one embodiment, the step of converting the original format architectural drawings into vector graphics includes: Based on the layers, different rendering methods are used for different types of entities in the architectural drawings to generate a first vector drawing with the same graphic features, layer distribution, and maintaining the entity mapping relationship of the architectural drawings; The first vector paper is subjected to uniform fitting processing to obtain a second vector paper suitable for sliding window slicing.

[0007] In one embodiment, the step of performing sliding window slicing on the vector paper to obtain multiple vector sub-images and raster images corresponding to each vector sub-image includes: By using a fixed window size and sliding step, overlapping sliding window slices are applied to the second vector map to obtain multiple vector sub-maps; Each of the vector sub-images is rendered to obtain a raster image corresponding to each vector sub-image.

[0008] In one embodiment, the step of obtaining vector embedding features based on each of the vector subgraphs includes: Discrete sampling is performed on the primitives in each of the vector sub-graphs to construct vector samples; The vector samples are input into the primitive sequence encoder to extract the vector embedding features of each primitive.

[0009] In one embodiment, the vector sample includes the sampling coordinates, geometric attributes, layer attributes, and the mapped coordinates of the primitive center at the center of its corresponding raster image for each primitive.

[0010] In one embodiment, the step of obtaining image embedding features based on each of the raster images includes: The raster image is input into a multi-scale visual backbone network to obtain a multi-scale image feature map. Based on the spatial location information of each primitive, multi-scale image embedding features corresponding to each primitive are extracted from the multi-scale image feature map.

[0011] In one embodiment, the step of performing cross-modal fusion of the vector embedding features and the image embedding features to obtain a joint feature representation includes: The vector embedding features of primitives in each vector sub-image are weighted, and the reweighted vector embedding features are concatenated with the image embedding features and mapped to a unified dimension to obtain the first fusion feature; For the first fusion feature of each primitive in the same layer, the average pooling feature, the max pooling feature, and the attention-weighted pooling feature are calculated respectively, and the three are aggregated to form the global feature of the layer. The global feature is then written back to each primitive belonging to the same layer to obtain the joint feature representation.

[0012] In one embodiment, the step of performing attention-based decoding on the joint feature representation and generating a panoramic segmentation result of the architectural drawing based on the decoding result includes: The joint feature representation is input into a decoder based on self-attention and cross-attention mechanisms to output the semantic prediction result, instance mask, instance category, and instance confidence of each primitive. The semantic prediction results are corrected by instance-aware voting based on instance mask and instance confidence, resulting in corrected slice-level panoramic segmentation results. Based on the corrected slice-level panoramic segmentation results, the semantic prediction results of the same primitives in different vector sub-maps belonging to the same vector paper are aggregated by cross-slice secondary voting to output the paper-level panoramic segmentation results.

[0013] According to a second aspect of the embodiments of this application, a computer device is provided, comprising: At least one processor; And a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the data processing method as described above.

[0014] According to a third aspect of the embodiments of this application, a computer-readable storage medium is provided, on which a computer program is stored, wherein the computer program, when executed by a processor, implements the above-described data processing method.

[0015] According to a fourth aspect of the embodiments of this application, a computer program product is provided, including a computer program that, when executed by a processor, implements the data processing method described above.

[0016] The data processing method provided in this application converts the original architectural drawings into vector graphics with the same graphic features and layer distribution as the architectural drawings, while maintaining the mapping relationship between the entities in the architectural drawings and the primitives in the vector graphics. Sliding window slicing is then performed to obtain vector sub-images and corresponding raster images. This method simultaneously preserves the structured vector information and rasterized visual information of the drawings, laying a data foundation for subsequent multimodal fusion processing. Based on this, vector embedding features are extracted from the vector sub-images, and image embedding features are extracted from the raster images. These two features are then fused across modally to obtain a joint feature representation that integrates precise geometric structure and texture context. Finally, the joint feature representation is decoded using an attention mechanism to generate a panoramic segmentation result of the architectural drawings. Compared with existing technologies, this application eliminates the need for simple splicing or switching between vector and image modalities. Instead, it achieves the fusion of information from both vector and image modalities, enabling more full utilization of the geometric precision of vector information and the texture semantics of image information. This improves the accuracy and robustness of component recognition and panoramic segmentation in architectural drawings. Attached Figure Description

[0017] Figure 1 A flowchart illustrating a data processing method provided in an embodiment of this application; Figure 2 A flowchart of step S100 in a data processing method provided in an embodiment of this application; Figure 3 A flowchart of step S300 in a data processing method provided in an embodiment of this application; Figure 4 This is a partial flowchart of step S500 in a data processing method provided in an embodiment of this application; Figure 5 This is a partial flowchart of step S500 in a data processing method provided in an embodiment of this application; Figure 6 A flowchart of step S700 in a data processing method provided in an embodiment of this application; Figure 7 This is a flowchart of step S900 in a data processing method provided in an embodiment of this application. Detailed Implementation

[0018] To facilitate understanding of this application, a more complete description will be provided below with reference to the accompanying drawings. Preferred embodiments of this application are shown in the drawings. However, this application can be implemented in many different forms and is not limited to the embodiments described herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the disclosure of this application.

[0019] In this application, unless otherwise expressly specified and limited, the terms "installation," "connection," "joining," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components, unless otherwise expressly limited. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances.

[0020] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0021] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein in the specification of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0022] In one embodiment, a data processing method is provided that can be used to identify components in CAD drawings and can be applied to building fields such as fire protection electrical scenarios, water supply and drainage, HVAC, and electrical systems.

[0023] Reference Figure 1 The data processing method provided in this embodiment includes steps S100 to S900, as detailed below: Step S100: Convert the original architectural drawings into vector graphics. The vector graphics have the same graphic features and layer distribution as the architectural drawings, and maintain the mapping relationship between the entities in the architectural drawings and the primitives in the vector graphics.

[0024] First, the original architectural drawings can be obtained, such as in DXF format. Then, these drawings can be converted into vector graphics with the same graphic features and layer distribution. Specifically, upon obtaining the original drawings, different rendering or fitting methods are applied to different types of entities within the drawings, layer by layer, thus converting them into vector graphics that maintain the same graphic features and layer distribution, such as SVG format. Simultaneously, the unique identifier (handleId) of each entity in the architectural drawings is preserved during the conversion process and written into the corresponding primitive's source code in the vector graphics, facilitating the return of model prediction results to the original architectural drawings.

[0025] Reference Figure 2 In one embodiment, step S100 may further include steps S110 and S120, as follows: Step S110: According to the layers, render different types of entities in the architectural drawings in different ways to generate a first vector drawing with the same graphic features, layer distribution, and maintain the entity mapping relationship of the architectural drawings.

[0026] By rendering different layers and entity types in different ways, architectural drawings can be transformed into a first vector graphic with the same graphic features, layer distribution, and entity mapping relationships, such as SVG format drawings. For example, if an entity in an architectural drawing is located on layer "FIRE-DETECT" and its entity type is ARC, it is converted to the same layer on the first vector graphic and rendered using the appropriate method to convert the ARC entity into a cubic Bézier curve with the same coordinate scale. Simultaneously, the entity's unique identifier (handleId) is written into the command sequence of the corresponding cubic Bézier curve.

[0027] Step S120: Perform uniform fitting processing on the first vector paper to obtain a second vector paper suitable for sliding window slicing.

[0028] Once the first vector graphic is obtained, it can be uniformly fitted, such as performing primitive loop detection and fitting the closed cubic Bézier curves into circles, ellipses, or connected arcs. This yields a second vector graphic suitable for sliding window slicing.

[0029] Step S300: Slide the vector paper to obtain multiple vector sub-images and the raster image corresponding to each vector sub-image.

[0030] Once a suitable vector graphic for sliding window slicing, i.e., the second vector graphic, is obtained, sliding window slicing can be performed on the second vector graphic to obtain multiple vector sub-graphics and raster images (PNG format images) corresponding to each vector sub-graphic.

[0031] Reference Figure 3 In one embodiment, step S300 may further include steps S310 and S320, as follows: Step S310: Using a fixed window size and sliding step, overlap and slide window slice the second vector paper to obtain multiple vector sub-images; Step S320: Render each of the vector sub-images to obtain a raster image corresponding to each vector sub-image.

[0032] Specifically, after rendering and fitting the original architectural drawings in layers, a second vector graphic can be obtained. Let's assume the second vector graphic is an SVG collection. Window size can be used With sliding step size Perform a sliding window slice on it to obtain a slice set. and the set of corresponding raster images obtained by rendering them. This refers to each vector sub-image and its corresponding raster image. When windows overlap, the same original primitive can appear in multiple slices, thus providing redundant observations for subsequent cross-slice results.

[0033] Here, the visible area of ​​a single original format architectural drawing is defined as R, and the size of the slice window is [value missing]. The sliding step size is Then the first A slice window It can be represented as: in, Indicates the first A slice window The x-coordinate of the top left vertex, Indicates the first A slice window The ordinate of the top left vertex.

[0034] Step S500: Obtain vector embedding features based on each of the vector sub-images, and obtain image embedding features based on each of the raster images.

[0035] After slicing to obtain each vector sub-image, the corresponding vector embedding features can be obtained based on each vector sub-image. After rendering to obtain each raster image, the corresponding image embedding features can be obtained based on each raster image.

[0036] Specifically, refer to Figure 4 The step of obtaining vector embedding features based on each of the vector subgraphs in step S500 may further include steps S510 and S520, as follows: Step S510: Discretely sample the primitives in each of the vector sub-graphs to construct vector samples; Step S520: Input the vector sample into the primitive sequence encoder to extract the vector embedding features of each primitive.

[0037] That is, after slicing to obtain each vector sub-image, the primitives in each vector sub-image can be discretely sampled according to a preset sampling rule, and the sampled content can be analyzed for attributes to construct a vector sample (e.g., in JSON format). The vector sample can include the sampling coordinates, geometric attributes, layer attributes, and the mapping coordinates of the primitive center to the center of its corresponding raster image for each primitive, so as to characterize the shape, orientation, and spatial position of the primitive.

[0038] Furthermore, during the vector sample construction process, discrete sampling can be performed on the primitives in each vector sub-image to obtain several line segments. For long curves or polylines, the number of sampling points can be dynamically increased according to the maximum sampling length; line segments exceeding the slice boundary can be clipped; and excessively short primitives can be ignored, thereby reducing noise interference. After sampling, a seven-dimensional geometric descriptor consisting of length, direction components, line segment center coordinates, and primitive center coordinates can be constructed for each sampled line segment. Simultaneously, the primitive's corresponding color, layer, length, handleId (used for mapping back to the original architectural drawing), and the mapping information from the primitive center to the raster image coordinate system can be recorded. All of this information together constitutes the vector sample corresponding to each primitive.

[0039] The vector samples corresponding to each primitive can form a vector point structure, which is then input into a pre-trained primitive sequence encoder (GS-Transformer architecture). The primitive sequence encoder is then used to extract primitive-level vector embedding features.

[0040] Specifically, the design logic of the primitive sequence encoder model is as follows: First, the input vector point structure P is decomposed spatially and topologically. Unlike traditional global sampling, the primitive sequence encoder uses a splitting operator. Divide the point cloud into K geometrically consistent sub-graph sets. : in, This represents the topological constraints (such as coplanarity and continuity) of primitives. This step ensures that subsequent serialization encoding is built upon meaningful geometric components, rather than random, unordered points.

[0041] For each subgraph Sparse convolution algorithm SConv can be used to extract local fine-grained geometric features. .set up Features of points within a subgraph The square convolution kernel has the following evolution formula: This step can efficiently capture key information such as the edges and chamfers of primitives while maintaining the sparsity of computation.

[0042] In obtaining the features of each subgraph Then, it can be regarded as a sequence unit, and the long-range spatial relationships between primitives (such as assembly, symmetry, parallelism) can be modeled through a position-aware attention mechanism. The specific calculation formula is as follows: in, Calculate weights for attention, query vector : Represents the active retrieval request of the current target subgraph. It encodes the geometric attributes of the subgraph (e.g., "I am a line segment discrete from a circle") and attempts to find other primitives in the global space that have topological logic with it (e.g., "from the same circle"). Key vector : Represents the index identifier of the candidate primitive. It identifies the structural features of other primitives in space (e.g., "I am a circle"), and is used to compare with them. Perform matching and calculate association weights. Value vector : Represents the geometric information of the primitives being transmitted. When and When the matching degree is high, the corresponding subgraph of The information will be aggregated into the target subgraph, completing the long-range feature fusion across primitives. It is a geometric relative bias term, calculated by the centroid of the subgraph. and The relative vector distance forces the model to learn global spatial constraints between primitives. The matrix dimension in the attention calculation formula. This is the computation of transform in linear algebra.

[0043] Represents subgraph features Subgraph vector features obtained after a single-round position-aware attention mechanism. By repeating the calculation multiple times, the vector embedding feature of each primitive in the subgraph can be obtained, denoted as . .

[0044] As described above, in this embodiment, a primitive sequence encoder is used to extract the vector embedding features of each primitive. This encoder first discretizes the vector point structure into a sequence of subgraphs with geometric semantics through a topology-guided splitting module. Then, it utilizes a sparse convolution operator in conjunction with a geometric bias attention mechanism to accurately model long-range topological relationships between primitives while reducing computational complexity. In addition, by loading pre-trained weights, the primitive sequence encoder can quickly converge and extract highly discriminative primitive-level vector embedding features, providing a solid geometric representation foundation for subsequent steps.

[0045] In this embodiment, the vector embedding features of each primitive extracted by the primitive sequence encoder that repeats multiple rounds are denoted as . .

[0046] Specifically, refer to Figure 5 The step of obtaining image embedding features based on each of the raster images in step S500 may further include steps S530 and S540, as follows: Step S530: Input the raster image into a multi-scale visual backbone network to obtain a multi-scale image feature map; Step S540: Based on the spatial location information of each graphic element, extract the multi-scale image embedding features corresponding to each graphic element from the multi-scale image feature map.

[0047] In this embodiment, a multi-scale visual backbone network (MSVB) is used as the visual backbone. For each raster image, multiple scale image feature maps can be obtained through the multi-scale visual backbone network. Then, based on the position of the primitive center in the raster image, local visual information can be extracted from the image feature maps at each scale, thus ensuring the spatial correspondence between image features and vector primitives. Finally, the extracted local visual information can be stitched and mapped to obtain image embedding features that correspond one-to-one with the primitives.

[0048] Specifically, each raster image is input into a multi-scale visual backbone network, which can extract multi-scale feature maps through four parallel processing branches with different spatial resolutions: High-resolution branch: Maintains 1 / 4 of the original resolution to capture extremely fine lines and geometric edge features in the drawing; Medium to high resolution branch: Downsampled to 1 / 8 resolution, extracting texture features of local components through convolutional sequences (Conv Blocks); Low-to-medium resolution branch: downsampled to 1 / 16 resolution, mainly responsible for medium-scale symbol entity recognition; Low-resolution branch: downsampled to 1 / 32 resolution, providing compressed representations for subsequent global semantic modeling.

[0049] In addition, an atrous spatial pyramid pooling (ASPP) module can be introduced into the medium-resolution branch, which obtains multi-scale receptive fields through atrous convolutions with different sampling rates to adapt to annotation symbols with variable scales in drawings, as detailed below: Wherein, denotes the input feature map, denotes the atrous convolution with an atrous rate of 12, is the output feature map of the module.

[0050] Then, a multi-scale feature fusion module (MSFF) can be used for information interaction: Wherein, is the input feature map with a resolution of j, is the output feature map with a resolution of i after fusion, is the activation function, is the cross-scale transformation function: if j>i (low resolution to high resolution), is bilinear interpolation upsampling followed by a 1×1 convolution; if j<i (high resolution to low resolution), is the strided convolution with a stride of , and if j=i, is an identity mapping or a standard residual connection.

[0051] The upsampling operation can inject deep semantic information from the low-resolution branch into the high-resolution branch to enhance the category attributes of fine-grained features; meanwhile, the strided convolution can downsample the geometric details of the high-resolution branch to the low-resolution stream to correct spatial position deviations.

[0052] In this embodiment, the multi-scale feature fusion module can realize feature alignment under different receptive fields, ensuring that the model can take into account both local component details and the overall layout logic in subsequent stages.

[0053] Finally, the four fully fused and enhanced parallel features can be subjected to final aggregation. Specifically, a scale unification operation can be performed: the feature maps at 1 / 8, 1 / 16 and 1 / 32 resolutions are all upsampled to the 1 / 4 reference resolution. As a result, an aggregated multi-scale image feature map that contains both high-precision line details and strong semantic layout information can be obtained, which serves as the input for the downstream decoder or detection head.

[0054] Then, based on the spatial location information of each primitive (i.e., the mapped coordinates of the primitive center in the raster image), local visual feature vectors at corresponding locations can be extracted from the multi-scale image feature map using a grid sampling method. After concatenating the feature vectors from different scales, they are mapped to the same dimension as the vector embedding features through a linear projection layer, thus obtaining the multi-scale image embedding features for each primitive. .

[0055] Step S700: Perform cross-modal fusion on the vector embedding features and the image embedding features to obtain a joint feature representation.

[0056] Specifically, after obtaining the vector embedding features and image embedding features of each primitive according to the previous steps, the gated vector raster map attention fusion module can be used to fuse the vector embedding features and image embedding features, and then combined with the layer feature fusion module to obtain the corresponding joint feature representation.

[0057] Furthermore, referring to Figure 6 Step S700 may include steps S710 and S720, as detailed below: Step S710: Weight the vector embedding features of primitives in each vector sub-image, and then concatenate the reweighted vector embedding features with the image embedding features and map them to a unified dimension to obtain the first fusion feature; Step S720: For the first fusion feature of each primitive in the same layer, calculate the average pooling feature, the max pooling feature, and the attention-weighted pooling feature respectively, and aggregate the three to form the global feature of the layer. Write the global feature back to each primitive belonging to the same layer to obtain the joint feature representation.

[0058] In this embodiment, in order to adaptively adjust the degree of modulation of image information on vector information, a gating weight can be calculated for each primitive. : in, These are, respectively, the gating weight, the vector feature fusion weight, and the image feature fusion weight. These are vector embedding features and image embedding features, respectively. and These are all common neural network activation functions.

[0059] Then, the reweighted vector embedding features can be concatenated with the image embedding features and mapped to a unified dimension to obtain the first fused feature. : in, The mapping matrix is ​​a matrix with updatable parameters defined in the neural network model. It is used to map the feature dimension to a new dimension. The training principle of the neural network model is to update various parameters. This refers to the connection operation in a neural network model, which is a matrix calculation operation. The circle represents the matrix dot product, a common matrix calculation operation.

[0060] For a collection of primitives on the same layer It can calculate layer-level global features. First, the attention weight of primitive i can be calculated using the following formula. : in, The first fusion feature of primitive j in layer l. To blend weights for layers, This is the first fusion feature of primitive i in layer l.

[0061] Then, for each layer l, the average pooling feature is calculated based on the first fusion feature of all primitives contained therein. Maximize pooling features Attention-weighted pooling features The three elements are aggregated and used as the global feature of the layer. .

[0062] Finally, the global features are written back to each primitive belonging to the same layer, thus obtaining the joint feature representation. : in, Let i be the first fusion feature of primitive i. This refers to the global features of layer l occupied by primitive i. For the mapping matrix, The connection operation in the neural network model works on the same principle as above, and will not be repeated here.

[0063] Step S900: Perform attention-based decoding on the joint feature representation, and generate panoramic segmentation results of the architectural drawings based on the decoding results.

[0064] Once the joint feature representation of each primitive in each vector sub-map is obtained, it can be input into a decoder based on self-attention and cross-attention (CAD decoder). Based on the decoding results, a panoramic segmentation result of the architectural drawing can be generated.

[0065] Reference Figure 7 In this embodiment, step S900 may further include steps S910 to S930, as follows: Step S910: Input the joint feature representation into a decoder based on self-attention and cross-attention mechanisms to output the semantic prediction result, instance mask, instance category, and instance confidence of each primitive; The decoder based on self-attention and cross-attention mechanisms can include multiple decoding blocks. Each decoding block sequentially performs query self-attention, cross-attention between the query and joint features, and feedforward network computation, ultimately outputting the semantic category (i.e., semantic prediction result), instance category, instance mask, and instance confidence score for each primitive. In this embodiment, the decoder can use an iterative prediction mechanism to optimize the prediction results layer by layer.

[0066] During the training phase of the decoder, an initial query can be selected from primitive features based on a query threshold; during the evaluation phase of the decoder, all primitive features can be used as query input.

[0067] Specifically, during the training phase, a subset of initial queries (assuming the number is Q) are randomly selected based on a query threshold. These queries are then decoded to obtain the semantic category logits of all graph primitives, Q instance category logits, instance mask logits, and instance confidence scores. The semantic category logits of all graph primitives are used to calculate the semantic loss. The Q instance mask logits are matched against real instances (assuming N instances), thus retaining N instance category logits, instance mask logits, and instance confidence scores for instance loss calculation.

[0068] During the evaluation phase, the initial query for all primitives (assuming P primitives) serves as input. The decoder then outputs the semantic category logits for all primitives, the instance category logits for the P instances, the instance mask logits, and the instance confidence scores. The semantic category logits for all primitives determine the semantic category of each primitive. Based on the instance confidence scores, several high-confidence results can be selected from the P instance calculations to obtain the mask coverage and instance categories for several instances.

[0069] Step S920: Based on the instance mask and instance confidence, perform instance-aware voting correction on the semantic prediction results to obtain the corrected slice-level panoramic segmentation results; After the decoder obtains the semantic prediction result, instance mask, instance category, and instance confidence score for each primitive, the semantic prediction score for each primitive can be updated using the instance mask and instance confidence score. If the maximum support score obtained by a primitive in the instance branch is higher than the corresponding score in the semantic branch, the original semantic label is overwritten with the instance label. Then, a vote is performed on the semantics of the primitives within each instance mask to determine the unified semantic category of the instance, and primitives whose semantic categories are inconsistent with the instance categories are removed from the current instance mask through a mask reshaping operation. This completes the instance-aware voting correction, resulting in the corrected slice-level panoramic segmentation result.

[0070] Step S930: Based on the corrected slice-level panoramic segmentation results, perform cross-slice secondary voting aggregation on the semantic prediction results of the same primitives in different vector sub-maps belonging to the same vector paper, and output the paper-level panoramic segmentation results.

[0071] Since the same original vector paper is divided into multiple overlapping slice windows, the same primitive (with the same handleId) may be predicted repeatedly in multiple windows. Therefore, the prediction results of all vector sub-graphs belonging to the same original vector paper can be collected, and primitives with the same unique primitive identifier (handleId) can be regarded as the same target. The final paper-level semantic category can be determined by majority voting, that is, cross-slice secondary voting aggregation can be performed, and finally a stable paper-level panoramic segmentation result can be output.

[0072] During the training phase of the decoder model used in step S900, the total loss function can be composed of a weighted average of the instance branch loss and the semantic branch loss: in, Represents the total loss. These represent instance loss and semantic loss, respectively. These represent the instance loss weight and the semantic loss weight, respectively.

[0073] The instance branch loss further includes classification loss, masking loss, and instance score loss: in, These represent the instance class classification loss, mask prediction binary classification loss, and instance confidence regression loss, respectively. These represent the instance category classification loss weight, the mask prediction binary classification loss weight, and the instance confidence regression loss weight, respectively.

[0074] The semantic branch loss uses focus loss: in, This represents the predicted probability for the true class t. represents the balance coefficient and the focus coefficient, respectively, and CE represents the semantic prediction cross-entropy loss.

[0075] To alleviate class imbalance, class weights can be constructed by combining batch statistics, global class frequency statistics, or exponential moving average statistics to improve the recognition performance of minority class components. Simultaneously, at the data sampling level, sample weights can be constructed based on class frequencies in the training set, increasing the sampling probability of high-weight minority class samples, and enhancing the magnitude of random flipping, translation, and scaling perturbations during repeated sampling.

[0076] The data processing method provided in this embodiment converts the original architectural drawings into vector graphics with the same graphic features and layer distribution as the architectural drawings, while maintaining the mapping relationship between the entities in the architectural drawings and the primitives in the vector graphics. Sliding window slicing is then performed to obtain vector sub-images and corresponding raster images. This method simultaneously preserves the structured vector information and rasterized visual information of the drawings, laying a data foundation for subsequent multimodal fusion processing. Based on this, vector embedding features are extracted from the vector sub-images and image embedding features are extracted from the raster images, and the two are fused across modally to obtain a joint feature representation that integrates precise geometric structure and texture context. Finally, a panoramic segmentation result of the architectural drawings is generated through attention-based decoding. Compared with existing technologies, this application eliminates the need for simple splicing or switching between vector and image modalities. Instead, it achieves the fusion of information from both vector and image modalities, enabling more full utilization of the geometric precision of vector information and the texture semantics of image information, thereby improving the accuracy and robustness of component recognition and panoramic segmentation in architectural drawings.

[0077] Based on the same inventive concept, in another embodiment, a computer device is provided, including at least one processor and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the data processing method described above.

[0078] The computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores various types of data involved in the data processing method. The network interface communicates with external terminals via a network connection. When the computer program is executed by the processor, it implements a data processing method.

[0079] Based on the same inventive concept, in another embodiment, a computer-readable storage medium is also provided, on which a computer program is stored, which, when executed by a processor, implements the steps of the method described above.

[0080] Based on the same inventive concept, in another embodiment, a computer program product is also provided, including a computer program that, when executed by a processor, implements the steps of the method described above.

[0081] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0082] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0083] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A data processing method, characterized in that, include: The original architectural drawings are converted into vector graphics. The vector graphics have the same graphic features and layer distribution as the architectural drawings, and maintain the mapping relationship between the entities in the architectural drawings and the primitives in the vector graphics. During the conversion process, the unique identifier of the entity in the architectural drawing is retained, and the unique identifier is written into the source code of the corresponding primitive in the vector graphics. The vector paper is sliced ​​using a sliding window to obtain multiple vector sub-images and the raster image corresponding to each vector sub-image; Vector embedding features are obtained based on each of the vector sub-graphs, and image embedding features are obtained based on each of the raster images; The vector embedding features and the image embedding features are fused across modally to obtain a joint feature representation; The joint feature representation is decoded using an attention mechanism, and a panoramic segmentation result of the architectural drawing is generated based on the decoding result. This includes: inputting the joint feature representation into a decoder based on self-attention and cross-attention mechanisms to output the semantic prediction result, instance mask, instance category, and instance confidence score for each primitive; performing instance-aware voting correction on the semantic prediction result based on the instance mask and instance confidence score to obtain a corrected slice-level panoramic segmentation result; based on the corrected slice-level panoramic segmentation result, obtaining the unique identifier corresponding to the primitive in each vector sub-image, identifying primitives with the same unique identifier in different vector sub-images as the same target primitive, performing cross-slice secondary voting aggregation on the semantic prediction results of the same target primitive in different vector sub-images, and outputting a drawing-level panoramic segmentation result.

2. The data processing method according to claim 1, characterized in that, The steps for converting original-format architectural drawings into vector graphics include: Based on the layers, different rendering methods are used for different types of entities in the architectural drawings to generate a first vector drawing with the same graphic features, layer distribution, and maintaining the entity mapping relationship of the architectural drawings; The first vector paper is subjected to uniform fitting processing to obtain a second vector paper suitable for sliding window slicing.

3. The data processing method according to claim 2, characterized in that, The step of performing sliding window slicing on the vector paper to obtain multiple vector sub-images and the raster image corresponding to each vector sub-image includes: By using a fixed window size and sliding step, overlapping sliding window slices are applied to the second vector map to obtain multiple vector sub-maps; Each of the vector sub-images is rendered to obtain a raster image corresponding to each vector sub-image.

4. The data processing method according to claim 1, characterized in that, The step of obtaining vector embedding features based on each of the vector subgraphs includes: Discrete sampling is performed on the primitives in each of the vector sub-graphs to construct vector samples; The vector samples are input into the primitive sequence encoder to extract the vector embedding features of each primitive.

5. The data processing method according to claim 4, characterized in that, The vector sample includes the sampling coordinates, geometric attributes, layer attributes, and the mapping coordinates of the primitive center to the center of its corresponding raster image for each primitive.

6. The data processing method according to claim 1, characterized in that, The step of obtaining image embedding features based on each of the raster images includes: The raster image is input into a multi-scale visual backbone network to obtain a multi-scale image feature map. Based on the spatial location information of each primitive, multi-scale image embedding features corresponding to each primitive are extracted from the multi-scale image feature map.

7. The data processing method according to claim 1, characterized in that, The step of performing cross-modal fusion of the vector embedding features and the image embedding features to obtain a joint feature representation includes: The vector embedding features of primitives in each vector sub-image are weighted, and the reweighted vector embedding features are concatenated with the image embedding features and mapped to a unified dimension to obtain the first fusion feature; For the first fusion feature of each primitive in the same layer, the average pooling feature, max pooling feature and attention-weighted pooling feature are calculated respectively, and the three are aggregated to form the global feature of the layer. The global feature is then written back to each primitive belonging to the same layer to obtain the joint feature representation.

8. A computer device, characterized in that, include: At least one processor; And a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the data processing method as described in any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the data processing method according to any one of claims 1 to 7.

Images