Building element information processing method and device and computer equipment

Abstract

The invention provides a building element information processing method and device and computer equipment, and belongs to the technical field of data classification and extraction. Comprising the following steps: acquiring historical image data of a chlamydogenous site, and training according to the historical image data to obtain a building element extraction and classification model; acquiring image data of the target ruins; performing data preprocessing on the image data to obtain a target image; and inputting the target image into a building element extraction and classification model, extracting feature information of the target image by the building element extraction and classification model, and outputting target building elements contained in the target ruins and target building element classification information based on the feature information, the building element extraction and classification model at least comprises an optic nerve backbone network and a feature enhancement network, and the feature information comprises color features, texture features, shape features and layout features. According to the method, the effects of improving the building element extraction efficiency of the wall ruins and improving the classification accuracy of the building elements can be achieved.

Description

Technical Field

[0001] The present application relates to the technical field of data classification and extraction, and in particular to a method, device and computer equipment for processing building element information. Background Art

[0002] As witnesses of history, city wall sites carry rich cultural heritage and historical stories. City wall sites are not only the boundaries of ancient cities, but also the marks of the development of ancient civilizations. The architectural elements of city wall sites are important objects of investigation and research. The extraction and classification of architectural elements will seriously affect the research results of city wall sites. Therefore, a high-precision architectural element extraction method is urgently needed.

[0003] In the relevant technologies, the architectural elements of city wall sites mainly include: city wall, city gate, battlement, moat, corner tower, etc. The architectural elements of city wall sites mainly rely on manual acquisition. Relevant staff obtain the plane layout, height, size and other data of city wall sites through on-site investigation, preliminary evaluation, literature review, data collection and measurement with measuring tools. Based on the plane layout, height, size and other data of city wall sites obtained manually, the architectural elements of city wall sites are extracted, such as the plane shape, size, structural characteristics, decorative elements, etc. of the city wall.

[0004] However, when extracting architectural elements from city wall sites based on relevant technologies, the process of extracting architectural elements is complicated and is constrained by multiple factors such as measurement technology limitations and the subjective consciousness of staff, resulting in low efficiency in extracting architectural elements from city wall sites. Summary of the invention

[0005] The purpose of this application is to provide a method, device and computer equipment for processing architectural element information, which can improve the efficiency of architectural element extraction from city wall ruins and improve the classification accuracy of architectural elements.

[0006] The embodiment of the present application is implemented as follows:

[0007] In an initial aspect of an embodiment of the present application, a method for processing building element information is provided, the method comprising:

[0008] Obtain historical image data of city wall sites, and train a building element extraction and classification model based on the historical image data;

[0009] Obtain image data of the target site, including: planar image data and remote sensing image data;

[0010] Perform data preprocessing on the image data to obtain a target image;

[0011] The target image is input into the architectural element extraction and classification model, and the architectural element extraction and classification model extracts the feature information of the target image, and outputs the target architectural elements and target architectural element classification information contained in the target site based on the feature information, wherein the architectural element extraction and classification model includes at least a visual neural backbone network and a feature enhancement network, and the feature information includes: color features, texture features, shape features and layout features.

[0012] As a possible implementation method, a building element extraction and classification model is trained based on historical image data, including:

[0013] Divide the historical image data into training data set, test data set and verification data set;

[0014] According to the training data set, the initial extraction and classification model is obtained through iterative training;

[0015] Testing the initial extraction classification model based on the test data set, and determining whether the initial extraction classification model is qualified based on the test result;

[0016] The initial extraction classification model is verified based on the verification data set, and whether the initial extraction classification model is overfitted is determined based on the verification result.

[0017] As a possible implementation method, based on the test results, determining whether the initial extraction classification model is qualified includes:

[0018] Inputting the test data set into the initial extraction classification model based on the preset batch capacity value, the initial extraction classification model outputs the corresponding first building element and the first building element classification information;

[0019] Whether the initial extraction classification model is qualified is determined according to the matching degree between the first building element and the test building element corresponding to the test data set and the matching degree between the first building element classification information and the test building element classification information corresponding to the test data set.

[0020] As a possible implementation manner, determining whether the initial extraction classification model is qualified according to the matching degree between the first building element and the test building element corresponding to the test data set and the matching degree between the first building element classification information and the test building element classification information corresponding to the test data set includes:

[0021] If the first building element is the same as the test building element corresponding to the test data set, and the first building element classification information is the same as the test building element classification information corresponding to the test data set, then the initial extraction classification model is qualified;

[0022] If the first building element is different from the test building element corresponding to the test data set, and / or the first building element classification information is different from the test building element classification information corresponding to the test data set, then the initial extraction classification model is unqualified.

[0023] As a possible implementation method, the initial extraction classification model is verified based on the verification data set, and according to the verification result, it is determined whether the initial extraction classification model is overfitting, including:

[0024] Based on a preset batch capacity value, the verification data set is input into an initial extraction classification model, and the initial extraction classification model outputs the corresponding second building element and second building element classification information;

[0025] Whether the initial extraction classification model is overfitted is determined according to the matching degree between the second building element and the verification building element corresponding to the verification data set and the matching degree between the second building element classification information and the verification building element classification information corresponding to the verification data set.

[0026] As a possible implementation manner, determining whether the initial extraction classification model is overfitting according to the matching degree between the second building element and the verification building element corresponding to the verification data set and the matching degree between the second building element classification information and the verification building element classification information corresponding to the verification data set includes:

[0027] If the second building element is the same as the verification building element corresponding to the verification data set, and the second building element classification information is the same as the verification building element classification information corresponding to the verification data set, then the initial extraction classification model is not overfitting;

[0028] If the second building element is different from the verification building element corresponding to the verification data set, and / or the second building element classification information is different from the verification building element classification information corresponding to the verification data set, the initial extraction classification model is overfitted.

[0029] As a possible implementation method, according to the training data set, iterative training is performed to obtain an initial extraction classification model, including:

[0030] Extract feature information corresponding to the training data set, the feature information includes: color features, texture features, shape features and layout features;

[0031] According to the pre-constructed loss function and feature information, the loss of feature information is balanced to obtain a target loss function, wherein the target loss function includes: a position loss function, a classification loss function and a confidence loss function;

[0032] Based on the training data set and the target loss function, an initial extraction classification model is obtained through iterative training, wherein the number of training data sets used in each iteration is determined according to a preset batch capacity.

[0033] As a possible implementation method, based on the training data set and the target loss function, iterative training is performed to obtain an initial extraction classification model, including:

[0034] According to the preset batch capacity, multiple target training data corresponding to the current iteration round are determined from the training data set;

[0035] Based on the target loss function and multiple target training data, determine the loss result of the to-be-trained model in the current iteration round;

[0036] According to the loss result of the model to be trained in the current iteration round, the model to be trained in the current iteration round is corrected.

[0037] According to a second aspect of an embodiment of the present application, a building element information processing device is provided, the device comprising:

[0038] The first acquisition module is used to acquire historical image data of city wall sites and train a building element extraction and classification model based on the historical image data;

[0039] The second acquisition module is used to acquire image data of the target site, the image data including: planar image data and remote sensing image data;

[0040] A data processing module is used to perform data preprocessing on the image data to obtain a target image;

[0041] The extraction and classification module is used to input the target image into the building element extraction and classification model, and the building element extraction and classification model extracts the feature information of the target image, and outputs the target building elements and target building element classification information contained in the target site based on the feature information, wherein the building element extraction and classification model includes at least a visual neural backbone network and a feature enhancement network, and the feature information includes: color features, texture features, shape features and layout features.

[0042] According to a third aspect of an embodiment of the present application, a computer device is provided, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed by the processor, the method for processing building element information described in the first aspect is implemented.

[0043] According to a fourth aspect of an embodiment of the present application, a computer-readable storage medium is provided, which stores a computer program. When the computer program is executed by a processor, the building element information processing method described in the first aspect is implemented.

[0044] The beneficial effects of the embodiments of the present application include:

[0045] The embodiment of the present application provides a method for processing architectural element information, which obtains a large amount of historical image data of city wall ruins, and obtains an architectural element extraction and classification model based on the historical image data training and learning; obtains image data such as plane image data and remote sensing image data of the target ruins, and obtains the target image after data preprocessing of the image data; inputs the target image into the architectural element extraction and classification model obtained by pre-training and learning, and the architectural element extraction and classification model extracts the feature information of the target image based on the visual neural backbone network and the feature enhancement network, and the architectural element extraction and classification model outputs the target architectural elements and the classification information of the target architectural elements that are consistently contained in the target based on the feature information of the target image. Among them, the feature information of the target image includes: color features, texture features, shape features and layout features. In this way, the efficiency of architectural element extraction of city wall ruins can be improved, and the classification accuracy of architectural elements can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0047] Figure 1 A flowchart of a first method for processing building element information provided in an embodiment of the present application;

[0048] Figure 2 A flowchart of a second method for processing building element information provided in an embodiment of the present application;

[0049] Figure 3 A flowchart of a third method for processing building element information provided in an embodiment of the present application;

[0050] Figure 4 A flowchart of a fourth method for processing building element information provided in an embodiment of the present application;

[0051] Figure 5 A flowchart of a fifth method for processing building element information provided in an embodiment of the present application;

[0052] Figure 6 A flowchart of a sixth method for processing building element information provided in an embodiment of the present application;

[0053] Figure 7 A flowchart of a seventh method for processing building element information provided in an embodiment of the present application;

[0054] Figure 8A flowchart of an eighth method for processing building element information provided in an embodiment of the present application;

[0055] Fig. 9 A system block diagram of a building element extraction and classification model provided in an embodiment of the present application;

[0056] Fig.10 A curve diagram of a training classification loss function provided in an embodiment of the present application;

[0057] Fig.11 A curve diagram of a verification classification loss function provided in an embodiment of the present application;

[0058] Fig.12 A curve diagram of a training position loss function provided in an embodiment of the present application;

[0059] Fig.13 A curve diagram of a verification position loss function provided in an embodiment of the present application;

[0060] Fig.14 A curve diagram of a training confidence loss function provided in an embodiment of the present application;

[0061] Fig.15 A curve diagram of a verification confidence loss function provided in an embodiment of the present application;

[0062] Fig.16 A schematic diagram of the structure of a building element information processing device provided in an embodiment of the present application;

[0063] Fig.17 A schematic diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0064] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations.

[0065] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for which protection is sought, but merely represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0066] At present, the relevant staff of the cultural relics protection organization obtain the data of the plane layout, height, size and other data of the city wall ruins through on-site investigation, preliminary assessment, literature review, data collection and measurement with measuring tools, and extract the architectural elements of the city wall ruins, such as the plane shape, size, structural characteristics, decorative elements, etc. of the city wall, based on the manually obtained data of the plane layout, height, size and other data of the city wall ruins. However, this scheme relies on manual acquisition, the process of extracting architectural elements is complicated, and it is affected by the subjective consciousness of the staff, which makes the extraction efficiency of architectural elements low.

[0067] To this end, an embodiment of the present application provides a method for processing architectural element information, which obtains a large amount of historical image data of city wall ruins, trains and learns to obtain an architectural element extraction and classification model; obtains image data of the target ruins, and performs data preprocessing to obtain a target image; inputs the target image into the architectural element extraction and classification model, and the architectural element extraction and classification model extracts feature information of the target image, and outputs target architectural elements and target architectural element classification information contained in the target ruins based on the feature information. Among them, the architectural element extraction and classification model includes at least a visual neural backbone network and a feature enhancement network to improve the efficiency of feature information extraction. In this way, the effect of improving the efficiency of architectural element extraction of city wall ruins and improving the classification accuracy of architectural elements can be achieved.

[0068] The building element information processing method provided in the embodiment of the present application is explained in detail below with reference to the accompanying drawings.

[0069] Figure 1 This is a flowchart of a method for processing building element information provided by this application. The method can be applied to computer equipment. Figure 1 The present application embodiment provides a method for processing building element information, including:

[0070] S101, obtaining historical image data of city wall ruins, and training a building element extraction classification model based on the historical image data.

[0071] Optionally, historical image data refers to image sample data randomly selected from image data of city wall sites taken or monitored in historical periods. The historical image data of city wall sites can be taken on-site by relevant staff, or remote sensing image data of city wall sites can be collected using satellite remote sensing technology.

[0072] Optionally, the historical image data is used as sample data for training and learning to obtain a building element extraction and classification model, and the building element extraction and classification model is used to extract building elements from the building image data and classify the extracted building elements.

[0073] S102, acquiring image data of the target site, the image data including: planar image data and remote sensing image data.

[0074] Optionally, the target site is a site randomly selected by the user from city wall sites, and the image data of the target site can be obtained by relevant staff taking real-time architectural photos on site, or by collecting remote sensing image data in real time using satellite remote sensing technology.

[0075] Optionally, the plane image data is used to indicate the building photos taken on site by the relevant staff using the shooting equipment. The plane image data is a two-dimensional image; the remote sensing image data is collected based on satellite remote sensing technology. As a three-dimensional image of the target site, the remote sensing image data can stereoscopically display the architectural features of the target site. It is worth noting that the plane image data is a high-resolution photo.

[0076] S103: Perform data preprocessing on the image data to obtain a target image.

[0077] Optionally, the image data is preprocessed, that is, the image data is denoised and contrast enhanced to improve the accuracy of the subsequent analysis of the image data. The image data is denoised and blurred and invalid image data is deleted.

[0078] Optionally, the target image refers to image data obtained after noise reduction, deduplication, contrast enhancement and other data processing of planar image data and remote sensing image data. The target image does not indicate a certain image of the image data of the target site, but refers to the valid image data of the target site. The target image can be multiple image data.

[0079] Optionally, the image size of the target image is unified, and the image size of the target image may be 608×608, which is not specifically limited in the present application.

[0080] S104. Input the target image into a building element extraction and classification model, extract feature information of the target image by the building element extraction and classification model, and output target building elements and target building element classification information contained in the target site based on the feature information, wherein the building element extraction and classification model includes at least a visual neural backbone network and a feature enhancement network, and the feature information includes: color features, texture features, shape features and layout features.

[0081] Optionally, the feature information is used to indicate image features contained in the target image, and the categories of the feature information include: color features, texture features, shape features, and layout features. Among them, the color feature is used to indicate the color information contained in the building contained in the target image; the texture feature is used to indicate the surface texture of the building contained in the target image; the shape feature is used to indicate the external shape of the building contained in the target image; and the layout feature is used to indicate the layout position of the building contained in the target image in the target site.

[0082] Optionally, the target building element is used to indicate the building function, material and technical conditions, building image, building shape and layout of the building contained in the target image. Among them, the building function is used to indicate the practicality of the building, and the building function of the building is determined by the use characteristics of the building; the material and technical conditions are used to indicate the building materials used in the building, the structural technology of the building, the construction methods of the building, etc.; the building image is used to indicate the size of the building, the three-dimensional form of the building, the color used in the building, and the texture of the building; the building shape and layout are used to indicate the plane layout of the building, such as the single-body layout is generally a rectangular plane, and the group layout is based on the courtyard enclosure.

[0083] Optionally, the target building element classification information is used to indicate the classification category of the target building element. For example, the classification of building elements of city wall ruins mainly includes: city wall body, city gate, battlement, moat (moat), corner tower, city tower, enemy tower (enemy tower), battlement (baluster wall), parapet, etc. This application does not make any specific limitation on this.

[0084] Optionally, after the target image is input into the building element extraction and classification model, the building element extraction and classification model parses the target image to obtain feature information corresponding to the target image, extracts the target building elements contained in the target image based on the feature information of the target image, and outputs the target building element classification information based on the target building elements.

[0085] Optionally, the architectural element extraction and classification model includes at least a visual neural backbone network and a feature enhancement network, wherein the visual neural backbone network is used as the feature extraction backbone network of the architectural element extraction and classification model, and the visual neural backbone network can be implemented by CSPDarknet53, and the visual neural backbone network has a stronger feature extraction capability for city wall type ruins. In addition, the feature enhancement network is used to enhance the feature information of the target image, and the feature enhancement network is often implemented by a spatial pyramid pooling structure (Spatial Pyramid Pooling, referred to as SPP) and a feature extraction network PANet (Path Aggregation Network). SPP can perform spatial pyramid pooling on the feature information of the target image, which can effectively avoid image distortion caused by operations such as cropping and scaling of the target image, and can effectively avoid repeated extraction of feature information of the target image by the convolutional neural network; PANet can perform path aggregation on the feature information of the target image, can aggregate and fuse the feature information extracted by different detection layers, and effectively fuse target images of different resolutions.

[0086] It is worth noting that since the scales of the feature information corresponding to the image data of city wall ruins are inconsistent, detection layers of various sizes are deployed in the architectural element extraction classification model to adapt to the feature information extraction of city wall ruins.

[0087] In an embodiment of the present application, a large amount of historical image data of city wall ruins is obtained, and a building element extraction classification model is obtained based on training and learning of the historical image data; image data such as plane image data and remote sensing image data of the target ruins are obtained, and the image data is preprocessed to obtain the target image; the target image is input into the building element extraction classification model obtained by pre-training and learning, and the building element extraction classification model extracts the feature information of the target image based on the visual neural backbone network and the feature enhancement network, and the building element extraction classification model outputs the target building elements and the classification information of the target building elements that are consistently contained in the target based on the feature information of the target image. Among them, the feature information of the target image includes: color features, texture features, shape features and layout features. In this way, the efficiency of extracting building elements of city wall ruins can be improved, and the classification accuracy of building elements can be improved.

[0088] For a possible implementation, see Figure 2 The operation of step S101 may specifically be:

[0089] S201, dividing the historical image data into a training data set, a test data set and a verification data set.

[0090] Optionally, data partitioning is used to indicate how many parts the historical image data should be split into. The user can randomly set the data partition ratio, such as using 1 / 3 of the historical image data as a training data set, 1 / 3 of the historical image data as a test data set, and 1 / 3 of the historical image data as a verification data set; or using 40% of the historical image data as a training data set, 30% of the historical image data as a test data set, and 30% of the historical image data as a verification data set, etc. This application does not make any specific limitations on this.

[0091] Optionally, the training data set includes multiple historical image data for iterative training to obtain a building feature extraction classification model, the test data set includes multiple historical image data for iterative testing the performance of the building feature extraction classification model, and the verification data set includes multiple historical image data for iterative verification whether the building feature extraction classification model is overfitting.

[0092] S202: According to the training data set, iterative training is performed to obtain an initial extraction and classification model.

[0093] Optionally, based on the historical image data contained in the training data set, the deep neural network model is iteratively trained to obtain an initial extraction classification model. The initial extraction classification model is used to indicate the deep neural network model after iterative training of the training data set. The initial extraction classification model is mainly used to extract architectural elements of city wall ruins and classify the architectural elements.

[0094] S203: Test the initial extraction classification model based on the test data set, and determine whether the initial extraction classification model is qualified based on the test result.

[0095] Optionally, a test data set is input into the initial extraction classification model, and whether the initial extraction classification model is qualified is determined based on the quality of the test result output by the initial extraction classification model.

[0096] S204: verifying the initial extraction classification model based on the verification data set, and determining whether the initial extraction classification model is overfitting according to the verification result.

[0097] Optionally, after the initial extraction classification model is qualified, a verification data set is input into the initial extraction classification model, and based on the verification result output by the initial extraction classification model, it is determined whether the initial extraction classification model is overfitted.

[0098] For a possible implementation, see Figure 3 The operation of step S203 may specifically be:

[0099] S301. Input a test data set into an initial extraction classification model based on a preset batch capacity value, and the initial extraction classification model outputs a corresponding first building element and first building element classification information.

[0100] Optionally, the preset batch capacity is the size of the data required for a single iteration of the initial extraction classification model set in advance by the user. The preset batch capacity refers to the size of the historical image data required for a single iteration of training the deep neural network model. The preset batch capacity can be 16, 32, etc., and this application does not make any specific limitations on this.

[0101] Optionally, the first building element is used to indicate the multiple building elements extracted by the initial extraction classification model for each historical image data contained in the test data set, and the first building element classification information is used to indicate the classification of the multiple building elements of each historical image data contained in the test data set by the initial extraction classification model.

[0102] S302: Determine whether the initial extracted classification model is qualified according to the matching degree between the first building element and the test building element corresponding to the test data set and the matching degree between the first building element classification information and the test building element classification information corresponding to the test data set.

[0103] Optionally, the test building elements are used to indicate the standard building elements corresponding to each historical image data in the test data set, and the standard building elements are building elements that have been manually verified to be correct in advance; the test building element classification information is used to indicate the classification of the standard building elements corresponding to each historical image data in the test data set.

[0104] Optionally, the performance of the initial extraction classification model is verified according to two dimensions, namely, the matching degree between the first building element and the test building element and the matching degree between the first building element classification information and the test building element classification information.

[0105] For a possible implementation, see Figure 4 The operation of step S302 may specifically be:

[0106] S401. If the first building element is identical to the test building element corresponding to the test data set, and the classification information of the first building element is identical to the classification information of the test building element corresponding to the test data set, then the initial extraction classification model is qualified.

[0107] Optionally, when the first building element output by the initial extraction classification model is exactly the same as the test building element, and the first building element classification information output by the initial extraction classification model is exactly the same as the test building element classification information, the initial extraction classification model is deemed qualified.

[0108] S402: If the first building element is different from the test building element corresponding to the test data set, and / or the first building element classification information is different from the test building element classification information corresponding to the test data set, then the initial extraction classification model is unqualified.

[0109] Optionally, when the first building element output by the initial extraction and classification model is different from the test building element and / or the first building element classification information output by the initial extraction and classification model is different from the test building element classification information, the initial extraction and classification model is deemed unqualified, that is, there is a deviation in the extraction and classification of building elements by the initial extraction and classification model.

[0110] For a possible implementation, see Figure 5 The operation of step S204 may specifically be:

[0111] S501. Inputting a verification data set into an initial extraction and classification model based on a preset batch capacity value, and the initial extraction and classification model outputs a corresponding second building element and second building element classification information.

[0112] Optionally, the second building element is used to indicate the multiple building elements extracted by the initial extraction classification model for each historical image data contained in the verification data set, and the second building element classification information is used to indicate the classification of the multiple building elements of each historical image data contained in the verification data set by the initial extraction classification model.

[0113] S502: Determine whether the initial extraction classification model is overfitting according to the matching degree between the second building element and the verification building element corresponding to the verification data set and the matching degree between the second building element classification information and the verification building element classification information corresponding to the verification data set.

[0114] Optionally, the verification building element is used to indicate the standard building element corresponding to each historical image data in the verification data set; the verification building element classification information is used to indicate the classification of the standard building element corresponding to each historical image data in the verification data set.

[0115] Optionally, whether the initial extraction classification model is overfitted is verified based on two dimensions, namely, the matching degree between the second building element and the verification building element and the matching degree between the second building element classification information and the verification building element classification information.

[0116] For a possible implementation, see Figure 6 The operation of step S502 may specifically be:

[0117] S601: If the second building element is identical to the verification building element corresponding to the verification data set, and the second building element classification information is identical to the verification building element classification information corresponding to the verification data set, then the initial extraction classification model is not overfitted.

[0118] Optionally, when the second building element output by the initial extraction classification model is exactly the same as the verified building element, and the second building element classification information output by the initial extraction classification model is exactly the same as the verified building element classification information, the initial extraction classification model is not overfitted, and the initial extraction classification model can be regarded as a building element extraction classification model.

[0119] S602: If the second building element is different from the verification building element corresponding to the verification data set, and / or the second building element classification information is different from the verification building element classification information corresponding to the verification data set, the initial extraction classification model is overfitted.

[0120] Optionally, when the second building element output by the initial extraction classification model is different from the verified building element and / or the second building element classification information output by the initial extraction classification model is different from the verified building element classification information, the initial extraction classification model is deemed to be overfitted, that is, the parameter information in the building element of the initial extraction classification model is overfitted, and the training process of the initial extraction classification model should be stopped immediately.

[0121] For a possible implementation, see Figure 7 The operation of step S202 may specifically be:

[0122] S701. Extract feature information corresponding to a training data set, where the feature information includes: color features, texture features, shape features, and layout features.

[0123] Optionally, feature information of each historical image data contained in the training data set is extracted through a visual neural backbone network and a feature enhancement network.

[0124] S702. According to the pre-constructed loss function and feature information, the loss of feature information is balanced to obtain a target loss function, wherein the target loss function includes: a position loss function, a classification loss function and a confidence loss function.

[0125] Optionally, the pre-constructed loss function is a multi-loss function set based on the characteristic information, structural materials and layout characteristics of city wall ruins. City wall ruins have the characteristics of similar features, similar structural materials and obviously different architectural layouts.

[0126] Optionally, the following formula (1) is a pre-constructed loss function, and the minimum loss function is obtained according to the following formula (1) and the feature information of the historical image data contained in the training data set.

[0127] Loss=λcoord×Coord_Loss+λobj×Obj_Loss+λclass×Class_Loss(1)

[0128] It is worth noting that Loss is used to represent the minimum loss function, λcoord is used to represent the weight value of the position loss function, Coord_Loss is used to represent the position loss, λobj is used to represent the weight value of the confidence loss function, Obj_Loss is used to represent the confidence loss, λclass is used to represent the weight value of the classification loss function, and Class_Loss is used to represent the classification loss.

[0129] Optionally, according to the pre-constructed loss function and feature information, the loss of feature information is adjusted from multiple dimensions to obtain a target loss function. The target loss function is used to indicate the loss function corresponding to the minimum loss when the result of the target loss function is the minimum loss. The target loss function includes: a position loss function, a classification loss function and a confidence loss function. The position loss function is used to calculate the deviation between the coordinate position, height, and width of the image border of the historical image data input into the initial extraction classification model and the coordinate position, height, and width of the actual image border of the historical image data; the classification loss function is used to calculate the deviation between the building element category output by the initial extraction classification model and the actual classification of the building elements contained in the historical image data in the training data set; the confidence loss function is used to calculate the probability that the building elements and building element classifications output by the initial extraction classification model are the same as the actual building elements and building element classifications.

[0130] S703: Based on the training data set and the target loss function, iterative training is performed to obtain an initial extraction classification model, wherein the number of training data sets used in each iteration is determined according to a preset batch capacity.

[0131] Optionally, according to the training data set and the target loss function, an initial extraction classification model is obtained by iterative training, and the batch size of the historical image data in the training data set used in each iteration is determined by a preset batch capacity.

[0132] For a possible implementation, see Figure 8 The operation of step S703 may specifically be:

[0133] S801. Determine a plurality of target training data corresponding to a current iteration round from a training data set according to a preset batch capacity.

[0134] Optionally, the current iteration round is used to indicate the period of the current iteration training initial extraction classification model. According to the preset batch capacity, historical image data matching the preset batch capacity quantity is randomly selected from the training data set, and these historical image data are used as target training data.

[0135] S802: Based on the target loss function and multiple target training data, determine the loss result of the to-be-trained model in the current iteration round.

[0136] Optionally, the model to be trained is used to indicate an initial deep neural network model. The loss result of the model to be trained can be calculated according to the target loss function and the target training data. The loss result is used to indicate the loss amount corresponding to the feature information of the target training data.

[0137] S803. Correct the model to be trained in the current iteration round according to the loss result of the model to be trained in the current iteration round.

[0138] Optionally, according to the loss result of the model to be trained, the parameter information of the model to be trained in the current iteration round is corrected in real time. The adjustment parameters of the model to be trained include: loss function and learning rate. The initial learning rate of the model to be trained can be set to 0.001, and the learning rate of the model to be trained can be updated in real time according to the derivative of the result of the target loss function adjusted in real time.

[0139] Fig. 9 A system block diagram of a building element extraction and classification model provided in an embodiment of the present application, see Fig. 9 The embodiment of the present application provides a training, testing and verification process of a building element extraction and classification model, which continuously optimizes the target loss function and learning rate in the initial extraction and classification model based on the training and testing process.

[0140] Fig.10 A curve diagram of a training classification loss function provided in an embodiment of the present application, Fig.11 A curve diagram of a verification classification loss function provided in an embodiment of the present application, see Fig.10 and Fig.11 It can be seen that in the initial extraction classification model, the curve of the classification loss function gradually tends to be flat. Based on the verification results of the classification loss function, the loss function in the initial extraction classification model is optimized.

[0141] Fig.12 A curve diagram of a training position loss function provided in an embodiment of the present application, Fig.13 A graph of a verification position loss function provided in an embodiment of the present application is shown in FIG. Fig.12 and Fig.13 It can be seen that in the initial extraction and classification model, the curve of the position loss function gradually tends to be flat. Based on the verification results of the position loss function, the loss function in the initial extraction and classification model is optimized.

[0142] Fig.14 A curve diagram of a training confidence loss function provided in an embodiment of the present application, Fig.15 A graph of a verification confidence loss function provided in an embodiment of the present application is shown in FIG. Fig.14 and Fig.15It can be seen that in the initial extraction classification model, the curve of the confidence loss function gradually tends to be flat. Based on the verification results of the confidence loss function, the loss function in the initial extraction classification model is optimized.

[0143] The following describes the devices, equipment, and computer-readable storage media used to execute the building element information processing method provided in this application. The specific implementation process and technical effects are described above and will not be repeated below.

[0144] Fig.16 This is a schematic diagram of the structure of a building element information processing device provided in an embodiment of the present application, see Fig.16 , the device comprises:

[0145] The first acquisition module 1601 is used to acquire historical image data of city wall ruins and train a building element extraction and classification model based on the historical image data;

[0146] The second acquisition module 1602 is used to acquire image data of the target site, the image data including: planar image data and remote sensing image data;

[0147] The data processing module 1603 is used to perform data preprocessing on the image data to obtain a target image;

[0148] The extraction and classification module 1604 is used to input the target image into the building element extraction and classification model, and the building element extraction and classification model extracts the feature information of the target image, and outputs the target building elements and target building element classification information contained in the target site based on the feature information, wherein the building element extraction and classification model includes at least a visual neural backbone network and a feature enhancement network, and the feature information includes: color features, texture features, shape features and layout features.

[0149] In a possible implementation, the first acquisition module 1601 is specifically configured to:

[0150] Divide the historical image data into training data set, test data set and verification data set;

[0151] According to the training data set, the initial extraction and classification model is obtained through iterative training;

[0152] Testing the initial extraction classification model based on the test data set, and determining whether the initial extraction classification model is qualified based on the test result;

[0153] The initial extraction classification model is verified based on the verification data set, and whether the initial extraction classification model is overfitted is determined based on the verification result.

[0154] In a possible implementation manner, the first acquisition module 1601 is further configured to:

[0155] Inputting the test data set into the initial extraction classification model based on the preset batch capacity value, the initial extraction classification model outputs the corresponding first building element and the first building element classification information;

[0156] Whether the initial extraction classification model is qualified is determined according to the matching degree between the first building element and the test building element corresponding to the test data set and the matching degree between the first building element classification information and the test building element classification information corresponding to the test data set.

[0157] In a possible implementation manner, the first acquisition module 1601 is further configured to:

[0158] If the first building element is the same as the test building element corresponding to the test data set, and the first building element classification information is the same as the test building element classification information corresponding to the test data set, then the initial extraction classification model is qualified;

[0159] If the first building element is different from the test building element corresponding to the test data set, and / or the first building element classification information is different from the test building element classification information corresponding to the test data set, then the initial extraction classification model is unqualified.

[0160] In a possible implementation manner, the first acquisition module 1601 is further configured to:

[0161] Based on a preset batch capacity value, the verification data set is input into an initial extraction classification model, and the initial extraction classification model outputs the corresponding second building element and second building element classification information;

[0162] Whether the initial extraction classification model is overfitted is determined according to the matching degree between the second building element and the verification building element corresponding to the verification data set and the matching degree between the second building element classification information and the verification building element classification information corresponding to the verification data set.

[0163] In a possible implementation manner, the first acquisition module 1601 is further configured to:

[0164] If the second building element is the same as the verification building element corresponding to the verification data set, and the second building element classification information is the same as the verification building element classification information corresponding to the verification data set, then the initial extraction classification model is not overfitting;

[0165] If the second building element is different from the verification building element corresponding to the verification data set, and / or the second building element classification information is different from the verification building element classification information corresponding to the verification data set, the initial extraction classification model is overfitted.

[0166] In a possible implementation manner, the first acquisition module 1601 is further configured to:

[0167] Extract feature information corresponding to the training data set, the feature information includes: color features, texture features, shape features and layout features;

[0168] According to the pre-constructed loss function and feature information, the loss of feature information is balanced to obtain a target loss function, wherein the target loss function includes: a position loss function, a classification loss function and a confidence loss function;

[0169] Based on the training data set and the target loss function, an initial extraction classification model is obtained through iterative training, wherein the number of training data sets used in each iteration is determined according to a preset batch capacity.

[0170] In a possible implementation, the first acquisition module 1601 may be further used to:

[0171] According to the preset batch capacity, multiple target training data corresponding to the current iteration round are determined from the training data set;

[0172] Based on the target loss function and multiple target training data, determine the loss result of the to-be-trained model in the current iteration round;

[0173] According to the loss result of the model to be trained in the current iteration round, the model to be trained in the current iteration round is corrected.

[0174] The above-mentioned device is used to execute the method provided by the aforementioned embodiment, and its implementation principle and technical effect are similar, which will not be repeated here.

[0175] The above modules may be one or more integrated circuits configured to implement the above methods, such as one or more application specific integrated circuits (ASICs), or one or more microprocessors, or one or more field programmable gate arrays (FPGAs). For another example, when a module is implemented in the form of a processing element scheduling program code, the processing element may be a general-purpose processor, such as a central processing unit (CPU) or other processor that can call program code. For another example, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0176] Fig.17 is a schematic diagram of the structure of a computer device provided in an embodiment of the present application. Fig.17The computer device includes: a memory 1701 and a processor 1702. The memory 1701 stores a computer program that can be run on the processor 1702. When the processor 1702 executes the computer program, the steps in any of the above method embodiments are implemented.

[0177] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0178] Optionally, the present application also provides a program product, such as a computer-readable storage medium, comprising a program, which, when executed by a processor, is used to execute any of the above-mentioned building element information processing method embodiments.

[0179] In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0180] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0181] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0182] The above-mentioned integrated unit implemented in the form of a software functional unit can be stored in a computer-readable storage medium. The above-mentioned software functional unit is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor (English: processor) to perform some steps of the methods of various embodiments of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (English: Read-Only Memory, abbreviated: ROM), random access memory (English: Random Access Memory, abbreviated: RAM), disk or optical disk and other media that can store program codes.

[0183] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0184] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for processing building element information, characterized in that: The method comprises: Acquire historical image data of city wall ruins, and train a building element extraction and classification model based on the historical image data; Acquire image data of the target site, the image data including: planar image data and remote sensing image data; Performing data preprocessing on the image data to obtain a target image; The target image is input into the architectural element extraction and classification model, and the architectural element extraction and classification model extracts feature information of the target image, and outputs target architectural elements and target architectural element classification information contained in the target site based on the feature information, wherein the architectural element extraction and classification model includes at least a visual neural backbone network and a feature enhancement network, and the feature information includes: color features, texture features, shape features and layout features.

2. The building element information processing method according to claim 1, characterized in that: The building element extraction and classification model obtained by training the historical image data includes: Performing data division on the historical image data to obtain a training data set, a test data set, and a verification data set; According to the training data set, iterative training is performed to obtain an initial extraction classification model; Testing the initial extraction classification model based on the test data set, and determining whether the initial extraction classification model is qualified based on the test result; The initial extraction classification model is verified based on the verification data set, and according to the verification result, it is determined whether the initial extraction classification model is overfitted.

3. The building element information processing method according to claim 2, characterized in that: The step of determining whether the initial extraction classification model is qualified based on the test result includes: Inputting the test data set into the initial extraction classification model based on a preset batch capacity value, the initial extraction classification model outputting the corresponding first building element and first building element classification information; Whether the initial extraction classification model is qualified is determined according to the matching degree between the first building element and the test building element corresponding to the test data set and the matching degree between the first building element classification information and the test building element classification information corresponding to the test data set.

4. The method for processing building element information according to claim 3, characterized in that: The determining whether the initial extraction classification model is qualified according to the matching degree between the first building element and the test building element corresponding to the test data set and the matching degree between the first building element classification information and the test building element classification information corresponding to the test data set includes: If the first building element is the same as the test building element corresponding to the test data set, and the first building element classification information is the same as the test building element classification information corresponding to the test data set, then the initial extraction classification model is qualified; If the first building element is different from the test building element corresponding to the test data set, and / or the first building element classification information is different from the test building element classification information corresponding to the test data set, then the initial extraction classification model is unqualified.

5. The method for processing building element information according to claim 4, characterized in that: The verifying the initial extraction classification model based on the verification data set, and determining whether the initial extraction classification model is overfitting according to the verification result, includes: Inputting the verification data set into the initial extraction classification model based on a preset batch capacity value, the initial extraction classification model outputting the corresponding second building element and second building element classification information; Whether the initial extraction classification model is overfitted is determined according to the matching degree between the second building element and the verification building element corresponding to the verification data set and the matching degree between the second building element classification information and the verification building element classification information corresponding to the verification data set.

6. The method for processing building element information according to claim 5, characterized in that: The determining whether the initial extraction classification model is overfitting according to the matching degree between the second building element and the verification building element corresponding to the verification data set and the matching degree between the second building element classification information and the verification building element classification information corresponding to the verification data set includes: If the second building element is the same as the verification building element corresponding to the verification data set, and the second building element classification information is the same as the verification building element classification information corresponding to the verification data set, then the initial extraction classification model is not overfitting; If the second building element is different from the verification building element corresponding to the verification data set, and / or the second building element classification information is different from the verification building element classification information corresponding to the verification data set, the initial extraction classification model is overfitted.

7. The building element information processing method according to claim 2, characterized in that: The iterative training according to the training data set to obtain an initial extraction classification model includes: Extracting feature information corresponding to the training data set, the feature information including: color feature, texture feature, shape feature and layout feature; According to the pre-constructed loss function and the feature information, the loss of the feature information is balanced to obtain a target loss function, wherein the target loss function includes: a position loss function, a classification loss function and a confidence loss function; Based on the training data set and the target loss function, the initial extraction classification model is obtained through iterative training, wherein the number of training data sets used in each iteration is determined according to a preset batch capacity.

8. The method for processing building element information according to claim 7, characterized in that: The iterative training based on the training data set and the target loss function to obtain the initial extraction classification model includes: According to the preset batch capacity, determining a plurality of target training data corresponding to the current iteration round from the training data set; Based on the target loss function and the multiple target training data, determine the loss result of the to-be-trained model in the current iteration round; According to the loss result of the model to be trained in the current iteration round, the model to be trained in the current iteration round is corrected.

9. A building element information processing device, characterized in that: The device comprises: The first acquisition module is used to acquire historical image data of city wall sites and train a building element extraction and classification model based on the historical image data; A second acquisition module is used to acquire image data of the target site, wherein the image data includes: planar image data and remote sensing image data; A data processing module, used for performing data preprocessing on the image data to obtain a target image; The extraction and classification module is used to input the target image into the building element extraction and classification model, extract the feature information of the target image by the building element extraction and classification model, and output the target building elements and target building element classification information contained in the target site based on the feature information, wherein the building element extraction and classification model includes at least a visual neural backbone network and a feature enhancement network, and the feature information includes: color features, texture features, shape features and layout features.

10. A computer device, characterized in that: include: A memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the computer program, the steps of the method described in any one of claims 1 to 7 are implemented.

Images