A method, device, medium and equipment for monitoring data quality

Abstract

The present invention is a method, device, medium and equipment for monitoring data quality, which relates to the field of data processing technology, including: obtaining first building data and second building data, obtaining a number of key building elements based on graphic data in the first building data, traversing all key building elements, obtaining the spatial relationship between the several key building elements, monitoring the graphic data in the first building data based on the spatial relationship to determine the first abnormal data, and obtaining the second abnormal data based on the text data in the first building data and the second building data. The present invention utilizes the three-dimensional visualization characteristics of the BIM model and combines sensor technology to achieve real-time and intelligent monitoring of BIM model data, reduce labor costs, improve the efficiency of data monitoring, and adopt different processing methods based on different data forms, so that the accuracy of the acquired abnormal data is higher, thereby improving the accuracy of the monitoring data quality.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a method, device, medium and equipment for monitoring data quality. Background Art

[0002] With the informatization and intelligent development of the construction industry, the application of BIM models in the stages of building design, construction, operation and maintenance is becoming more and more extensive. Among them, the BIM model is an integrated digital technology solution that integrates all relevant information of a construction project into a three-dimensional model. However, due to the diversity of data sources, the complexity of data processing and the uncertainty of human operation, the data quality problem in the BIM model has become increasingly prominent, seriously affecting the application effect of BIM technology. Therefore, the development of an intelligent data quality monitoring system based on the BIM model is of great significance to improving the data quality of the BIM model and ensuring the smooth progress of BIM application. Summary of the invention

[0003] In view of the above technical problems, the technical solution adopted by the present invention is a method for monitoring data quality, which comprises the following steps:

[0004] Acquire first building data and second building data, wherein the first building data is building data acquired through the API interface of the BIM model system, and the second building data is building data corresponding to the target geographical location acquired through sensor transmission.

[0005] Based on the graphic data in the first building data, a plurality of key building elements are obtained, wherein the key building elements are building elements obtained by processing the graphic data in the first building data using a Canny edge detection algorithm and a gray level co-occurrence matrix.

[0006] All the key building elements among the key building elements are traversed to obtain the spatial relationship between the key building elements, wherein the spatial relationship is obtained by embedding a logic judgment statement generated based on a spatial relationship rule in a script, and identifying the spatial relationship between the key building elements according to the position relationship and size corresponding to the key building elements.

[0007] Based on the spatial relationship, the graphic data in the first building data is monitored to determine the first abnormal data, wherein when the spatial relationship corresponding to a certain graphic data in the first building data is missing or the spatial relationship matching is incorrect or the number of spatial relationships exceeds a preset threshold, the spatial relationship data corresponding to this graphic data is determined to be the first abnormal data.

[0008] Based on the text data in the first building data and the second building data, second abnormal data is acquired.

[0009] The present invention also provides a device for monitoring data quality, the device comprising:

[0010] The first execution module is used to obtain first building data and second building data, wherein the first building data is building data obtained through the API interface of the BIM model system, and the second building data is building data corresponding to the target geographical location obtained through sensor transmission.

[0011] The second execution module is used to obtain a number of key building elements based on the graphic data in the first building data, wherein the key building elements are building elements obtained by processing the graphic data in the first building data using a Canny edge detection algorithm and a gray level co-occurrence matrix.

[0012] The third execution module is used to traverse all the key building elements among the key building elements to obtain the spatial relationship between the key building elements, wherein the spatial relationship is obtained by embedding a logical judgment statement generated based on the spatial relationship rule in the script, and identifying the spatial relationship between the key building elements according to the corresponding position relationship and size of the key building elements.

[0013] The fourth execution module is used to monitor the graphic data in the first building data based on the spatial relationship to determine the first abnormal data, wherein when the spatial relationship corresponding to a certain graphic data in the first building data is missing or the spatial relationship matching is incorrect or the number of spatial relationships exceeds a preset threshold, the spatial relationship data corresponding to this graphic data is determined to be the first abnormal data.

[0014] The fifth execution module is used to obtain second abnormal data based on the text data in the first building data and the second building data.

[0015] The present invention also provides a non-transitory computer-readable storage medium, in which at least one instruction or at least one program is stored, and the at least one instruction or at least one program is loaded and executed by a processor to implement the above-mentioned method for monitoring data quality.

[0016] The present invention also provides an electronic device, including a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by the processor to implement the above-mentioned method for monitoring data quality.

[0017] The present invention has at least the following beneficial effects: the present invention can obtain the first building data and the second building data, obtain a number of key building elements based on the graphic data in the first building data, traverse all the key building elements in the several key building elements, obtain the spatial relationship between the several key building elements, monitor the graphic data in the first building data based on the spatial relationship to determine the first abnormal data, and obtain the second abnormal data based on the text data in the first building data and the second building data. It can be seen that by utilizing the three-dimensional visualization characteristics of the BIM model and combining with sensor technology, real-time and intelligent monitoring of the BIM model data is realized. Through intelligent data monitoring, the workload of manual inspection is reduced, the labor cost is reduced, and the efficiency of data monitoring is improved. Different processing methods are adopted based on different data forms, so that the accuracy of the acquired abnormal data is higher, thereby improving the accuracy of the monitoring data quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0019] Figure 1 A flow chart of a method for monitoring data quality provided in Embodiment 1 of the present invention;

[0020] Figure 2 This is a flow chart of S200 provided in the first embodiment of the present invention;

[0021] Figure 3 This is a flow chart of S500 provided in the first embodiment of the present invention;

[0022] Figure 4 A schematic diagram of the structure of a device for monitoring data quality provided in Embodiment 2 of the present invention;

[0023] Figure 5 A schematic diagram of the structure of a second execution module 2 provided in Embodiment 2 of the present invention;

[0024] Figure 6 This is a schematic diagram of the structure of the fifth execution module 5 provided in the second embodiment of the present invention. DETAILED DESCRIPTION

[0025] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0026] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or server that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0027] Embodiment 1,

[0028] This embodiment provides a method for monitoring data quality, the method comprising the following steps: Figure 1 As shown:

[0029] S100, obtaining first building data and second building data, wherein the first building data is building data obtained through the API interface transmission of the BIM model system, and the second building data is building data corresponding to the target geographical location obtained through sensor transmission.

[0030] Specifically, the first building data includes building geometry data, building attribute data and building association data, wherein the building association data includes logical relationship data and spatial relationship data between building components, for example: building geometry data such as building size, location and shape, and building attribute data such as building material type, cost and manufacturer information.

[0031] Furthermore, the target geographical location is a geographical location corresponding to a construction site.

[0032] Furthermore, the second construction data includes construction progress data, building material usage data and target geographic location equipment data, including, for example: construction progress data such as daily engineering volume, building material usage data such as building material consumption rate and inventory quantity, and target geographic location equipment data such as on / off time of equipment used in the target geographic location and fault information.

[0033] Specifically, the first building data and the second building data are data that have been subjected to real-time stream processing and data compression.

[0034] Specifically, the method for monitoring data quality in this embodiment processes the first building data and the second building data under a real-time stream processing framework, for example, a real-time stream processing framework such as Apache Kafka or Apache Flink.

[0035] S200, based on the graphic data in the first building data, obtaining a plurality of key building elements, wherein the key building elements are building elements obtained by processing the graphic data in the first building data using a Canny edge detection algorithm and a gray level co-occurrence matrix.

[0036] Specifically, the graphic data is data that displays building structures and building elements in a graphical manner.

[0037] Specifically, the building elements are basic units and components constituting a building, such as walls, doors, windows and other building elements.

[0038] Specifically, S200 also includes the following steps: Figure 2 As shown:

[0039] S201, obtaining a sample image list set, wherein the sample image list set includes a plurality of sample image lists, the sample image lists include a plurality of sample images, the sample images are acquired images including a plurality of specific architectural elements, the specific architectural elements corresponding to the sample images in each sample image list are of the same type, and the specific architectural elements corresponding to the sample images in different sample image lists are of different types.

[0040] S202, obtaining shape features and texture features corresponding to each sample image in the sample image list set, wherein the shape features corresponding to the sample image are the shape features obtained by performing a Canny edge detection algorithm on the sample image, and the texture features corresponding to the sample image are the texture features obtained by performing a grayscale co-occurrence matrix calculation on the sample image after performing a Canny edge detection algorithm.

[0041] S203, obtaining shape features and texture features corresponding to the candidate image to be matched, wherein the candidate image to be matched is any image obtained from the graphic data in the first building data, the shape features corresponding to the candidate image to be matched are the shape features obtained by performing a Canny edge detection algorithm on the candidate image to be matched, and the texture features corresponding to the candidate image to be matched are the texture features obtained by performing a grayscale co-occurrence matrix calculation on the candidate image to be matched after performing a Canny edge detection algorithm.

[0042] S204, obtaining a first similarity between the candidate image to be matched and each sample image, wherein the first similarity is a similarity obtained by using a normalized cross correlation (NCC) algorithm based on a shape feature corresponding to the candidate image to be matched and a shape feature corresponding to each sample image.

[0043] S205, obtaining a second similarity between the candidate image to be matched and each sample image, wherein the second similarity is the similarity between the texture features corresponding to the candidate image to be matched and the texture features corresponding to each sample image obtained by using a cosine similarity algorithm after normalizing the texture features corresponding to the candidate image to be matched and the texture features corresponding to each sample image respectively.

[0044] S206: When the sum of the first similarity and the second similarity between the candidate image to be matched and a sample image is greater than zero and is a maximum value, a specific architectural element in the sample image is obtained as a key architectural element corresponding to the candidate image to be matched.

[0045] S300, traversing all the key building elements among the plurality of key building elements, and obtaining the spatial relationship between the plurality of key building elements, wherein the spatial relationship is obtained by embedding a logic judgment statement generated based on a spatial relationship rule in a script, and identifying the spatial relationship between the key building elements according to the position relationship and size corresponding to the key building elements.

[0046] Specifically, those skilled in the art know that the spatial relationship rules can be set according to actual needs, all of which fall within the protection scope of the present invention and will not be described in detail here.

[0047] Specifically, the spatial relationship includes but is not limited to an adjacent relationship, a containing relationship, an intersecting relationship and a directional relationship.

[0048] S400, monitoring the graphic data in the first building data based on the spatial relationship to determine the first abnormal data, wherein when the spatial relationship corresponding to a certain graphic data in the first building data is missing or the spatial relationship matching is incorrect or the number of spatial relationships exceeds a preset threshold, the spatial relationship data corresponding to this graphic data is determined to be the first abnormal data.

[0049] Specifically, those skilled in the art know that the setting of the quantity threshold value can be preset according to the actual demand model, and all of them fall within the protection scope of the present invention.

[0050] S500: Acquire second abnormal data based on text data in the first building data and the second building data.

[0051] Specifically, S500 also includes the following steps: Figure 3 As shown:

[0052] S501, performing data preprocessing on the second building data, wherein the preprocessing includes data cleaning, data deduplication and data standardization.

[0053] Specifically, the data cleaning is to use the data processing tool Pandas to identify and process missing values, error values ​​and outliers, the data deduplication is to use a hash function to calculate the hash value of each record in the second building data, and quickly identify duplicates by comparing the hash values, and the data standardization is to convert the data according to a unified format or standard.

[0054] S502, obtaining second abnormal data, wherein the second abnormal data is abnormal data obtained by using a K-means clustering algorithm and a DBSCAN algorithm to preprocess the second building data and the text data in the first building data.

[0055] S503, marking an abnormality category for the second abnormal data, wherein the abnormality category includes: missing, format error, and logical error.

[0056] From the above content, we can see that by utilizing the three-dimensional visualization characteristics of the BIM model and combining it with sensor technology, real-time and intelligent monitoring of BIM model data can be achieved. Intelligent data monitoring reduces the workload of manual inspection, reduces labor costs, and improves the efficiency of data monitoring. Different processing methods are used based on different data forms, so that the accuracy of the acquired abnormal data is higher, thereby improving the accuracy of data quality monitoring.

[0057] Specifically, after step S500, the following steps are also included:

[0058] S1, obtaining target abnormal data, wherein the target abnormal data includes first abnormal data and second abnormal data.

[0059] S2, when the target abnormal data is simple abnormal data, the target abnormal data is corrected by a first processing method, wherein the simple abnormal data is data with format problems and data with missing value problems, wherein the format problems include date format inconsistency or non-numeric characters appearing in numeric fields.

[0060] Specifically, S2 also includes the following steps:

[0061] S21, when the target abnormal data is data with a format problem, calling a format conversion function corresponding to the format problem to perform format conversion on the target abnormal data.

[0062] S22, when the target abnormal data is data with a missing value problem and the target abnormal data is the first abnormal data, the target abnormal data is processed and supplemented. When the target abnormal data is building geometry data, the model's API interface is called to obtain size and location information for data supplement. When the target abnormal data is building attribute data, the data is supplemented by searching for the attribute supplement of adjacent elements of the same category of the element.

[0063] S23, when the target abnormal data is data with missing values ​​and the target abnormal data is the second abnormal data, the target abnormal data is processed and supplemented, wherein, when the target abnormal data is trendy and non-periodic data, it is supplemented by using a near-data linear interpolation estimation method, when the target abnormal data is trendless and non-periodic data, it is estimated and supplemented by using a statistical method, and when the target abnormal data is trendy and cyclical data, it is supplemented by using seasonal adjustment.

[0064] Specifically, the trend-free data are data that show a clear long-term growth or decline trend over time, but do not follow a periodic fluctuation pattern, for example, trend-free data such as total construction progress and inventory.

[0065] Specifically, the trendless and non-periodic data are data that neither show a clear long-term growth or decline trend over time nor follow a periodic fluctuation pattern, for example: trendless and non-periodic data such as material consumption rate and daily workload.

[0066] Specifically, the trended and cyclical data are data that present a clear long-term growth or decline trend over time and follow a periodic fluctuation pattern, for example, trended and cyclical data such as equipment operating status.

[0067] Specifically, the corrected data will be quality-monitored again based on the method included in step S500 to verify the reliability of the corrected data. If the corrected data still has logical anomalies, it will be corrected again. If the corrected data is corrected more than three times, manual correction will be performed directly.

[0068] S3, when the target abnormal data is complex abnormal data, the target abnormal data is corrected by the second processing method, wherein the complex abnormal data is data with problems other than the problems corresponding to the simple abnormal data, and the method of correcting the target abnormal data by the second processing method is to recommend a correction operation method to the user, and then the user reviews the recommended correction operation method and selects the final recommended correction operation method to correct the target abnormal data.

[0069] Specifically, the complex abnormal data can be understood as relatively complex data that is not a simple format problem or data missing problem, for example: logical contradictions between data (such as mismatch between room area and door and window sizes), large-scale data anomalies, data spatial relationship errors, and other complex abnormal data.

[0070] Specifically, S3 also includes the following steps:

[0071] S31, when the target abnormal data is building geometry data, obtaining a target priority, wherein the target priority is a difference between the target abnormal data and the real data.

[0072] S32, when the target priority is greater than a preset priority threshold, the real data corresponding to the building model is obtained through the API interface to correct the target abnormal data.

[0073] Specifically, those skilled in the art know that the preset priority threshold can be selected according to actual needs, all of which fall within the protection scope of the present invention and will not be described in detail here.

[0074] S33: When the target priority is not greater than a preset priority threshold, a geometric information modification model is recommended to the user to correct the abnormal data.

[0075] S34, when the target abnormal data is building attribute data, it is recommended to the user to replace the target abnormal data with the attribute data that appears most frequently in adjacent building elements of the same type.

[0076] S35, when the target abnormal data is building-related data, recommending the first law of geography to the user to correct the target abnormal data.

[0077] Specifically, for the case of missing spatial relationships, the nearest target type building element is searched based on the building element, and recommendations are given. For example, if the wall element lacks an adjacency relationship with other walls, other elements of the wall category are searched nearby, and correction recommendations are given: W08 (elementType: wall) may be adjacent to W09 (elementType: wall) and W10 (elementType: wall), and two adjacency relationships are missing; for the case of incorrect spatial relationships, it is recommended to modify the model, and it is prompted that the element should not have such spatial relationships; for the case of redundant spatial relationships, the farthest few related elements of this type are searched based on the element, and it is recommended to modify the model to delete the redundant spatial relationships.

[0078] S36, when the target abnormal data is time series data, the target abnormal data is corrected using a method consistent with the method of correcting the abnormal data in step S23.

[0079] Specifically, the method for data correction in this embodiment sets an RLHF learning feedback algorithm, and intelligently adjusts the parameter settings and algorithm selection of the correction strategy based on the correction results and user feedback. During the correction process, possible algorithms are spontaneously matched for correction, and the calculation results of multiple strategies are fed back by users. The system collects this feedback training reward model to strengthen the learning strategy, fine-tune the correction strategy, and improve the accuracy and efficiency of the correction.

[0080] From the above content, we can see that different correction methods are used for different types of abnormal data. Simple errors can be corrected intelligently. For complex problems, correction suggestions are provided and confirmed or modified by the user. This real-time correction mechanism can ensure the accuracy and consistency of BIM model data, reduce resource waste, improve the efficiency and quality of construction, and ensure the quality of the building.

[0081] The first embodiment of the present invention can obtain the first building data and the second building data, obtain a number of key building elements based on the graphic data in the first building data, traverse all the key building elements in the said number of key building elements, obtain the spatial relationship between the several key building elements, monitor the graphic data in the first building data based on the spatial relationship to determine the first abnormal data, and obtain the second abnormal data based on the text data in the first building data and the second building data. It can be seen that by utilizing the three-dimensional visualization characteristics of the BIM model and combining with sensor technology, real-time and intelligent monitoring of the BIM model data is realized. Through intelligent data monitoring, the workload of manual inspection is reduced, the labor cost is reduced, and the efficiency of data monitoring is improved. Different processing methods are adopted based on different data forms, so that the accuracy of the acquired abnormal data is higher, thereby improving the accuracy of the monitoring data quality.

[0082] Embodiment 2:

[0083] like Figure 4 As shown, this embodiment provides a device for monitoring data quality, the device comprising:

[0084] The first execution module 1 is used to obtain first building data and second building data, wherein the first building data is building data obtained through the API interface of the BIM model system, and the second building data is building data corresponding to the target geographical location obtained through sensor transmission.

[0085] Specifically, the first building data includes building geometry data, building attribute data and building association data, wherein the building association data includes logical relationship data and spatial relationship data between building components, for example: building geometry data such as building size, location and shape, and building attribute data such as building material type, cost and manufacturer information.

[0086] Furthermore, the target geographical location is a geographical location corresponding to a construction site.

[0087] Furthermore, the second construction data includes construction progress data, building material usage data and target geographic location equipment data, including, for example: construction progress data such as daily engineering volume, building material usage data such as building material consumption rate and inventory quantity, and target geographic location equipment data such as on / off time of equipment used in the target geographic location and fault information.

[0088] Specifically, the first building data and the second building data are data that have been subjected to real-time stream processing and data compression.

[0089] Specifically, the method for monitoring data quality in this embodiment processes the first building data and the second building data under a real-time stream processing framework, for example, a real-time stream processing framework such as Apache Kafka or Apache Flink.

[0090] The second execution module 2 is used to obtain a number of key building elements based on the graphic data in the first building data, wherein the key building elements are building elements obtained by processing the graphic data in the first building data using a Canny edge detection algorithm and a gray level co-occurrence matrix.

[0091] Specifically, the graphic data is data that displays building structures and building elements in a graphical manner.

[0092] Specifically, the building elements are basic units and components constituting a building, such as walls, doors, windows and other building elements.

[0093] Specifically, Figure 5 As shown, the second execution module 2 also includes:

[0094] The first acquisition module 21 is used to acquire a sample image list set, wherein the sample image list set includes a plurality of sample image lists, the sample image list includes a plurality of sample images, the sample images are acquired images including a plurality of specific architectural elements, the specific architectural elements corresponding to the sample images in each sample image list are of the same type, and the specific architectural elements corresponding to the sample images in different sample image lists are of different types.

[0095] The second acquisition module 22 is used to obtain the shape features and texture features corresponding to each sample image in the sample image list set, wherein the shape features corresponding to the sample image are the shape features obtained by performing the Canny edge detection algorithm on the sample image, and the texture features corresponding to the sample image are the texture features obtained by performing grayscale co-occurrence matrix calculation on the sample image after performing the Canny edge detection algorithm.

[0096] The third acquisition module 23 is used to obtain shape features and texture features corresponding to the candidate image to be matched, wherein the candidate image to be matched is any image obtained from the graphic data in the first building data, the shape features corresponding to the candidate image to be matched are the shape features obtained by performing a Canny edge detection algorithm on the candidate image to be matched, and the texture features corresponding to the candidate image to be matched are the texture features obtained by performing a grayscale co-occurrence matrix calculation on the candidate image to be matched after performing a Canny edge detection algorithm.

[0097] The fourth acquisition module 24 is used to obtain a first similarity between the candidate image to be matched and each sample image, wherein the first similarity is a similarity obtained by using a normalized cross correlation (NCC) algorithm based on the shape features corresponding to the candidate image to be matched and the shape features corresponding to each sample image.

[0098] The fifth acquisition module 25 is used to obtain the second similarity between the candidate image to be matched and each sample image, wherein the second similarity is the similarity between the texture features corresponding to the candidate image to be matched and the texture features corresponding to each sample image obtained by using a cosine similarity algorithm after normalizing the texture features corresponding to the candidate image to be matched and the texture features corresponding to each sample image respectively.

[0099] The sixth acquisition module 26 is used to acquire a specific architectural element in a sample image as a key architectural element corresponding to the candidate image to be matched when the sum of the first similarity and the second similarity between the candidate image to be matched and a sample image is greater than zero and is a maximum value.

[0100] The third execution module 3 is used to traverse all the key building elements among the plurality of key building elements, and obtain the spatial relationship between the plurality of key building elements, wherein the spatial relationship is obtained by embedding a logical judgment statement generated based on a spatial relationship rule in the script, and identifying the spatial relationship between the key building elements according to the position relationship and size corresponding to the key building elements.

[0101] Specifically, those skilled in the art know that the spatial relationship rules can be set according to actual needs, all of which fall within the protection scope of the present invention and will not be described in detail here.

[0102] Specifically, the spatial relationship includes but is not limited to an adjacent relationship, a containing relationship, an intersecting relationship and a directional relationship.

[0103] The fourth execution module 4 is used to monitor the graphic data in the first building data based on the spatial relationship to determine the first abnormal data, wherein when the spatial relationship corresponding to a certain graphic data in the first building data is missing or the spatial relationship matching is incorrect or the number of spatial relationships exceeds a preset quantity threshold, the spatial relationship data corresponding to this graphic data is determined to be the first abnormal data.

[0104] Specifically, those skilled in the art know that the setting of the quantity threshold value can be preset according to the actual demand model, and all of them fall within the protection scope of the present invention.

[0105] The fifth execution module 5 is used to obtain second abnormal data based on the text data in the first building data and the second building data.

[0106] Specifically, Figure 6 As shown, the fifth execution module 5 also includes:

[0107] The data preprocessing module 51 is used to perform data preprocessing on the second building data, wherein the preprocessing includes data cleaning, data deduplication and data standardization.

[0108] Specifically, the data cleaning is to use the data processing tool Pandas to identify and process missing values, error values ​​and outliers, the data deduplication is to use a hash function to calculate the hash value of each record in the second building data, and quickly identify duplicates by comparing the hash values, and the data standardization is to convert the data according to a unified format or standard.

[0109] The second abnormal data acquisition module 52 is used to acquire second abnormal data, wherein the second abnormal data is abnormal data acquired by using the K-means clustering algorithm and the DBSCAN algorithm to preprocess the second building data and the text data in the first building data.

[0110] The abnormal data marking module 53 is used to mark the abnormal category of the second abnormal data, wherein the abnormal category includes: missing, format error, and logical error.

[0111] From the above content, we can see that by utilizing the three-dimensional visualization characteristics of the BIM model and combining it with sensor technology, real-time and intelligent monitoring of BIM model data can be achieved. Intelligent data monitoring reduces the workload of manual inspection, reduces labor costs, and improves the efficiency of data monitoring. Different processing methods are used based on different data forms, so that the accuracy of the acquired abnormal data is higher, thereby improving the accuracy of data quality monitoring.

[0112] Specifically, the abnormal data marking module 53 also includes:

[0113] The first intermediate module is used to obtain target abnormal data, wherein the target abnormal data includes first abnormal data and second abnormal data.

[0114] The second intermediate module is used to correct the target abnormal data by adopting the first processing method when the target abnormal data is simple abnormal data, wherein the simple abnormal data is data with format problems and data with missing value problems, wherein the format problems include date format inconsistency or non-numeric characters appearing in numeric fields.

[0115] Specifically, the second intermediate module further includes:

[0116] The format conversion module is used to call the format conversion function corresponding to the format problem to convert the format of the target abnormal data when the target abnormal data is data with a format problem.

[0117] The first processing module is used to process and supplement the target abnormal data when the target abnormal data is data with missing values ​​and the target abnormal data is the first abnormal data. When the target abnormal data is building geometry data, the model's API interface is called to obtain size and position information for data supplement. When the target abnormal data is building attribute data, the data is supplemented by searching for the attribute supplement of adjacent elements of the same category of the element.

[0118] The second processing module is used to process and supplement the target abnormal data when the target abnormal data is data with missing values ​​and the target abnormal data is the second abnormal data. When the target abnormal data is trendy and non-periodic data, it is supplemented by using a near-data linear interpolation estimation method; when the target abnormal data is trendless and non-periodic data, it is estimated and supplemented by using a statistical method; and when the target abnormal data is trendy and cyclical data, it is supplemented by using seasonal adjustment.

[0119] Specifically, the trend-free data are data that show a clear long-term growth or decline trend over time, but do not follow a periodic fluctuation pattern, for example, trend-free data such as total construction progress and inventory.

[0120] Specifically, the trendless and non-periodic data are data that neither show a clear long-term growth or decline trend over time nor follow a periodic fluctuation pattern, for example: trendless and non-periodic data such as material consumption rate and daily workload.

[0121] Specifically, the trended and cyclical data are data that present a clear long-term growth or decline trend over time and follow a periodic fluctuation pattern, for example, trended and cyclical data such as equipment operating status.

[0122] Specifically, the corrected data will be quality-monitored again based on the method included in step S500 to verify the reliability of the corrected data. If the corrected data still has logical anomalies, it will be corrected again. If the corrected data is corrected more than three times, manual correction will be performed directly.

[0123] The third intermediate module is used to correct the target abnormal data by adopting the second processing method when the target abnormal data is complex abnormal data, wherein the complex abnormal data is data with problems other than the problems corresponding to the simple abnormal data, and the method of correcting the target abnormal data by adopting the second processing method is to correct the target abnormal data by recommending a correction operation method to the user, and then the user reviews the recommended correction operation method and selects a final recommended correction operation method.

[0124] Specifically, the complex abnormal data can be understood as relatively complex data that is not a simple format problem or data missing problem, for example: logical contradictions between data (such as mismatch between room area and door and window sizes), large-scale data anomalies, data spatial relationship errors, and other complex abnormal data.

[0125] Specifically, the third intermediate module further includes:

[0126] The target priority acquisition module is used to acquire the target priority when the target abnormal data is building geometric data, wherein the target priority is the difference between the target abnormal data and the real data.

[0127] The first correction module is used to obtain the real data corresponding to the building model through the API interface to correct the target abnormal data when the target priority is greater than a preset priority threshold.

[0128] Specifically, those skilled in the art know that the preset priority threshold can be selected according to actual needs, all of which fall within the protection scope of the present invention and will not be described in detail here.

[0129] The second correction module is used to recommend a geometric information modification model to the user to correct the abnormal data when the target priority is not greater than a preset priority threshold.

[0130] The third correction module is used for recommending to the user to replace the target abnormal data with the attribute data that appears most frequently in adjacent building elements of the same type when the target abnormal data is building attribute data.

[0131] The fourth correction module is used to recommend the first law of geography to the user to correct the target abnormal data when the target abnormal data is building-related data.

[0132] Specifically, for the case of missing spatial relationships, the nearest target type building element is searched based on the building element, and recommendations are given. For example, if the wall element lacks an adjacency relationship with other walls, other elements of the wall category are searched nearby, and correction recommendations are given: W08 (elementType: wall) may be adjacent to W09 (elementType: wall) and W10 (elementType: wall), and two adjacency relationships are missing; for the case of incorrect spatial relationships, it is recommended to modify the model, and it is prompted that the element should not have such spatial relationships; for the case of redundant spatial relationships, the farthest few related elements of this type are searched based on the element, and it is recommended to modify the model to delete the redundant spatial relationships.

[0133] The fifth correction module is used to correct the target abnormal data using a method consistent with the method of correcting the abnormal data in step S23 when the target abnormal data is time series data.

[0134] Specifically, the method for data correction in this embodiment sets an RLHF learning feedback algorithm, and intelligently adjusts the parameter settings and algorithm selection of the correction strategy based on the correction results and user feedback. During the correction process, possible algorithms are spontaneously matched for correction, and the calculation results of multiple strategies are fed back by users. The system collects this feedback training reward model to strengthen the learning strategy, fine-tune the correction strategy, and improve the accuracy and efficiency of the correction.

[0135] From the above content, we can see that different correction methods are used for different types of abnormal data. Simple errors can be corrected intelligently. For complex problems, correction suggestions are provided and confirmed or modified by the user. This real-time correction mechanism can ensure the accuracy and consistency of BIM model data, reduce resource waste, improve the efficiency and quality of construction, and ensure the quality of the building.

[0136] Embodiment 3,

[0137] An embodiment of the present invention provides a non-transitory computer-readable storage medium, wherein at least one instruction or at least one program is stored in the non-transitory computer-readable storage medium, and the at least one instruction or at least one program is loaded and executed by a processor to implement the steps:

[0138] Acquire first building data and second building data, wherein the first building data is building data acquired through the API interface of the BIM model system, and the second building data is building data corresponding to the target geographical location acquired through sensor transmission.

[0139] Based on the graphic data in the first building data, a plurality of key building elements are obtained, wherein the key building elements are building elements obtained by processing the graphic data in the first building data using a Canny edge detection algorithm and a gray level co-occurrence matrix.

[0140] All the key building elements among the key building elements are traversed to obtain the spatial relationship between the key building elements, wherein the spatial relationship is obtained by embedding a logic judgment statement generated based on a spatial relationship rule in a script, and identifying the spatial relationship between the key building elements according to the position relationship and size corresponding to the key building elements.

[0141] Based on the spatial relationship, the graphic data in the first building data is monitored to determine the first abnormal data, wherein when the spatial relationship corresponding to a certain graphic data in the first building data is missing or the spatial relationship matching is incorrect or the number of spatial relationships exceeds a preset threshold, the spatial relationship data corresponding to this graphic data is determined to be the first abnormal data.

[0142] Based on the text data in the first building data and the second building data, second abnormal data is acquired.

[0143] A person of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0144] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0145] Embodiment 4:

[0146] An embodiment of the present invention provides an electronic device, including a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by the processor to implement the steps:

[0147] Acquire first building data and second building data, wherein the first building data is building data acquired through the API interface of the BIM model system, and the second building data is building data corresponding to the target geographical location acquired through sensor transmission.

[0148] Based on the graphic data in the first building data, a plurality of key building elements are obtained, wherein the key building elements are building elements obtained by processing the graphic data in the first building data using a Canny edge detection algorithm and a gray level co-occurrence matrix.

[0149] All the key building elements among the key building elements are traversed to obtain the spatial relationship between the key building elements, wherein the spatial relationship is obtained by embedding a logic judgment statement generated based on a spatial relationship rule in a script, and identifying the spatial relationship between the key building elements according to the position relationship and size corresponding to the key building elements.

[0150] Based on the spatial relationship, the graphic data in the first building data is monitored to determine the first abnormal data, wherein when the spatial relationship corresponding to a certain graphic data in the first building data is missing or the spatial relationship matching is incorrect or the number of spatial relationships exceeds a preset threshold, the spatial relationship data corresponding to this graphic data is determined to be the first abnormal data.

[0151] Based on the text data in the first building data and the second building data, second abnormal data is acquired.

[0152] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment as above, it is not used to limit the present invention. Any technician familiar with this profession can make some changes or modifications to equivalent embodiments of equivalent changes using the technical contents disclosed above without departing from the scope of the technical solution of the present invention. However, any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention are still within the scope of the technical solution of the present invention.

Claims

1. A method for monitoring data quality, characterized in that: The method comprises the following steps: Acquire first building data and second building data, wherein the first building data is building data acquired through an API interface of a BIM model system, and the second building data is building data corresponding to a target geographical location acquired through sensor transmission; Based on the graphic data in the first building data, a plurality of key building elements are obtained, wherein the key building elements are building elements obtained by processing the graphic data in the first building data using a Canny edge detection algorithm and a gray level co-occurrence matrix. Based on the graphic data in the first building data, the plurality of key building elements are obtained, including the following steps: Acquire a sample image list set, wherein the sample image list set includes a plurality of sample image lists, the sample image lists include a plurality of sample images, the sample images are acquired images including a plurality of specific architectural elements, the specific architectural elements corresponding to the sample images in each sample image list are of the same type, and the specific architectural elements corresponding to the sample images in different sample image lists are of different types; Obtain shape features and texture features corresponding to each sample image in the sample image list set, wherein the shape features corresponding to the sample image are the shape features obtained by performing a Canny edge detection algorithm on the sample image, and the texture features corresponding to the sample image are the texture features obtained by performing a gray level co-occurrence matrix calculation on the sample image after performing a Canny edge detection algorithm; Obtaining shape features and texture features corresponding to a candidate image to be matched, wherein the candidate image to be matched is any image obtained from the graphic data in the first building data, the shape features corresponding to the candidate image to be matched are shape features obtained by performing a Canny edge detection algorithm on the candidate image to be matched, and the texture features corresponding to the candidate image to be matched are texture features obtained by performing a gray level co-occurrence matrix calculation on the candidate image to be matched after performing a Canny edge detection algorithm; Obtaining a first similarity between the candidate image to be matched and each sample image, wherein the first similarity is a similarity obtained by using a normalized cross correlation (NCC) algorithm based on a shape feature corresponding to the candidate image to be matched and a shape feature corresponding to each sample image; Obtaining a second similarity between the candidate image to be matched and each sample image, wherein the second similarity is the similarity between the texture features corresponding to the candidate image to be matched and the texture features corresponding to each sample image obtained by using a cosine similarity algorithm after normalizing the texture features corresponding to the candidate image to be matched and the texture features corresponding to each sample image; When the sum of the first similarity and the second similarity between the candidate image to be matched and a sample image is greater than zero and is a maximum value, obtaining a specific architectural element in the sample image as a key architectural element corresponding to the candidate image to be matched; Traversing all the key building elements among the plurality of key building elements, and obtaining the spatial relationship between the plurality of key building elements, wherein the spatial relationship is obtained by embedding a logic judgment statement generated based on a spatial relationship rule in the script, and identifying the spatial relationship between the key building elements according to the position relationship and size corresponding to the key building elements; The graphic data in the first building data is monitored based on the spatial relationship to determine the first abnormal data, wherein when the spatial relationship corresponding to a certain graphic data in the first building data is missing or the spatial relationship matching is wrong or the number of spatial relationships exceeds a preset number threshold, the spatial relationship data corresponding to the graphic data is determined to be the first abnormal data; Based on the text data in the first building data and the second building data, second abnormal data is acquired.

2. The method for monitoring data quality according to claim 1, characterized in that: The graphic data is data that displays building structures and building elements in a graphical manner.

3. The method for monitoring data quality according to claim 1, characterized in that: Based on the text data in the first building data and the second building data, obtaining the second abnormal data comprises the following steps: Performing data preprocessing on the second building data, wherein the preprocessing includes data cleaning, data deduplication and data standardization; Acquire second abnormal data, wherein the second abnormal data is abnormal data acquired by using a K-means clustering algorithm and a DBSCAN algorithm to preprocess the second building data and the text data in the first building data; The second abnormal data is labeled with an abnormal category, wherein the abnormal category includes: missing, format error, and logical error.

4. The method for monitoring data quality according to claim 3, characterized in that: The data cleaning is to use the data processing tool Pandas to identify and process missing values, error values ​​and outliers. The data deduplication is to use a hash function to calculate the hash value of each record in the second building data, and quickly identify duplicates by comparing the hash values. The data standardization is to convert the data according to a unified format or standard.

5. A device for monitoring data quality, characterized in that: The device comprises: A first execution module is used to obtain first building data and second building data, wherein the first building data is building data obtained through the API interface transmission of the BIM model system, and the second building data is building data corresponding to a target geographical location obtained through sensor transmission; The second execution module is used to obtain a plurality of key building elements based on the graphic data in the first building data, wherein the key building elements are building elements obtained by processing the graphic data in the first building data using a Canny edge detection algorithm and a gray level co-occurrence matrix, and the second execution module also includes: A first acquisition module is used to acquire a sample image list set, wherein the sample image list set includes a plurality of sample image lists, the sample image lists include a plurality of sample images, the sample images are acquired images including a plurality of specific architectural elements, the specific architectural elements corresponding to the sample images in each sample image list are of the same type, and the specific architectural elements corresponding to the sample images in different sample image lists are of different types; The second acquisition module is used to obtain the shape features and texture features corresponding to each sample image in the sample image list set, wherein the shape features corresponding to the sample image are the shape features obtained by performing a Canny edge detection algorithm on the sample image, and the texture features corresponding to the sample image are the texture features obtained by performing a gray level co-occurrence matrix calculation on the sample image after performing a Canny edge detection algorithm; A third acquisition module is used to acquire shape features and texture features corresponding to the candidate image to be matched, wherein the candidate image to be matched is any image acquired from the graphic data in the first building data, the shape features corresponding to the candidate image to be matched are the shape features acquired by performing a Canny edge detection algorithm on the candidate image to be matched, and the texture features corresponding to the candidate image to be matched are the texture features acquired by performing a gray level co-occurrence matrix calculation on the candidate image to be matched after performing a Canny edge detection algorithm; A fourth acquisition module, configured to acquire a first similarity between the candidate image to be matched and each sample image, wherein the first similarity is a similarity acquired by using a normalized cross correlation (NCC) algorithm based on a shape feature corresponding to the candidate image to be matched and a shape feature corresponding to each sample image; a fifth acquisition module, configured to acquire a second similarity between the candidate image to be matched and each sample image, wherein the second similarity is the similarity between the texture features corresponding to the candidate image to be matched and the texture features corresponding to each sample image, obtained by using a cosine similarity algorithm after normalizing the texture features corresponding to the candidate image to be matched and the texture features corresponding to each sample image; a sixth acquisition module, configured to, when the sum of the first similarity and the second similarity between the candidate image to be matched and a sample image is greater than zero and is a maximum value, acquire a specific architectural element in the sample image as a key architectural element corresponding to the candidate image to be matched; A third execution module is used to traverse all the key building elements in the plurality of key building elements, and obtain the spatial relationship between the plurality of key building elements, wherein the spatial relationship is obtained by embedding a logic judgment statement generated based on a spatial relationship rule in the script, and identifying the spatial relationship between the key building elements according to the position relationship and size corresponding to the key building elements; A fourth execution module is used to monitor the graphic data in the first building data based on the spatial relationship to determine the first abnormal data, wherein when the spatial relationship corresponding to a certain graphic data in the first building data is missing or the spatial relationship matching is wrong or the number of spatial relationships exceeds a preset number threshold, the spatial relationship data corresponding to the graphic data is determined to be the first abnormal data; The fifth execution module is used to obtain second abnormal data based on the text data in the first building data and the second building data.

6. The device for monitoring data quality according to claim 5, characterized in that: The graphic data is data that displays building structures and building elements in a graphical manner.

7. The device for monitoring data quality according to claim 5, characterized in that: The fifth execution module 5 further includes: A data preprocessing module, used for performing data preprocessing on the second building data, wherein the preprocessing includes data cleaning, data deduplication and data standardization; A second abnormal data acquisition module is used to acquire second abnormal data, wherein the second abnormal data is abnormal data acquired by using a K-means clustering algorithm and a DBSCAN algorithm to preprocess the second building data and the text data in the first building data; The abnormal data labeling module is used to label the abnormal category of the second abnormal data, wherein the abnormal category includes: missing, format error, and logical error.

8. The device for monitoring data quality according to claim 7, characterized in that: The data cleaning is to use the data processing tool Pandas to identify and process missing values, error values ​​and outliers. The data deduplication is to use a hash function to calculate the hash value of each record in the second building data, and quickly identify duplicates by comparing the hash values. The data standardization is to convert the data according to a unified format or standard.

9. A non-transitory computer-readable storage medium, wherein at least one instruction or at least one program is stored in the non-transitory computer-readable storage medium, characterized in that: The at least one instruction or the at least one program is loaded and executed by the processor to implement the method for monitoring data quality as described in any one of claims 1-4.

10. An electronic device, characterized in that: It comprises a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by the processor to implement the method for monitoring data quality as described in any one of claims 1 to 4.

Images