Data processing method, query method, device and electronic equipment

Abstract

The present invention relates to the field of data processing technology, and specifically to a data processing method, a data query method, a device, and an electronic device. The data processing method includes obtaining model data to be processed; analyzing the model data to be processed to determine the attribute type of each attribute in the model data to be processed; compressing the model data to be processed based on the attribute type, and determining the target data corresponding to the model data to be processed. The model data to be processed is compressed using the attribute type of each attribute in the model data to be processed. The compression is performed based on different attribute types. Since the attribute types are different, the data volume is different. Compression based on the attribute type can reduce the data storage volume on the one hand, and reduce the compression time on the other hand, that is, avoid compressing data that does not need to be compressed, so as to improve data processing efficiency.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a data processing method, a query method, a device and an electronic device. Background Art

[0002] As the basis for information sharing and exchange, Building Information Modeling (BIM) contains various model data and business data, such as component attributes and model attributes. These data will flow between various participants in the building life cycle, promoting information sharing and collaborative work.

[0003] To improve the efficiency of using this data, one feasible solution is to parse model files, extract the BIM data within them, and save it in a database. However, as the scale and complexity of building models increase, the amount of data contained in them increases rapidly. A single file model may contain hundreds of thousands of components, and a single business scenario may contain millions of data items. The storage requirements can reach hundreds of billions, placing enormous pressure on the database. Summary of the Invention

[0004] In view of this, embodiments of the present invention provide a data processing method, a query method, a device, and an electronic device to solve the problem of large data storage capacity.

[0005] According to a first aspect, an embodiment of the present invention provides a data processing method, including:

[0006] Get the model data to be processed;

[0007] Analyzing the model data to be processed to determine the attribute type of each attribute in the model data to be processed;

[0008] The model data to be processed is compressed based on the attribute type, and target data corresponding to the model data to be processed is determined.

[0009] The data processing method provided by the embodiment of the present invention compresses the model data to be processed by utilizing the attribute type of each attribute in the model data to be processed. The compression is performed based on different attribute types. Since the attribute types are different, the data amounts are different. Compression is performed based on the attribute types. On the one hand, the data storage amount can be reduced, and on the other hand, the compression time can be reduced, that is, compression of data that does not need to be compressed can be avoided to improve data processing efficiency.

[0010] In combination with the first aspect, in a first implementation of the first aspect, the attribute type includes a compression attribute, and compressing the model data to be processed based on the attribute type and determining the target data corresponding to the model data to be processed includes:

[0011] Determine attribute index values ​​using the names of the attributes in the model data to be processed;

[0012] Compressing the attribute value of the compression attribute and determining the attribute value index value of the compression attribute;

[0013] The target data is determined based on the attribute index value and the attribute value index value.

[0014] In the data processing method provided by the embodiment of the present invention, since both the index value and the attribute value index value are simple numerical values, these simple numerical values ​​are used to compress the model data to be processed, thereby further reducing the amount of data storage.

[0015] In combination with the first implementation of the first aspect, in the second implementation of the first aspect, compressing the attribute value of the compressed attribute and determining the attribute value index value of the compressed attribute includes:

[0016] Counting the attribute values ​​of all the compression attributes;

[0017] A compression dictionary is constructed by respectively using a unique attribute value index value corresponding to each attribute value to determine the attribute value index value of the compression attribute.

[0018] The data processing method provided by the embodiment of the present invention uses the form of a compression dictionary to represent the correspondence between attribute values ​​and attribute value index values, so as to accurately determine the attribute value index value corresponding to each compressed attribute, thereby improving the efficiency of data processing.

[0019] In combination with the second implementation of the first aspect, in the third implementation of the first aspect, generating attribute index values ​​by using each attribute name in the to-be-processed model data includes:

[0020] Counting the attribute names in the model data to be processed;

[0021] The attribute names are sorted in a preset order, and an attribute index table is constructed to determine the attribute index value, where the attribute index value is consistent with the sequence number of the attribute name in the attribute index table.

[0022] The data processing method provided by the embodiment of the present invention uses a sorting sequence number to represent an attribute index value, simplifies the correspondence between attribute names and attribute index values, and further reduces the amount of data storage.

[0023] In combination with the third implementation manner of the first aspect, in the fourth implementation manner of the first aspect, determining the target data based on the attribute index value and the attribute value index value includes:

[0024] Deleting the attribute name of each piece of data in the to-be-processed model data, and sorting the deleted data according to the order of the attribute names in the attribute index table;

[0025] The compression dictionary is used to replace the attribute value of the compression attribute in the sorted data with the corresponding attribute value index value to obtain compressed data, so as to determine the target data.

[0026] The data processing method provided by the embodiment of the present invention makes the arrangement order of the compressed data the same as the attribute index table when compressing data, the sorting sequence number and the attribute index value are equal, and the attribute name and the attribute index value are directly matched using the arrangement order, thereby simplifying data processing.

[0027] In combination with the first aspect, in a fifth implementation of the first aspect, compressing the to-be-processed model data based on the attribute type and determining the target data corresponding to the to-be-processed model data further includes:

[0028] Get grouping requirements;

[0029] Based on the grouping requirement, the compressed data is grouped to determine the target data.

[0030] The data processing method provided by the embodiment of the present invention groups compressed data based on grouping requirements, and places compressed data that meet the same requirements in the same group, so as to facilitate rapid location of data in subsequent queries.

[0031] In combination with the fifth implementation manner of the first aspect, in the sixth implementation manner of the first aspect, grouping the compressed data based on the grouping requirement to determine the target data includes:

[0032] Based on the grouping requirements, determine the attributes for establishing the query index;

[0033] Determine whether the attribute of the query index exists in the attribute index table;

[0034] When the attribute of the query index does not exist in the attribute index table, creating an extended attribute to create the query index;

[0035] The compressed data is grouped based on the query index to determine the target data.

[0036] The data processing method provided by the embodiment of the present invention adds extended attributes based on actual needs on the basis of the attributes of the data to be processed, so as to ensure that the corresponding data can be accurately queried subsequently.

[0037] In combination with the first aspect, in a seventh implementation of the first aspect, the method further includes:

[0038] Serialize the target data and write it into a corresponding file to obtain a target file;

[0039] The target file is compressed and saved to a preset file system.

[0040] The data processing method provided by the embodiment of the present invention serializes the target data according to actual needs and then stores it, so that it has higher performance; and the target data is stored in a file system manner, which can achieve simple horizontal expansion and avoid the data reallocation problem that needs to be solved when the database is horizontally expanded.

[0041] In combination with the first aspect, in an eighth implementation of the first aspect, the method further includes:

[0042] Get query conditions;

[0043] A query is performed in the target data based on the query condition and the query index to determine a query result.

[0044] The data processing method provided by the embodiment of the present invention uses query conditions and query indexes to determine query results. Since the query indexes have divided the data, querying based on them can improve query efficiency.

[0045] According to a second aspect, an embodiment of the present invention further provides a data query method, comprising:

[0046] Get query conditions;

[0047] A query is performed in the target data based on the query condition to determine a query result, wherein the target data is obtained by compressing the model data to be processed based on the attribute type of each attribute in the model data to be processed.

[0048] The data query method provided by the embodiment of the present invention performs data query in compressed data, which can reduce the amount of data query and improve query efficiency.

[0049] In conjunction with the second aspect, in a first implementation of the second aspect, querying the target data based on the query condition and determining the query result includes:

[0050] Obtaining a query index and a compression dictionary, wherein the compression dictionary is used to represent a correspondence between attribute values ​​and attribute value index values;

[0051] Determine corresponding data groups and corresponding data files using the query condition and the query index;

[0052] Based on the compression dictionary and the query condition, the target data in the data file is matched to determine the query result.

[0053] In combination with the first implementation of the second aspect, in the second implementation of the second aspect, matching the target data in the data file based on the compression dictionary and the query condition to determine the query result includes:

[0054] Preprocessing the query condition using the compression dictionary to obtain a processed query condition;

[0055] Obtaining an attribute index value from the processed query condition;

[0056] Determine the position of the query result in the target data using the attribute index value, and extract the value of the corresponding position;

[0057] The extracted value is processed based on the attribute type of the attribute corresponding to the attribute index value, and matched with the value in the query condition to determine the query result.

[0058] The data query method provided by the embodiment of the present invention uses a compression dictionary to preprocess the query conditions, and can replace the corresponding query conditions with the attribute value index values ​​in the compression dictionary. In the specific query process, the attribute value index values ​​can be directly used for matching queries, thereby improving query efficiency.

[0059] In combination with the second embodiment of the second aspect, in the third embodiment of the second aspect, the attribute type of the attribute corresponding to the attribute index value, restoring the extracted value to the original value to determine the query result, includes:

[0060] Determine whether the query condition is a fuzzy match;

[0061] When the query condition is the fuzzy match, a fuzzy match is performed based on the original value and the value in the processed query condition to determine the query result.

[0062] The data query method provided by the embodiment of the present invention, during fuzzy matching, after obtaining the original value, needs to use the original value and the processed query condition again to perform a matching query, thereby improving the accuracy of the query result.

[0063] According to a third aspect, an embodiment of the present invention further provides a data processing device, including:

[0064] A first acquisition module is used to acquire model data to be processed;

[0065] An analysis module, configured to analyze the model data to be processed and determine the attribute type of each attribute in the model data to be processed;

[0066] A compression module is used to compress the model data to be processed based on the attribute type and determine the target data corresponding to the model data to be processed.

[0067] The data processing device provided by an embodiment of the present invention compresses the model data to be processed by utilizing the attribute type of each attribute in the model data to be processed. The compression is performed based on different attribute types. Since the attribute types are different, the data amounts are different. Compression based on the attribute types can reduce the data storage amount on the one hand, and on the other hand, it can also reduce the compression time, that is, avoid compressing data that does not need to be compressed, so as to improve data processing efficiency.

[0068] According to a fourth aspect, an embodiment of the present invention further provides a data query device, comprising:

[0069] The second acquisition module is used to obtain query conditions;

[0070] The query module is used to query the target data based on the query conditions and determine the query results. The target data is obtained by compressing the model data to be processed based on the attribute type of each attribute in the model data to be processed.

[0071] The data query device provided by the embodiment of the present invention performs data query in compressed data, which can reduce the amount of data query and improve query efficiency.

[0072] According to the fifth aspect, an embodiment of the present invention provides an electronic device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the computer instructions to execute the data processing method described in the first aspect or any one of the embodiments of the first aspect, or executing the data query method described in the second aspect or any one of the embodiments of the second aspect.

[0073] According to the sixth aspect, an embodiment of the present invention provides a computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable the computer to execute the data processing method described in the first aspect or any one of the embodiments of the first aspect, or to execute the data query method described in the second aspect or any one of the embodiments of the second aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0075] Figure 1 is a flow chart of a data processing method according to an embodiment of the present invention;

[0076] Figure 2 is a flow chart of a data processing method according to an embodiment of the present invention;

[0077] Figure 3 is an example diagram of compression of model data to be processed according to an embodiment of the present invention;

[0078] Figure 4 is a flow chart of a data processing method according to an embodiment of the present invention;

[0079] Figure 5 is a flow chart of a data processing method according to an embodiment of the present invention;

[0080] Figure 6 is a flow chart of a data query method according to an embodiment of the present invention;

[0081] Figure 7 is a flow chart of a data query method according to an embodiment of the present invention;

[0082] Figure 8 is a flow chart of a data query method according to an embodiment of the present invention;

[0083] Figure 9 is a structural block diagram of a data processing device according to an embodiment of the present invention;

[0084] Figure 10 is a structural block diagram of a data query device according to an embodiment of the present invention;

[0085] Figure 11 It is a schematic diagram of the hardware structure of the electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0086] To make the purpose, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative efforts shall fall within the scope of protection of the present invention.

[0087] For the convenience of the following description, the following terms are explained as follows:

[0088] (1) Common attributes: attributes that are included in all data in the model and have the same value; for example, in a model of a single building, all components belong to the same building and all components have the same basic building information, then this same basic building information is classified as common attributes.

[0089] (2) Non-compressible attributes: attributes in the model whose value type is a simple numeric value and attributes whose value is unique; for example, a component has attributes such as floor ID and component ID, and the value type is integer; or each component has a unique identifier, then these attributes are classified as non-compressible attributes.

[0090] (3) Compressed attributes: In addition to common attributes and non-compressed attributes, attributes with the same value for multiple data items and other attributes in the model; for example, in a model of a single building, all components belonging to the same floor have the same floor information, and components on different floors have different floor information. In this case, attributes in the floor information that are not simple numerical types are classified as compressed attributes.

[0091] (4) Attribute index value: Each attribute name is matched with a unique integer number, which is called the "attribute index value".

[0092] (5) Attribute value index value: For each possible value of a compressed attribute, a unique integer number corresponds to it. This integer number is called the "attribute value index value".

[0093] (6) Compression dictionary: The correspondence between all compression attribute values ​​and attribute index values ​​is summarized into a dictionary table, which is called the "compression dictionary".

[0094] (7) Data compression: Use the compression dictionary to replace the attribute values ​​in the model data with the attribute value index value to reduce the size of the data itself.

[0095] (8) File compression: Use existing file compression tools to process data files into compressed files.

[0096] The data processing method provided by the embodiment of the present invention analyzes the attributes contained in the data based on the data characteristics, determines the attribute type, and compresses the data according to different attribute types to obtain compressed data to determine the target data.

[0097] Furthermore, embodiments of the present invention construct a query index for the compressed data according to business requirements, and group the compressed data according to the query index to determine target data. The query index can locate the corresponding data group according to the query conditions to obtain the corresponding query results.

[0098] Furthermore, in an embodiment of the present invention, the grouped compressed data is optionally stored in the cloud in a universal compressed file format. The files corresponding to each model can be independently stored in a folder marked by a unique identifier, and each group of compressed data is stored as a separate compressed file in that folder. Accordingly, upon receiving a query request, the electronic device can obtain a compression dictionary from the cloud, pre-process the query criteria based on the compression dictionary, determine the compressed data group based on the query index, and then pull the compressed data file corresponding to the group from the cloud for decompression, processing, and restoration.

[0099] According to an embodiment of the present invention, an embodiment of a data processing method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0100] In this embodiment, a data processing method is provided, which can be used in electronic devices such as computers, mobile phones, tablet computers, etc. Figure 1 is a flow chart of a data processing method according to an embodiment of the present invention. Figure 1 As shown, the process includes the following steps:

[0101] S11, obtaining the model data to be processed.

[0102] The model data to be processed can be all data in a BIM model, or part of the data, etc., and there is no limitation on this. The BIM model can be newly created on the electronic device, or obtained from the outside by the electronic device, etc.

[0103] S12, analyzing the model data to be processed, and determining the attribute type of each attribute in the model data to be processed.

[0104] Each piece of data in the model data to be processed may include one or more attributes; and the attributes included in each piece of data may be the same or different; the attribute value of the same attribute in different pieces of data may be the same or different.

[0105] It should be noted that the model data to be processed is not limited to component attributes, but rather depends on the attributes contained in the data to be stored in a specific business scenario. For example, in a business scenario, the data to be stored is component information. In addition to the component's own attributes, each data point may also contain additional attributes such as the room name and floor name to which these components belong. In other words, this embodiment focuses only on how the attributes contained in a piece of data can be classified and the processing of the classified data, without considering whether the attributes are specific to components or floors.

[0106] After acquiring the model data to be processed, the electronic device can analyze each piece of data to determine the attribute type of each attribute in each piece of data. The attribute type is determined by the value of the attribute. That is, by analyzing the attribute value of each attribute in each piece of data, the electronic device can determine the attribute type of the attribute. Specifically, it can be divided into the following situations:

[0107] (1) If an attribute has only one value in all data, its attribute type is determined to be a universal attribute;

[0108] (2) If the number of values ​​of an attribute in all data is equal to the number of data items, then its attribute type is determined to be a non-compressed attribute;

[0109] (3) If the value type of an attribute is a simple numeric value, then the attribute type is determined to be a non-compressed attribute, wherein the simple numeric value is an integer or a decimal number;

[0110] (4) For attributes that do not meet the above three conditions, their attribute type is determined to be a compressed attribute.

[0111] S13: compress the model data to be processed based on the attribute type, and determine the target data corresponding to the model data to be processed.

[0112] After determining the attribute type of each attribute, the electronic device compresses the corresponding data according to the attribute type to determine the target data. For example, for general attributes, compression is not required; for non-compressed attributes, only the attribute name can be compressed; for compressed attributes, both the attribute name and attribute value can be compressed, etc.

[0113] Specifically, compression uses unique numerical values ​​to represent each attribute name or value; that is, each attribute in the data is represented numerically to reduce the amount of data stored. This is because, after compression, only one copy of the original character value is saved, and the remaining character values ​​are replaced with numbers. Compared to character data, the amount of data in numerical format is smaller.

[0114] This step will be described in detail below.

[0115] The data processing method provided in this embodiment compresses the model data to be processed by utilizing the attribute type of each attribute in the model data to be processed. The compression is performed based on different attribute types. Since different attribute types have different data volumes, compression based on the attribute type can reduce the amount of data storage on the one hand, and on the other hand, can also reduce the compression time, that is, avoid compressing data that does not need to be compressed, so as to improve data processing efficiency.

[0116] In this embodiment, a data processing method is provided, which can be used in electronic devices such as computers, mobile phones, tablet computers, etc. Figure 2 is a flow chart of a data processing method according to an embodiment of the present invention. Figure 2 As shown, the process includes the following steps:

[0117] S21, obtaining the model data to be processed.

[0118] For details, please see Figure 1 S11 of the illustrated embodiment will not be described in detail here.

[0119] S22: Analyze the model data to be processed and determine the attribute type of each attribute in the model data to be processed.

[0120] For details, please see Figure 1 S12 of the illustrated embodiment will not be described in detail here.

[0121] S23: compress the model data to be processed based on the attribute type, and determine the target data corresponding to the model data to be processed.

[0122] Specifically, the above S23 includes:

[0123] S231: Determine attribute index values ​​using the names of the attributes in the model data to be processed.

[0124] As described above, common attributes are stored separately. Therefore, the attribute index values ​​here refer to both compressed and uncompressed attributes. Electronic devices can use unique numbers to represent each attribute name. In this embodiment, these unique numbers are referred to as attribute index values. Electronic devices can assign an attribute index value to each attribute name to establish a corresponding relationship between the attribute name and the attribute index value.

[0125] In some optional implementations of this embodiment, the above S231 may include:

[0126] (1) Count the attribute names in the model data to be processed.

[0127] The electronic device can count the attribute names of compressed attributes and non-compressed attributes in the model data to be processed. Figure 3 As shown, for original data 1, the attributes included are: general attribute 1, non-compressed attribute 1, non-compressed attribute 2, compressed attribute 1, and compressed attribute 2. For original data 2, the attributes included are: general attribute 1, non-compressed attribute 1, non-compressed attribute 2, compressed attribute 1, and compressed attribute 2.

[0128] Both raw data 1 and raw data 2 include three attribute types: general attributes, uncompressed attributes, and compressed attributes. Raw data 1 includes two uncompressed attributes: uncompressed attribute 1 and uncompressed attribute 2; and two compressed attributes: compressed attribute 1 and compressed attribute 2. Raw data 2 includes two uncompressed attributes: uncompressed attribute 1 and uncompressed attribute 2; and two compressed attributes: compressed attribute 1 and compressed attribute 2.

[0129] (2) Sort the attribute names in a preset order, construct an attribute index table, and determine an attribute index value, which is consistent with the sequence number of the attribute name in the attribute index table.

[0130] Electronic devices first analyze the attribute type of each attribute, then count the attribute names under each attribute type. Each attribute name is associated with a unique integer, which becomes the attribute index value. The structure that stores the correspondence between attribute index values ​​and attribute names is called the attribute index table.

[0131] The correspondence between the attribute index value and the attribute name may be in a fixed order or in other ways, and there is no limitation on this. The corresponding settings may be made according to actual needs.

[0132] In this embodiment, attribute names are sorted according to a preset order to construct an attribute index table to determine attribute index values. Continuing with the previous example, raw data 1 includes five attribute values, each corresponding to an attribute name. Therefore, when constructing the attribute index table, the ordering of the attribute index values ​​in the attribute index table, with the exception of the common attributes, is consistent with the ordering of the attribute names in raw data 1. In raw data 1, uncompressed attribute 1 is ranked first, excluding common attributes 1. Accordingly, in the attribute index table, the attribute index value corresponding to uncompressed attribute 1 is also ranked first.

[0133] Using sorting numbers to represent attribute index values ​​simplifies the correspondence between attribute names and attribute index values, further reducing the amount of data storage.

[0134] S232: Compress the attribute value of the compression attribute and determine the attribute value index value of the compression attribute.

[0135] The electronic device may use a unique number to represent each attribute value of the compression attribute. In this embodiment, the unique number is referred to as an attribute value index. The electronic device may assign an attribute value index to each attribute value to establish a corresponding relationship between the attribute value and the attribute value index.

[0136] In some optional implementations of this embodiment, the above S232 may include:

[0137] (1) Count the attribute values ​​of all compression attributes.

[0138] The electronic device counts the values ​​of the compression attributes in all data, that is, the attribute values.

[0139] (2) Using the unique attribute value index value corresponding to each attribute value, a compression dictionary is constructed to determine the attribute value index value of the compressed attribute.

[0140] For each possible value, the electronic device assigns a unique integer to it, which becomes the attribute value index. The correspondence between the attribute values ​​and attribute value indexes of all compressed attributes is summarized into a dictionary table, which is called the compression dictionary.

[0141] The corresponding relationship between attribute values ​​and attribute value index values ​​is represented in the form of a compression dictionary, so as to accurately determine the attribute value index value corresponding to each compressed attribute, thereby improving the efficiency of data processing.

[0142] like Figure 3 As shown, corresponding to the original data, the established attribute index table and compression dictionary can be expressed as:

[0143] Attribute index table: {"uncompressed attribute 1","uncompressed attribute 2","compressed attribute 1","compressed attribute 2"};

[0144] Compression dictionary: {"attribute value index value 1":"value 3-1",

[0145] "Attribute value index value 2": "value 3-2",

[0146] "Attribute Value Index Value 3": "Value 3-3"

[0147] }

[0148] It should be noted that different models have independent compression dictionaries and attribute index tables, and different business scenarios of the same model can also generate independent compression dictionaries and attribute index tables. For example, if the same model has data for two business scenarios, component information and quantity calculation, independent compression dictionaries and attribute index tables can be generated for each.

[0149] S233: Determine target data based on the attribute index value and the attribute value index value.

[0150] After processing the above steps, the electronic device has constructed an attribute index table and a compression dictionary. By using the correspondence between the attribute name and the attribute index value in the attribute index table, and the correspondence between the attribute value and the attribute index value in the compression dictionary, the model data to be processed can be processed to determine the target data.

[0151] During specific processing, for each piece of data in the model data to be processed, the corresponding attribute name and attribute value can be replaced by the attribute index value and the attribute value index value respectively, so as to determine the target data.

[0152] In some optional implementations of this embodiment, the above S233 may include:

[0153] (1) Delete the attribute name of each data in the model data to be processed, and sort the deleted data according to the order of the attribute names in the attribute index table.

[0154] The electronic device erases the attribute name of each data in the model data to be processed, and only retains the attribute values ​​of the compressed attribute and the uncompressed attribute. The arrangement order is the same as the attribute index table, and the sorting sequence number is equal to the attribute index value.

[0155] (2) Using the compression dictionary, the attribute value of the compressed attribute in the sorted data is replaced with the corresponding attribute value index value to obtain the compressed data to determine the target data.

[0156] For non-compressible attributes, their attribute values ​​do not need to be processed. For compressed attributes, the compression dictionary is used to replace the attribute values ​​with the corresponding attribute index values ​​to obtain compressed data. The compressed data can be used directly as the target data, or it can be grouped and then used as the target data. The specific settings can be adjusted according to actual needs and are not limited here.

[0157] Optionally, the data after deleting the attribute name can be saved in array form, and the attribute value can be retrieved with O(1) time complexity through the attribute index value. Specifically, each data is represented by an array. If there are N data in the model data to be processed, then there are N corresponding arrays.

[0158] like Figure 3 As shown, corresponding to the original data 1, the compressed data can be expressed as:

[0159] ["value 2 - 1","value 2 - 2","attribute value index value 1","attribute value index value 2"].

[0160] Corresponding to the above-mentioned attribute index table, the first attribute name in the compressed data is: non-compressed attribute 1, and its attribute value is: value 2-1; the second attribute name in the compressed data is: non-compressed attribute 2, and its attribute value is: value 2-2; the third attribute name in the compressed data is: compressed attribute 1, and its attribute value index value is: attribute value index value 1. By searching the compression dictionary, it is determined that the attribute value index value 1 corresponds to the value 3-1. Therefore, it can be determined that the attribute value of compressed attribute 1 is: value 3-1; the fourth attribute name in the compressed data is: compressed attribute 2, and its attribute value index value is: attribute value index value 2. By searching the compression dictionary, it is determined that the attribute value index value 2 corresponds to the value 3-2. Therefore, it can be determined that the attribute value of compressed attribute 2 is: value 3-2.

[0161] like Figure 3 As shown, common attributes are saved separately, non-compressed attributes and compressed attributes are arranged in a fixed order, and the order of values ​​in the compressed data is consistent with the attribute index table; the attribute value of the compressed attribute is replaced by the attribute value index value, and the correspondence between the attribute value and the attribute value index value is saved in the compression dictionary.

[0162] When compressing data, the arrangement order of the compressed data is made the same as the attribute index table, the sorting sequence number and the attribute index value are equal, and the attribute name and the attribute index value are directly matched using the arrangement order, which simplifies data processing.

[0163] In the data processing method provided in this embodiment, since both the index value and the attribute value index value are simple numerical values, these simple numerical values ​​are used to compress the model data to be processed, thereby further reducing the amount of data storage.

[0164] In this embodiment, a data processing method is provided that can be used in electronic devices such as computers, mobile phones, and tablet computers. In this embodiment, by grouping data and querying indexes, when acquiring model data, only the model data of the corresponding group needs to be transmitted, which reduces the network time consumed by data transmission and alleviates the memory pressure of the data user. Specifically, Figure 4 is a flow chart of a data processing method according to an embodiment of the present invention. Figure 4 As shown, the process includes the following steps:

[0165] S31, obtaining the model data to be processed.

[0166] For details, please see Figure 2 S21 of the illustrated embodiment will not be described in detail here.

[0167] S32: Analyze the model data to be processed and determine the attribute type of each attribute in the model data to be processed.

[0168] For details, please see Figure 2 S22 of the illustrated embodiment will not be described in detail here.

[0169] S33: compress the model data to be processed based on the attribute type, and determine the target data corresponding to the model data to be processed.

[0170] For details on how to obtain compressed data, see Figure 2 S23 of the embodiment shown is not described here any more. In this embodiment, the main problem described is how to group and store the compressed data.

[0171] Specifically, the above S33 may include:

[0172] S331, obtaining grouping requirements.

[0173] Grouping requirements can be determined by business needs or manually defined. There are no restrictions here and you can set them accordingly based on your actual needs. For example, if you need to filter components by floor in real business, you can use the floor ID as an attribute to build a query index.

[0174] S332: Group the compressed data based on grouping requirements to determine target data.

[0175] After obtaining the grouping requirements, the electronic device groups the compressed data and performance according to the requirements, and then determines the target data.

[0176] In some optional implementations of this embodiment, the above S332 may include:

[0177] (1) Based on the grouping requirements, determine the attributes for establishing the query index.

[0178] After obtaining the grouping requirements, we can determine which attributes to use to construct the query index. For example, if we need to group by floor, then the attribute required for the query index is floor; if we need to group by room, then the attribute required for the query index is room.

[0179] (2) Determine whether the attribute of the query index exists in the attribute index table.

[0180] When the attribute of the query index does not exist in the attribute index table, step (3) is executed; otherwise, the query index is directly established using the attributes in the attribute index table.

[0181] (3) Create extended attributes to build query indexes.

[0182] Specifically, in addition to the attributes inherent in the model data being processed, electronic devices can also add extended attributes based on actual needs. These attributes can be new attributes generated during business processing or attributes specifically used to build query indexes. For example, in some business scenarios, components may have grouping criteria other than floor and room. This grouping criterion can be added to the metadata as an extended attribute. The extended attributes are then subjected to the same attribute type analysis, data compression, and query index construction as described above.

[0183] (4) Group the compressed data based on the query index to determine the target data.

[0184] Optionally, the following rules can be used for grouping:

[0185] 1) For attributes that serve as unique identifiers, data can be sorted based on this attribute and divided into multiple data groups of similar size. Each data group has a value range as index information. For example, each component has a unique ID. All components can be sorted and grouped based on component ID. Each group has a component ID range. To query based on component ID, only the groups with matching component ID ranges need to be searched.

[0186] 2) For attributes with one-to-many relationships, data can be grouped based on the attribute, with the attribute value serving as index information. For example, if there is a one-to-many relationship between rooms and components, and each room has a unique ID, components can be grouped based on the room ID. To query based on the room ID, simply search for the component group corresponding to that room ID.

[0187] 3) For attributes with many-to-many relationships, the grouping strategy of one-to-many relationships can be used, in which data that is classified into multiple data groups at the same time is deduplicated during the search. It is also possible to build an index information that contains multiple values ​​at the same time. For example, suppose there are rooms 1 and 2, which both contain component 1, while component 2 only belongs to room 2. Then you can build the following grouping: component group 1 is marked by the IDs of room 1 and room 2, in which all components that belong to both room 1 and room 2 are stored, that is, component 1; component group 2 is marked by the ID of room 2, in which all components that belong only to room 2 are stored, that is, component 2. When querying the components belonging to room 2, take the union of component group 1 and component group 2.

[0188] 4) When two query indexes have a subordinate relationship, you can map the higher-level query index to the other query index instead of creating a group. For example, if you have already created a query index from room ID to component, and there is a one-to-many relationship between floors and rooms, you can create an index from floor ID to room ID instead of a floor to component query index. To query based on floor ID, first obtain the corresponding room ID, then the corresponding component group.

[0189] In addition to the attributes of the data being processed, extended attributes are added based on actual needs to ensure accurate data retrieval. The query index can be used to indicate how the data is grouped. Once the query conditions are obtained, the query index can be combined to determine the data group and data file containing the query results.

[0190] Furthermore, the electronic device can optionally determine the size of each data group based on the total amount of model data and query requirements. For attributes with potential traversal query requirements, the data group size can be set at 50,000 data items and adjusted based on the total amount of data and the frequency of traversal queries. For attributes where individual data groups can be accurately located, the data group size can be set at 10,000 data items. In actual applications, this size can be adjusted based on factors such as electronic device performance and network transmission speed.

[0191] S34, serializing the target data and writing it into a corresponding file to obtain a target file.

[0192] After obtaining the target data, the electronic device uses files to store it, and the storage space occupied can be further reduced by file compression.

[0193] Specifically, the target data is written to a file. Based on the grouping results, each data group is saved as a separate data file, and the data file location of each data group is saved in the query index. The data file corresponding to all target data of the model data to be processed is called the target file.

[0194] Optionally, use custom serialization and deserialization methods to write and read compressed data. Each compressed data entry is saved as a line in a data file, with the attribute values ​​of each attribute in a line separated by a custom delimiter. Special characters that may be present in uncompressed attributes are escaped, such as the delimiter used in the custom serialization method.

[0195] When you need to read data from a data file, you can read and deserialize the data row by row, without having to read all the data at once, reducing memory pressure.

[0196] S35, compressing the target file and saving it to a preset file system.

[0197] Save the target file in a compressed file format to the file system. Compared with directly transmitting the original file, the data user first obtains the compressed file and then decompresses it, which can reduce data transmission time and achieve better performance.

[0198] Optionally, saving in 7z format can achieve a higher compression ratio, saving in gzip format allows the data file to be opened directly on the browser, and other compression algorithms can also be used according to actual needs.

[0199] Optionally, define a unique identifier for each model, such as the MD5 digest of the entire model, and save all the model's data files, compression dictionaries, query indexes, and other information to the folder defined by the unique identifier. Data for each model is stored independently to avoid the impact of increased server-side data on query efficiency.

[0200] Furthermore, with the help of file storage systems such as OSS, you can directly locate the specified data file by file name. The correspondence between the model and the data file storage location can be directly saved in a common database.

[0201] The data processing method provided in this embodiment groups compressed data based on grouping requirements, placing compressed data that meets the same requirements in the same group. This facilitates rapid data location during subsequent queries. Furthermore, target data is serialized and stored based on actual needs, resulting in higher performance. Storing target data in a file system facilitates simple horizontal expansion, avoiding the data reallocation issues that often arise when scaling a database.

[0202] Figure 5 The main data processing process is shown, including: after obtaining the model data to be processed, the attributes of each data item in the model data are classified to determine the attribute type; then, whether a compression attribute exists in the attribute type is determined. If a compression attribute exists, the data is compressed to generate a compression dictionary, and the attribute value of the compressed attribute is replaced with the attribute value index; the attribute names of the compressed and non-compressed attributes are erased to generate an attribute index; then, based on business needs, the compressed data is grouped and a query index is generated. Finally, the data files are grouped and saved to the file system, and the compression dictionary and query index are stored.

[0203] In some optional implementations of this embodiment, the data processing method further provides a data query function. Specifically, the method further includes:

[0204] (1) Obtain query conditions.

[0205] The electronic device provides a data query interface on which the user can enter query conditions. The query conditions can be the data the user wants to obtain, for example, the floor name query condition is "floorName=first floor".

[0206] (2) Query the target data based on the query conditions and query index to determine the query results.

[0207] The electronic device uses the query conditions and the query index constructed above to determine the group where the query result in the target data is located and extracts the corresponding data file for query, and finally determines the query result.

[0208] The data query process will be described in detail below.

[0209] The data processing method provided in this embodiment uses query conditions and query indexes to determine query results. Since the query indexes have divided the data, querying based on them can improve query efficiency.

[0210] The data processing method provided in the embodiments of the present invention can simultaneously store massive amounts of data while providing a certain level of data retrieval capability. Even with a total data volume in the tens of billions, a single model with hundreds of thousands of data items can achieve a response time of milliseconds to seconds. Furthermore, because data is stored in a file system, simple horizontal expansion is possible, avoiding the data reallocation issue that typically arises when scaling a database.

[0211] According to an embodiment of the present invention, an embodiment of a data query method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0212] In this embodiment, a data query method is provided, which can be used in electronic devices such as computers, mobile phones, tablet computers, etc. The data query method corresponds to the data processing method described above. The data processing method is used to describe the process of compressing and storing the original data after obtaining it, and the data query method is used to describe the query process of the compressed data. Specifically, Figure 6 is a flow chart of a data processing method according to an embodiment of the present invention. Figure 6 As shown, the process includes the following steps:

[0213] S41, obtaining query conditions.

[0214] The electronic device provides a data query interface on which the user can enter query conditions. The query conditions can be the data the user wants to obtain, for example, the floor name query condition is "floorName=first floor".

[0215] S42: Query the target data based on the query conditions and determine the query result.

[0216] The target data is obtained by compressing the model data to be processed based on the attribute type of each attribute in the model data to be processed.

[0217] The electronic device uses the query conditions and the query index constructed above to determine the group where the query result in the target data is located and extracts the corresponding data file for query, and finally determines the query result.

[0218] Specifically, the electronic device uses the query index and query conditions to determine the group where the query data is located, and obtains the corresponding data file from the group; since the data in the data file is compressed data, it also needs to be decompressed, that is, each attribute index value and attribute value index value is restored to the original attribute name and attribute value.

[0219] This step will be described in detail below.

[0220] The data query method provided in this embodiment performs data query on compressed data, which can reduce the amount of data to be queried and improve query efficiency.

[0221] In this embodiment, a data query method is provided, which can be used in electronic devices such as computers, mobile phones, tablet computers, etc. Figure 7 is a flow chart of a data processing method according to an embodiment of the present invention. Figure 7 As shown, the process includes the following steps:

[0222] S51, obtaining query conditions.

[0223] For details, please see Figure 6 S41 of the illustrated embodiment will not be described in detail here.

[0224] S52: Query the target data based on the query conditions and determine the query result.

[0225] The target data is obtained by compressing the model data to be processed based on the attribute type of each attribute in the model data to be processed.

[0226] Specifically, the above S52 may include:

[0227] S521: Obtain a query index and a compression dictionary.

[0228] The compression dictionary is used to represent the corresponding relationship between attribute values ​​and attribute value index values.

[0229] As mentioned above, the compression dictionary and query index of different models are stored in different locations. Therefore, the electronic device can obtain the compression dictionary and query index of the specified model from the corresponding folder using the unique identifier of the model.

[0230] S522: Determine the corresponding data group and the corresponding data file using the query condition and the query index.

[0231] As mentioned above, the query index is used to represent the data grouping. Therefore, by using the query condition and the query index, the corresponding data grouping and the corresponding data file can be determined.

[0232] S523: Match the target data in the data file based on the compression dictionary and the query condition to determine the query result.

[0233] The electronic device uses the query condition and the query index to determine the location of the data file; then uses the compression dictionary and the query condition to match the target data in the determined data file, thereby determining the query result.

[0234] In some optional implementations of this embodiment, the above S523 may include:

[0235] (1) Use the compression dictionary to preprocess the query conditions to obtain the processed query conditions. When a conditional query is required, preprocess the query conditions according to the compression dictionary. Obtain the attribute index value and attribute value index value of the query condition according to the compression dictionary and replace the original query condition. For example, the floor name query condition is "floorName = 'first floor'", the attribute index value of the floor name is 2, and the attribute value index value of "first floor" is 10, then the preprocessed query condition is "2 = 10".

[0236] If the attribute type corresponding to the attribute index value in the query condition is a non-compressed attribute, the corresponding attribute index value does not exist in the compression dictionary and is not replaced. For example, if the component name query condition is "elementName = 'door'" and the attribute index value of component name is 1, and component name is a non-compressed attribute, the pre-processed query condition will be "1 = 'door'."

[0237] Optionally, before pre-processing the query condition, the electronic device needs to determine whether the query condition contains a fuzzy match. If a fuzzy match occurs in the query condition, the attribute value index value is not replaced. For example, if you need to query data where the floor name contains "first floor", the query condition is "floorName LIKE 'first floor'", then the pre-processed query condition is "2LIKE 'first floor'";

[0238] Whether a query condition includes a fuzzy match query depends on the predicate definition during solution implementation, that is, the fuzzy match keyword. For example, you can refer to "LIKE" in SQL and define "LIKE" as a fuzzy match predicate. When "LIKE" appears in a query condition, the corresponding single query condition is considered a fuzzy match.

[0239] (2) Obtain the attribute index value from the processed query condition.

[0240] As described above, after the electronic device pre-processes the query condition, it replaces the attribute name with the attribute index value, so that the attribute index value can be directly obtained for subsequent queries.

[0241] (3) Use the attribute index value to determine the position of the attribute to be matched in the target data and extract the value of the corresponding position.

[0242] As mentioned above, the electronic device can use the query conditions and query index to determine the data group files. Then, the required data files can be downloaded and decompressed from the cloud in sequence, the compressed data can be read from the data files, and the query conditions can be matched or other processing can be performed on the target data one by one to obtain the intermediate results.

[0243] Specifically, the electronic device can traverse each piece of data in the data file and extract the value of the attribute to be matched:

[0244] a. Obtain attribute index values ​​from preprocessed query conditions;

[0245] b. According to the attribute index value, get the value of the corresponding position of the compressed data; for example, if the attribute index value is 1, get the value of the first bit of the compressed data.

[0246] (4) The extracted value is processed based on the attribute type of the attribute corresponding to the attribute index value and matched with the value in the query condition to determine the query result.

[0247] The extracted value is processed based on the attribute type of the attribute corresponding to the attribute index value to determine whether the extracted value needs to be restored to the original value.

[0248] After processing the extracted values, they are matched with the values ​​in the processed query conditions, and all data that meets the query conditions are collected and output as intermediate results.

[0249] Finally, the attribute index values ​​and attribute value index values ​​in the intermediate results are restored to their original values ​​according to the compression dictionary.

[0250] In some optional implementations of this embodiment, the above step (4) may further include the following steps:

[0251] 4.1) Determine whether the query condition is a fuzzy match.

[0252] When the query condition is the fuzzy match, execute step 4.2); otherwise, execute step 4.3).

[0253] 4.2) Perform fuzzy matching based on the original value and the processed value in the query condition to determine the query result.

[0254] 4.3) Directly match the extracted value with the value in the processed query condition to determine the query result.

[0255] Specifically, if the query condition is a fuzzy match, the recovered original value is retrieved and then a fuzzy match is performed against the original value in the query condition. The implementation details of fuzzy matching, such as string fuzzy matching, are implemented in specific code and are not specifically limited. You can configure it accordingly based on your actual needs. If the query condition is not a fuzzy match, the extracted value is directly matched against the preprocessed query condition value, without first restoring the original value before matching.

[0256] Among them, if the attribute type of the attribute corresponding to the attribute index value is not a compressed attribute, the extracted value and the value in the pre-processed query condition are both original values ​​and can also be directly matched without processing.

[0257] Restoring the extracted value to its original value is implemented as follows: If the attribute type corresponding to the attribute index value is compressed, the attribute index value is used as the attribute value index value to query the compression dictionary and obtain the corresponding attribute value. If the attribute type corresponding to the attribute index value is uncompressed, the extracted value is the original value and no processing is required. Restoring the attribute index value and attribute value index value in the intermediate result is implemented similarly.

[0258] The data query method provided in this embodiment uses a compression dictionary to preprocess the query conditions, and can replace the corresponding query conditions with the attribute value index values ​​in the compression dictionary. In the specific query process, the attribute value index values ​​can be directly used for matching queries, and digital matching can be used instead of character matching, thereby improving query efficiency.

[0259] Figure 8It shows a specific process of data query, which includes obtaining the compression dictionary and query index, preprocessing the query conditions, and determining the data file based on the query index; obtaining the data file from the file system and decompressing it, matching the query conditions or performing business processing, and finally realizing data restoration.

[0260] In this embodiment, a data processing device or data query device is also provided, which is used to implement the above-mentioned embodiments and preferred embodiments. The details already described will not be repeated here. As used below, the term "module" can refer to a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation using hardware, or a combination of software and hardware, is also possible and contemplated.

[0261] This embodiment provides a data processing device, such as Figure 9 Shown, including:

[0262] A first acquisition module 61 is used to acquire model data to be processed;

[0263] An analysis module 62 is configured to analyze the model data to be processed and determine the attribute type of each attribute in the model data to be processed;

[0264] The compression module 63 is configured to compress the model data to be processed based on the attribute type and determine target data corresponding to the model data to be processed.

[0265] The data processing device provided in this embodiment compresses the model data to be processed by utilizing the attribute type of each attribute in the model data to be processed. The compression is performed based on different attribute types. Since different attribute types have different data volumes, compression based on the attribute type can reduce the amount of data storage on the one hand, and on the other hand, can also reduce the compression time, that is, avoid compressing data that does not need to be compressed, so as to improve data processing efficiency.

[0266] This embodiment also provides a data query device, such as Figure 10 Shown, including:

[0267] The second acquisition module 71 is used to obtain query conditions;

[0268] The query module 72 is configured to query the target data based on the query conditions and determine query results. The target data is obtained by compressing the model data to be processed based on the attribute types of each attribute in the model data to be processed.

[0269] The data query device provided in this embodiment performs data query in compressed data, which can reduce the amount of data to be queried and improve query efficiency.

[0270] The data processing device or data query device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0271] The further functional description of each of the above modules is the same as that of the above corresponding embodiments and will not be repeated here.

[0272] An embodiment of the present invention further provides an electronic device having the above Figure 9 The data processing device shown, or the above Figure 10 The data query device shown.

[0273] See also Figure 11 , Figure 11 is a structural diagram of an electronic device provided by an optional embodiment of the present invention, such as Figure 11 As shown, the electronic device may include: at least one processor 81, such as a CPU (Central Processing Unit), at least one communication interface 83, a memory 84, and at least one communication bus 82. The communication bus 82 is used to realize the connection and communication between these components. The communication interface 83 may include a display screen (Display), a keyboard (Keyboard), and the optional communication interface 83 may also include a standard wired interface and a wireless interface. The memory 84 may be a high-speed RAM memory (Random Access Memory, volatile random access memory) or a non-volatile memory (non-volatile memory), such as at least one disk memory. The memory 84 may optionally be at least one storage device located away from the aforementioned processor 81. The processor 81 may be combined with Figure 9 or Figure 10 In the described apparatus, the memory 84 stores an application program, and the processor 81 calls the program code stored in the memory 84 to execute any of the above method steps.

[0274] The communication bus 82 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The communication bus 82 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 11 Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0275] The memory 84 may include volatile memory, such as random-access memory (RAM); the memory may also include non-volatile memory, such as flash memory, hard disk drive (HDD) or solid-state drive (SSD); the memory 84 may also include a combination of the above types of memory.

[0276] The processor 81 may be a central processing unit (CPU), a network processor (NP), or a combination of a CPU and a NP.

[0277] The processor 81 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The PLD may be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0278] Optionally, the memory 84 is also used to store program instructions. The processor 81 can call the program instructions to implement the present application. Figures 1 to 5 The data processing method shown in the embodiment, or Figures 6 to 8 The data query method shown in the embodiment.

[0279] An embodiment of the present invention further provides a non-transitory computer storage medium, wherein the computer storage medium stores computer-executable instructions, and the computer-executable instructions can execute the data processing method or data query method in any of the above method embodiments. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory (Flash Memory), a hard disk drive (HDD) or a solid-state drive (SSD), etc.; the storage medium can also include a combination of the above types of memory.

[0280] Although the embodiments of the present invention have been described with reference to the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention. Such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A data processing method, characterized in that: include: Acquire model data to be processed, where the model data to be processed is data in a BIM model; Analyzing the model data to be processed to determine the attribute type of each attribute in the model data to be processed, the attribute types including general attributes, non-compressed attributes, and compressed attributes, wherein no compression is required for the general attributes, the attribute names are compressed for the non-compressed attributes, and the attribute names and attribute values ​​are compressed for the compressed attributes; Compressing the model data to be processed based on the attribute type, and determining target data corresponding to the model data to be processed; Wherein, the attribute type includes a compression attribute, and compressing the model data to be processed based on the attribute type and determining the target data corresponding to the model data to be processed includes: Determine attribute index values ​​using the names of the attributes in the model data to be processed; Counting the attribute values ​​of all the compression attributes; Using a unique attribute value index value corresponding to each of the attribute values, a compression dictionary is constructed to determine the attribute value index value of the compression attribute; The target data is determined based on the attribute index value and the attribute value index value.

2. The method according to claim 1, characterized in that The generating of attribute index values ​​by using each attribute name in the model data to be processed includes: Counting the attribute names in the model data to be processed; The attribute names are sorted in a preset order, and an attribute index table is constructed to determine the attribute index value, where the attribute index value is consistent with the sequence number of the attribute name in the attribute index table.

3. The method according to claim 2, characterized in that The determining the target data based on the attribute index value and the attribute value index value includes: Deleting the attribute name of each piece of data in the to-be-processed model data, and sorting the deleted data according to the order of the attribute names in the attribute index table; The compression dictionary is used to replace the attribute value of the compression attribute in the sorted data with the corresponding attribute value index value to obtain compressed data, so as to determine the target data.

4. The method according to claim 1, wherein The compressing the model data to be processed based on the attribute type and determining the target data corresponding to the model data to be processed further includes: Get grouping requirements; Based on the grouping requirement, the compressed data is grouped to determine the target data.

5. The method according to claim 4, characterized in that The step of grouping the compressed data based on the grouping requirement to determine the target data includes: Based on the grouping requirements, determine the attributes for establishing the query index; Determine whether the attribute of the query index exists in the attribute index table; When the attribute of the query index does not exist in the attribute index table, creating an extended attribute to create the query index; The compressed data is grouped based on the query index to determine the target data.

6. The method according to claim 1, wherein The method further comprises: Serialize the target data and write it into a corresponding file to obtain a target file; The target file is compressed and saved to a preset file system.

7. The method according to claim 1, characterized in that The method further comprises: Get query conditions; A query is performed in the target data based on the query condition and the query index to determine a query result.

8. A data query method, characterized in that: include: Get query conditions; performing a query in target data based on the query condition and determining a query result, wherein the target data is obtained by compressing the model data to be processed based on the attribute type of each attribute in the model data to be processed, the attribute types including general attributes, non-compressed attributes, and compressed attributes, wherein no compression is required for the general attributes, the attribute names for the non-compressed attributes are compressed, and the attribute names and attribute values ​​for the compressed attributes are compressed, and the model data to be processed is data in a BIM model; The querying of the target data based on the query condition and determining the query result includes: Obtaining a query index and a compression dictionary, wherein the compression dictionary is used to represent a correspondence between attribute values ​​and attribute value index values; Determine corresponding data groups and corresponding data files using the query condition and the query index; Based on the compression dictionary and the query condition, the target data in the data file is matched to determine the query result.

9. The method according to claim 8, characterized in that The matching the target data in the data file based on the compression dictionary and the query condition to determine the query result includes: Preprocessing the query condition using the compression dictionary to obtain a processed query condition; Obtaining an attribute index value from the processed query condition; Determine the position of the attribute to be matched in the target data using the attribute index value, and extract the value of the corresponding position; The extracted value is processed based on the attribute type of the attribute corresponding to the attribute index value, and matched with the value in the query condition to determine the query result.

10. The method according to claim 9, characterized in that The restoring the extracted value to the original value based on the attribute type of the attribute corresponding to the attribute index value to determine the query result includes: Determine whether the query condition is a fuzzy match; When the query condition is the fuzzy match, a fuzzy match is performed based on the original value and the value in the processed query condition to determine the query result.

11. A data processing device, characterized in that: include: A first acquisition module is used to acquire model data to be processed, where the model data to be processed is data in a BIM model; an analysis module, configured to analyze the model data to be processed and determine the attribute type of each attribute in the model data to be processed, wherein the attribute type includes a general attribute, an uncompressed attribute, and a compressed attribute. For the general attribute, no compression is required. For the uncompressed attribute, the attribute name is compressed. For the compressed attribute, the attribute name and attribute value are compressed. a compression module, configured to compress the model data to be processed based on the attribute type, and determine target data corresponding to the model data to be processed; Wherein, the attribute type includes a compression attribute, and compressing the model data to be processed based on the attribute type and determining the target data corresponding to the model data to be processed includes: Determine attribute index values ​​using the names of the attributes in the model data to be processed; Counting the attribute values ​​of all the compression attributes; Using a unique attribute value index value corresponding to each of the attribute values, a compression dictionary is constructed to determine the attribute value index value of the compression attribute; The target data is determined based on the attribute index value and the attribute value index value.

12. A data query device, characterized in that: include: The second acquisition module is used to obtain query conditions; a query module, configured to query target data based on the query condition and determine a query result, wherein the target data is obtained by compressing the model data to be processed based on the attribute type of each attribute in the model data to be processed, wherein the attribute types include general attributes, non-compressed attributes, and compressed attributes, wherein no compression is required for the general attributes, the attribute name is compressed for the non-compressed attributes, and the attribute name and attribute value are compressed for the compressed attributes, and the model data to be processed is data in a BIM model; The querying of the target data based on the query condition and determining the query result includes: Obtaining a query index and a compression dictionary, wherein the compression dictionary is used to represent a correspondence between attribute values ​​and attribute value index values; Determine corresponding data groups and corresponding data files using the query condition and the query index; Based on the compression dictionary and the query condition, the target data in the data file is matched to determine the query result.

13. An electronic device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the data processing method according to any one of claims 1 to 7, or the data query method according to any one of claims 8 to 10, by executing the computer instructions.

14. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the data processing method according to any one of claims 1 to 7, or to execute the data query method according to any one of claims 8 to 10.

Images