A live load variation interval data acquisition method based on text big data

Abstract

The present application provides a live load variation interval data acquisition method based on text big data, which replaces the artificial investigation paradigm by mining a large amount of business text data to obtain a large amount of live load variation interval data, and strives to solve the problems of load data acquisition difficulty and research stagnation. The present application comprises the following steps: step S1, text big data collection; step S2, data quality inspection; step S3, intelligent address screening; step S4, address change verification; and step S5, load variation interval calculation.

Description

Technical Field

[0001] This invention relates to the field of building structure technology, and in particular to a method for collecting live load variation interval data based on text big data. Background Technology

[0002] Loads change over time, and understanding their time-varying patterns and obtaining the probability distribution of the maximum load value within a given time period is crucial for determining the standard load value. In time-varying live load models, the time interval between adjacent load changes is a critical parameter. A single live load change can be understood as the current user moving out of a building area, followed by the next user moving in. Therefore, the time interval between two adjacent load changes also refers to the duration of the current user's continuous occupation of that building area.

[0003] To understand the randomness of live load variation intervals, load collection is necessary to gather sufficient sample data. Currently, the commonly used collection methods mainly include three types: consulting telephone directories, questionnaires, and oral inquiries. However, consulting telephone directories may miss short load variation interval samples, while questionnaires and oral inquiries rely on the subjective memory of the respondents, resulting in low reliability of the collected load variation interval samples. In addition, these collection methods all rely on manual labor, making the collection work costly and inefficient. The time-consuming and laborious load collection leads to the following two problems in terms of data for time-varying model research: (1) First, the sample size is small. For example, the number of variation interval samples collected in previous studies is concentrated in the range of 100-1000; (2) Second, the data updates are very slow. For example, the live load variation interval data in some current domestic and foreign load codes still mainly come from load collection in the 1970s.

[0004] Inevitably, the scarcity of research data further limits the development and improvement of time-varying live load models. For example, the time-varying live load model has remained unchanged for nearly fifty years. This naturally raises questions such as: do the current patterns of live load variation still conform to the preconditions of the existing model, and if so, have the model parameters changed? In fact, due to the difficulty in conducting load data acquisition, the parameters of existing models have not been updated regularly, let alone the development of new models. However, the widespread adoption and application of big data and related technologies, such as internet big data and natural language processing, have brought new opportunities for live load research. Therefore, it is necessary to develop a new, low-cost, sustainable, and efficient data acquisition method to obtain large amounts of live load variation interval data, thereby promoting the updating of the data foundation and time-varying models. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of traditional manual load variation interval data collection methods and provide a live load variation interval data collection method based on text big data. By mining a large amount of industrial and commercial text data to replace the manual survey paradigm, a large amount of live load variation interval data can be obtained, aiming to solve the problems of difficulty in load data collection and research stagnation.

[0006] The objective of this invention can be achieved through the following technical solution: a method for acquiring live load variation interval data based on text big data, comprising the following steps: Step S1: Text Big Data Collection: Collect publicly available business registration information and change records of different market entities from the Internet to form text big data; Step S2, Data Quality Check: For the large amount of text data from Step S1, clean the registration information and change record information to remove missing or abnormal data, and filter out the text data containing address change records. Step S3, Intelligent Address Filtering: Utilize natural language processing technology to establish an intelligent address filtering model, automatically classify Chinese addresses in the text data from Step S2, and mine the text data of market entities located in the target building type; Step S4, Address Change Verification: For the market entity text data from Step S3, verify whether each address change record corresponds to a real relocation, in order to exclude address change records that have not undergone actual spatial location changes, and to ensure that the load in the building has changed. Step S5, Load Variation Interval Calculation: For the text data verified in Step S4, the business registration information and valid address change records of the same market entity are associated, and an address occupancy sequence is constructed according to the chronological order of the registration date and the valid address change date. The registration date or the previous valid address change date is taken as the move-in time node of the corresponding address, and the subsequent valid address change date is taken as the move-out time node of the corresponding address. The continuous occupancy time of the market entity at the corresponding address is calculated based on the time difference between the move-out time node and the move-in time node. The continuous occupancy time is taken as the load variation interval sample data to complete the load variation interval collection.

[0007] Step S1 of the present invention specifically involves: collecting publicly available business registration information and change record information of different market entities from the Internet. The business registration information includes the market entity name, market entity type, registration date, city, registered capital, business location and industry type, and includes at least one of the following: unified social credit code, registration number or other information that can distinguish the same market entity; the change record information includes the change item, change content and change date. The two parts of data together constitute text big data.

[0008] In step S3 of this invention, the intelligent address filtering model comprises three parts: an input layer, an encoder, and an output layer. The input layer converts Chinese addresses into input vectors. The encoder consists of several stacked encoding layers that extract features from the input vectors based on a self-attention mechanism for subsequent address classification. The output layer consists of a classification layer that outputs the category labels of the Chinese addresses.

[0009] In step S3 of this invention, the encoding layer consists of two residual structures. The first residual structure includes a multi-head attention layer and a normalization layer, and the second residual structure includes a feedforward network layer and a normalization layer.

[0010] In step S4 of this invention, when verifying the validity of the address change, text similarity discrimination and geocoding discrimination are performed on the address before and after the change, respectively, to exclude address change records that have not actually undergone spatial location changes; wherein, text similarity discrimination is used to identify address information modification records, and geocoding discrimination is used to identify address change records where the text description has changed due to changes in the address description method, but the actual spatial location has not changed.

[0011] Compared with the prior art, the present invention has the following advantages: (1) No manual on-site investigation is required, which significantly reduces the cost of manual investigation and improves the efficiency of load investigation; (2) The obtained load variation interval sample data is large in volume, highly reliable and has a wide coverage, which can provide a more realistic data basis for the fine formulation of load standard values ​​in different regions of the country; (3) It has strong sustainability and can update the statistical data of load variation intervals in a timely manner to keep up with the times, so that the load time-varying model can keep pace with the times; (4) It has strong scalability and can be extended to the investigation of load variation interval samples of other building types, such as shopping malls, industrial plants and other building types. Attached Figure Description

[0012] Figure 1 This is a flowchart illustrating an embodiment of the present invention; Figure 2 This is a network structure diagram of the intelligent address filtering model in an embodiment of the present invention; Figure 3 This is a detailed structural diagram of the coding layer in the intelligent address filtering model of this invention. Figure 4 This is a graph showing the quantity and proportion of valid business registration data in each province according to an embodiment of the present invention; Figure 5 This is a data distribution diagram of the live load variation interval of an office building according to an embodiment of the present invention. Detailed Implementation

[0013] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. These embodiments are based on the technical solution of the present invention and provide detailed implementation methods and specific operating procedures. However, the scope of protection of the present invention is not limited to the following embodiments.

[0014] like Figure 1 As shown, a method for collecting live load variation interval data based on text big data is described, the method comprising: Step S1: Text Big Data Collection: Collect publicly available business registration information and change records of different market entities from the Internet to form text big data.

[0015] The specific process is as follows: collect publicly available business registration information of different market entities from the Internet, including the market entity name, market entity type, registration date, city, registered capital, business location and industry type, and at least one of the following: unified social credit code, registration number or other information that can distinguish the same market entity; and change record information, including change items, change content and change date, etc. The two parts of data together constitute text big data. The collection method can be to use web crawling technology or to download in batches from websites containing the above information.

[0016] Step S2: Check the quality of the large text data in the database. First, clean the text data in the registration information and change record information where the keys and values ​​are missing or abnormal. Then, filter out the text data in the change record that contains address changes.

[0017] Step S3: Establish an intelligent address filtering model based on natural language processing technology to automatically classify Chinese addresses in the text data from step S2 and mine the text data of market entities located in the target building type.

[0018] The network structure of the intelligent address filtering model is as follows: Figure 2 As shown, the system comprises an input layer, an encoder, and an output layer. The input layer converts Chinese characters into input vectors and adds the special characters [CLS] and [SEP] to the beginning and end positions of each address sentence, respectively, for address classification and address end-of-address identification. The output layer consists of classification layers; the number of categories in the classification layer is determined based on the target building type selection task. In this embodiment, a binary classification layer is used to determine whether a Chinese address belongs to an office building address. Its input is the output tensor of the special character [CLS] after processing by the encoder, and its output is the class label of the Chinese address. The encoder consists of 12 stacked encoding layers, whose function is to extract features from the input vector based on a self-attention mechanism for subsequent address classification. Figure 3 The detailed structure of the coding layer is given. Each coding layer consists of two residual structures, such as... Figure 3In part (a), the first residual structure contains a multi-head attention layer and a normalization layer, and the second residual structure contains a feedforward network layer and a normalization layer. The mathematical expression of the feedforward network layer can be written as: , In the formula, x is the input vector, W1 and W2 are learnable weight parameters, and b1 and b2 are learnable bias parameters. The mathematical expression of the normalization layer can be written as: , In the formula, x is the input vector, and μ x and σ x ε and β are the mean and standard deviation of the input vector, respectively, ε is a small constant, and γ and β are the learnable scale parameter and offset parameter, respectively.

[0019] The specific structure of the multi-head attention layer is as follows: Figure 3 Part (b) can be expressed mathematically as follows: , In the formula, Q, K, and V are the query vector, key vector, and value vector, respectively, all obtained from the input vector through linear mapping; Concat is the concatenation and fusion operation; W o For output mapping weight parameters; h i The result of the self-attention operation in the i-th subspace is as follows: Figure 3 Part (c) can be expressed mathematically as follows: , In the formula, d k Q is the dimension parameter of the key vector in the corresponding subspace. i K i and V i These are the results of mapping the query vector, key vector, and value vector to the i-th subspace, respectively. Let Q... i For example, its mapping process can be expressed as: , In the formula, Let be the mapping matrix that maps the query vector Q to the i-th subspace.

[0020] Step S4, Address Change Verification: For the market entity text data from Step S3, verify whether each address change record corresponds to a real relocation, in order to exclude address change records that have not undergone actual spatial location changes, and ensure that the load in the building has changed.

[0021] Some address change records fail to generate valid load change interval samples, mainly including two categories: address information modification and address description adjustment. For address information modification records, the text similarity between the address before and after the change is calculated. If the text similarity is greater than or equal to a preset similarity threshold, the address change record is determined not to reflect a real relocation and is not considered a valid load change record. For address description adjustment records, the address before and after the change are geocoded to obtain the corresponding spatial location information. If the spatial locations of the two are the same, or the spatial distance between them is less than or equal to a preset distance threshold, the address change record is determined to be only a change in address description method and does not represent a real change in spatial location, and is not considered a valid load change record. Address change records whose actual spatial location has been confirmed to have changed after the above criteria are included as valid address change records in subsequent load change interval calculations.

[0022] Step S5, Load Variation Interval Calculation: For the text data verified in Step S4, the business registration information and valid address change records of the same market entity are first linked. Specifically, the registration date, business location, valid address change date, address before change, and address after change of the same market entity can be collected based on the market entity name, unified social credit code, registration number, or other information that can distinguish the same market entity.

[0023] Then, the date field is standardized, and the address occupancy sequence of the market entity is constructed according to the chronological order of the registration date and the effective address change date. For the first registered address, the registration date of the market entity is used as the move-in time node, and the first effective address change date is used as the move-out time node; for subsequent addresses, the previous effective address change date is used as the move-in time node, and the next effective address change date is used as the move-out time node.

[0024] Based on the time difference between the move-out and move-in times, the continuous occupancy time of the market entity at the corresponding address is calculated, and this continuous occupancy time is used as sample data for load variation intervals. Records with missing dates, abnormal date order, or move-out times earlier than or equal to move-in times are discarded. For the current address after the last address change, if no subsequent valid address change date is obtained, it is not included in the complete load variation interval sample, or it is marked as an unclosed interval. Finally, the load variation interval samples formed by each market entity are summarized to complete the load variation interval collection.

[0025] Specific Implementation Example: This example utilizes the above steps to collect data on live load variation intervals for office buildings nationwide. By mining nearly 190 million text data entries from 31 provincial-level administrative regions across the country, 2,163,541 sample data points on live load variation intervals for office buildings were obtained. The above data scale is merely an example in this embodiment and does not constitute a limitation on the data source, data scale, or applicable area of ​​this invention. Figure 4 The quantity and proportion of valid text data for each province are given. Figure 5 The data distribution of live load variation intervals for office buildings is presented. Compared with the sample sizes of load variation intervals obtained in recent domestic and international surveys, such as 499 samples in the United States (1976), 317 samples in China (1977), and 100 samples in India (1993), the data volume obtained in this survey is four orders of magnitude larger than that of previous studies due to the innovation of the method in this invention. This provides a comprehensive and realistic data foundation for the study of time-varying load models. Further classifying the live load variation interval data of office buildings by prefecture-level city and calculating their mean values ​​yields a distribution map of the national average live load variation intervals for office buildings. This helps to achieve refined determination of load standard values ​​in different regions.

[0026] Furthermore, it should be noted that the specific embodiments described in this specification are only used to explain the technical solutions of the present invention and are not intended to limit the scope of protection of the present invention. For those skilled in the art, equivalent substitutions, simple modifications, or combinations of adjustments made to data sources, data cleaning methods, address filtering model structures, address change verification methods, load change interval calculation methods, and statistical summarization methods without departing from the concept of the present invention should all fall within the scope of protection of the present invention.

Claims

1. A method for collecting live load variation interval data based on text big data, characterized in that: Includes the following steps: Step S1: Text Big Data Collection: Collect publicly available business registration information and change records of different market entities from the Internet to form text big data; Step S2, Data Quality Check: For the large amount of text data from Step S1, clean the registration information and change record information to remove missing or abnormal data, and filter out the text data containing address change records. Step S3, Intelligent Address Filtering: Utilize natural language processing technology to establish an intelligent address filtering model, automatically classify Chinese addresses in the text data from Step S2, and mine the text data of market entities located in the target building type; Step S4, Address Change Verification: For the market entity text data from Step S3, verify whether each address change record corresponds to a real relocation, in order to exclude address change records that have not undergone actual spatial location changes, and to ensure that the load in the building has changed. Step S5, Load Variation Interval Calculation: For the text data verified in Step S4, the business registration information and valid address change records of the same market entity are associated, and an address occupancy sequence is constructed according to the chronological order of the registration date and the valid address change date. The registration date or the previous valid address change date is taken as the move-in time node of the corresponding address, and the subsequent valid address change date is taken as the move-out time node of the corresponding address. The continuous occupancy time of the market entity at the corresponding address is calculated based on the time difference between the move-out time node and the move-in time node. The continuous occupancy time is taken as the load variation interval sample data to complete the load variation interval collection.

2. The method for collecting live load variation interval data based on text big data according to claim 1, characterized in that: Step S1 specifically involves collecting publicly available business registration information and change record information from different market entities on the Internet. The business registration information includes the market entity's name, market entity type, registration date, city, registered capital, business location, and industry type, and includes at least one of the following: unified social credit code, registration number, or other information that can distinguish the same market entity. The change record information includes the change item, change content, and change date. The two parts of data together constitute text big data.

3. The method for collecting live load variation interval data based on text big data according to claim 1, characterized in that: In step S3, the intelligent address filtering model comprises three parts: an input layer, an encoder, and an output layer. The input layer converts Chinese addresses into input vectors. The encoder consists of several stacked encoding layers that extract features from the input vectors based on a self-attention mechanism for subsequent address classification. The output layer consists of a classification layer that outputs the category labels of the Chinese addresses.

4. The method for collecting live load variation interval data based on text big data according to claim 3, characterized in that: In step S3, the encoding layer consists of two residual structures. The first residual structure includes a multi-head attention layer and a normalization layer, and the second residual structure includes a feedforward network layer and a normalization layer.

5. The method for collecting live load variation interval data based on text big data according to claim 4, characterized in that: The mathematical expression of the feedforward network layer is as follows: ， In the formula, x is the input vector, W1 and W2 are learnable weight parameters, and b1 and b2 are learnable bias parameters.

6. The method for collecting live load variation interval data based on text big data according to claim 4, characterized in that: The mathematical expression for the standardization layer is as follows: ， In the formula, x is the input vector, and μ x and σ x ε and β are the mean and standard deviation of the input vector, respectively, ε is a small constant, and γ and β are the learnable scale parameter and offset parameter, respectively.

7. The method for collecting live load variation interval data based on text big data according to claim 4, characterized in that: The mathematical expression for the multi-head attention layer is as follows: ， In the formula, Q, K, and V are the query vector, key vector, and value vector, respectively, all obtained from the input vector through linear mapping; Concat is the concatenation and fusion operation; W o For output mapping weight parameters; h i This represents the result of the self-attention operation in the i-th subspace.

8. The method for collecting live load variation interval data based on text big data according to claim 7, characterized in that: The h mentioned i The mathematical expression is: ， In the formula, d k Q is the dimension parameter of the key vector in the corresponding subspace. i K i and V i These are the results of mapping the query vector, key vector, and value vector to the i-th subspace, respectively.

9. The method for collecting live load variation interval data based on text big data according to claim 1, characterized in that: In step S4, the text similarity between the address before and after the change is calculated, and the addresses before and after the change are geocoded to obtain the corresponding spatial location information. When the text similarity is greater than or equal to a preset similarity threshold, or the spatial location information is the same, or the distance between the spatial location information is less than or equal to a preset distance threshold, it is determined that the address change record does not reflect the actual relocation, and it is cleaned as an invalid address change record.

Images