Noise monitoring data management system based on ShardingJDBC dynamic database building and table building

Abstract

The invention relates to the technical field of noise monitoring data management, in particular to a noise monitoring data management system based on ShardingJDBC dynamic database building and table building. The system comprises a traversal storage distribution module, a traversal path storage module and a dynamic library and table building module. The traversal path in each noise monitoring data storage process is obtained through the traversal path storage module, the occurrence probability of each traversal path is calculated, when the memory of the current storage database is insufficient, the database needs to be built again, and the probability of occurrence of each traversal path is calculated. The dynamic database and table building module is combined with the occurrence probability of each traversal path and the corresponding noise monitoring data source project to obtain the association between the traversal path and the noise monitoring data source project, and a new database or data table is built according to the association; namely, the traversal sequence of the storage items of the storage database is rearranged according to the occurrence rate, so that the traversal time is shortened, and the data storage efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of noise monitoring data management, and in particular to a noise monitoring data management system based on ShardingJDBC dynamic database and table construction. Background Art

[0002] Noise monitoring data mainly include: Noise source monitoring data: used to evaluate the noise emission levels of specific noise sources (such as factories, construction sites, entertainment venues, etc.) to ensure that they meet relevant environmental standards; Regional environmental noise monitoring data: used to assess the overall noise level in a specific area, usually monitored by grid method or fixed-point measurement method; Road traffic noise monitoring data: used to assess the impact of road traffic noise on the surrounding environment and provide a basis for traffic management and urban planning; Noise monitoring data of various functional areas in the city: used to evaluate the noise levels of different functional areas (such as residential areas, cultural and educational areas, commercial areas, industrial areas, etc.) to ensure the quality of life and health of residents.

[0003] As society continues to progress and the trend of industrialization increases, more and more new types of noise are constantly generated. In order to deal with noise, most cities now deploy a large number of noise monitoring equipment in specific areas to obtain corresponding noise monitoring data.

[0004] The existing database for storing noise monitoring data mainly adopts ShardingJDBC to build the database, and establishes partition storage libraries according to the noise monitoring data to store different types of noise monitoring data. During the storage process, the partition storage library is traversed according to the storage content of the noise monitoring data to match the corresponding storage location. However, with the rapid expansion of the scale of noise monitoring data, the corresponding partition storage libraries are continuously expanded and increased. During the traversal process, most of the noise monitoring data types in the same area are the same. If the database can be expanded and increased according to its traversal process, the traversal time can be greatly reduced, the traversal path can be reduced, and the storage location matching efficiency can be improved.

[0005] In order to deal with the above problems, there is an urgent need for a noise monitoring data management system based on ShardingJDBC dynamic database and table construction. Summary of the invention

[0006] The purpose of the present invention is to provide a noise monitoring data management system based on ShardingJDBC dynamic database and table construction to solve the problems raised in the above background technology.

[0007] To achieve the above purpose, a noise monitoring data management system based on ShardingJDBC dynamic database and table construction is provided, including a noise monitoring data acquisition module and a noise monitoring data classification processing module. The noise monitoring data acquisition module is used to collect noise data acquired by different devices. The noise monitoring data classification processing module combines the acquired noise data and classifies different noise monitoring data according to the source project of the noise monitoring data. It also includes a dynamic sharding rule configuration module, a traversal storage allocation module, a traversal path storage module and a dynamic database and table construction module. The dynamic sharding rule configuration module establishes a partition storage database in ShardingJDBC according to the noise monitoring data source project, and marks the storage project of each partition storage database; The traversal storage allocation module traverses each partition storage database according to the source items of the collected noise monitoring data to locate the noise monitoring data storage location; The traversal path storage module is used to obtain the traversal path in each noise monitoring data storage process and calculate the occurrence probability of each traversal path; The dynamic database and table building module combines the occurrence probability of each traversal path and the corresponding noise monitoring data source item, obtains the association between the traversal path and the noise monitoring data source item, and establishes a new database or data according to the association.

[0008] As a further improvement of the technical solution, the noise monitoring data source items in the noise monitoring data classification processing module include equipment number, monitoring time, noise intensity, data memory and monitoring area; Wherein, the equipment number is the equipment number of the noise monitoring instrument; The monitoring time is the time taken by the noise monitoring instrument to obtain the noise monitoring data; The noise intensity is the noise intensity range in the noise monitoring data; The data memory is the memory size occupied by the noise monitoring data acquired by the noise monitoring instrument; The monitoring area is the monitoring coverage of the noise monitoring instrument.

[0009] As a further improvement of the technical solution, the dynamic sharding rule configuration module establishes a partition storage database in ShardingJDBC and adopts the dynamic sharding rule configuration of ShardingJDBC.

[0010] As a further improvement of the technical solution, the configuration method of the dynamic sharding rule configuration of ShardingJDBC includes the following steps: S301, establish a main and secondary area coverage relationship strategy, and divide the storage items of the main area and the secondary area according to the source items of noise monitoring data; S302: Based on the specific values ​​corresponding to the source items of the noise monitoring data, the traversal order of the storage items in each main area and sub-area is divided.

[0011] As a further improvement of the technical solution, the method for traversing the storage allocation module to locate the storage location of noise monitoring data includes the following steps: S401, disassemble the specific values ​​corresponding to the source items of the noise monitoring data to obtain the values ​​of the single source items; S402, comparing the matched single source item values ​​according to the main area traversal order divided in S302, and obtaining the main area storage item to which the current single source item value belongs; S403, according to the sub-area traversal order and the sub-area storage items in the main area storage items that have been matched, locate the position of the value of each single source item.

[0012] As a further improvement of the present technical solution, the traversal path storage module includes a unit path division unit and a path occurrence rate calculation unit. The unit path division unit combines each noise monitoring data storage path and divides the unit storage path according to the matching storage items. The path occurrence rate calculation unit is used to calculate the occurrence rate of each unit storage path.

[0013] As a further improvement of the technical solution, the method for calculating the occurrence rate of each unit storage path in the path occurrence rate calculation unit includes the following steps: S5201. Set unit statistical time and obtain the total number of unit storage paths counted within the unit statistical time. ; S5202: Obtain the occurrence of each unit storage path within the unit statistical time , calculate the corresponding occurrence rate = occurrence amount / Total number of unit storage paths ; S5203. Sequentially divide each unit storage path according to the occurrence rate.

[0014] Compared with the prior art, the present invention has the following beneficial effects: In the noise monitoring data management system based on ShardingJDBC dynamic database and table construction, the traversal path in each noise monitoring data storage process is obtained through the traversal path storage module, and the occurrence probability of each traversal path is calculated. When the current storage database memory is insufficient, it is necessary to rebuild the database. The dynamic database and table construction module is combined with the occurrence probability of each traversal path and the corresponding noise monitoring data source project to obtain the association between the traversal path and the noise monitoring data source project, and establish a new database or data table according to the association, that is, the traversal order of the storage database storage items is rearranged according to the occurrence rate, which reduces the time spent on traversal and improves data storage efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is the overall structural diagram of the present invention; Figure 2 It is a structural block diagram of the traversal path storage module of the present invention; Figure 3 A step diagram of a configuration method for dynamic sharding rule configuration of ShardingJDBC of the present invention; Figure 4 A diagram showing the steps of a method for locating a noise monitoring data storage location according to the present invention; Figure 5 This is a step diagram of a method for calculating the occurrence rate of each unit storage path according to the present invention.

[0016] The meaning of each number in the figure is: 10. Noise monitoring data acquisition module; 20. Noise monitoring data classification processing module; 30. Dynamic sharding rule configuration module; 40. Traverse the storage allocation module; 50. traversal path storage module; 510. unit path division unit; 520. path occurrence rate calculation unit; 60. Dynamic database and table creation module. DETAILED DESCRIPTION

[0017] The following will be combined with the accompanying drawings in the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0018] See also Figure 1As shown, a noise monitoring data management system based on ShardingJDBC dynamic database and table construction is provided, including a noise monitoring data acquisition module 10 and a noise monitoring data classification processing module 20. The noise monitoring data acquisition module 10 is used to collect noise data acquired by different devices. The noise monitoring data classification processing module 20 combines the acquired noise data and classifies and processes different noise monitoring data according to the source items of the noise monitoring data. It also includes a dynamic sharding rule configuration module 30, a traversal storage allocation module 40, a traversal path storage module 50 and a dynamic database and table construction module 60. The dynamic sharding rule configuration module 30 establishes a partition storage database in ShardingJDBC according to the noise monitoring data source project, and marks each partition storage database storage project; The traversal storage allocation module 40 traverses each partition storage database according to the source items of the collected noise monitoring data to locate the noise monitoring data storage location; The traversal path storage module 50 is used to obtain the traversal path in each noise monitoring data storage process and calculate the occurrence probability of each traversal path; The dynamic database and table building module 60 combines the occurrence probability of each traversal path and the corresponding noise monitoring data source item, obtains the association between the traversal path and the noise monitoring data source item, and builds a new database or data table according to the association.

[0019] When used specifically, in the process of noise monitoring data management, the noise monitoring data acquisition module 10 is first required to collect noise data obtained by different devices, and the noise monitoring data classification processing module 20 combines the obtained noise data, according to the source items of the noise monitoring data, such as the equipment number, because different devices are located in different locations, the corresponding monitoring data will be different, so in order to facilitate classification and storage, different noise monitoring data need to be classified and processed; In order to adapt to different types of noise monitoring data storage work, it is necessary to establish a partition storage database in ShardingJDBC according to the noise monitoring data source project through the dynamic sharding rule configuration module 30, and mark the storage project of each partition storage database. For example, different partition storage databases are divided according to the device number. During the storage process, the device number of the noise monitoring data will be identified for adaptive storage; Because when performing specific data storage, it is necessary to identify and process the noise monitoring data from different sources, and traverse the storage allocation module 40 according to the source items of the collected noise monitoring data to traverse each partition storage database, locate the noise monitoring data storage location, that is, match it with the storage items of the storage database in different zones, and store the noise monitoring data in the successfully matched storage database. In this process, when the storage database is initially established, the corresponding storage items are randomly distributed, and the matching order during matching is fixed, it is easy to have repeated traversal paths. For example, the storage database includes five partitions, namely A, B, C, D and E. During the traversal process, in the current monitoring area, the corresponding traversal order is ABCDE. In the actual matching process, the noise monitoring data is stored in the D partition in most cases. Therefore, in the matching process, the three partitions ABC must be traversed each time before the matching work is completed. However, if the traversal order of the D partition can be advanced to directly complete the matching work, the traversal time can be reduced.

[0020] In order to address the above problems, the traversal path in each noise monitoring data storage process is obtained through the traversal path storage module 50, and the occurrence probability of each traversal path is calculated. When the current storage database memory is insufficient, it is necessary to rebuild the database. Since the source of the stored noise monitoring data is consistent, when rebuilding the database, the dynamic database and table construction module 60 is used to combine the occurrence probability of each traversal path and the corresponding noise monitoring data source item to obtain the association between the traversal path and the noise monitoring data source item, and establish a new database or data table according to the association, that is, the traversal order of the storage database storage items is rearranged according to the occurrence rate, thereby reducing the time spent on traversal and improving data storage efficiency.

[0021] In addition, the noise monitoring data source items in the noise monitoring data classification processing module 20 include equipment number, monitoring time, noise intensity, data memory and monitoring area; Among them, the equipment number is the equipment number of the noise monitoring instrument; Monitoring time refers to the time it takes for the noise monitoring instrument to obtain noise monitoring data; Noise intensity is the noise intensity range in the noise monitoring data, expressed in weighted sound level, measured in dBA. The weighted sound level is calculated based on the sensitivity of the human ear to sounds of different frequencies, and can better reflect the noise intensity felt by the human ear; Data memory is the memory size occupied by the noise monitoring data obtained by the noise monitoring instrument; The monitoring area is the monitoring coverage of the noise monitoring instrument.

[0022] When used specifically, in the process of configuring different storage database storage items in the storage database, the device number is the main storage item, and the corresponding monitoring time, noise intensity, data memory and monitoring area are the secondary storage items of the device number. That is, after completing the matching of the device number, the matching of the monitoring area is performed, followed by the matching of the noise intensity, and then the matching of the monitoring time, and finally the matching of the monitoring data memory. For example, there are three sets of device numbers. as well as , the traversal order is - - , the device number of the currently matched noise monitoring data is , that is, the matching of the main area storage items is completed, and the device number is The monitoring area of ​​the noise monitoring instrument includes as well as There are four regions, and the corresponding traversal order is - - - , the monitoring area of ​​the currently matched noise monitoring data is , corresponding to The monitoring areas include , , as well as There are four groups of noise intensities, and the corresponding traversal order is - - - , the noise intensity of the currently matched noise monitoring data belongs to Range, corresponding to The noise intensity range includes as well as Four monitoring times, the corresponding traversal order is , the monitoring time of the currently matched noise monitoring data is , corresponding to Monitoring time includes , , as well as There are four sets of monitoring data memory, and the corresponding traversal order is - - - , the data memory of the currently matched noise monitoring data belongs to , complete the entire traversal work, and the final matching path is - - - - .

[0023] Furthermore, the dynamic sharding rule configuration module 30 establishes a partitioned storage database in ShardingJDBC and adopts the dynamic sharding rule configuration of ShardingJDBC to horizontally split the noise data according to data characteristics such as the source project of the noise monitoring data. The number of shards can be easily adjusted according to the data growth situation, ensuring the flexibility and scalability of the system when the data scale expands.

[0024] Further, if Figure 3 As shown in the figure, the configuration method of ShardingJDBC's dynamic sharding rule configuration includes the following steps: S301, establish a main and secondary area coverage relationship strategy, and divide the storage items of the main area and the secondary area according to the source items of noise monitoring data; S302: Based on the specific values ​​corresponding to the source items of the noise monitoring data, the traversal order of the storage items in each main area and sub-area is divided.

[0025] Specifically, Figure 4 As shown, the method for traversing the storage allocation module 40 to locate the noise monitoring data storage location includes the following steps: S401, disassemble the specific values ​​corresponding to the source items of the noise monitoring data to obtain the values ​​of the single source items; S402, comparing the matched single source item values ​​according to the main area traversal order divided in S302, and obtaining the main area storage item to which the current single source item value belongs; S403, according to the sub-area traversal order and the sub-area storage items in the main area storage items that have been matched, locate the position of the value of each single source item.

[0026] In specific use, in the process of locating the storage location of noise monitoring data, due to the small number of samples of noise monitoring data obtained during the initial storage management, it is impossible to select an adaptive traversal path. At this time, in order to unify the management of storage, when establishing the initial storage database, by establishing a main-sub-area coverage relationship strategy, the storage items of the main area and the sub-area are divided in combination with the noise monitoring data source items, and the initial storage database is established in combination with the reference data in the industry, and the main area and the sub-area are divided according to the noise monitoring data source items. This needs to be determined according to the monitoring scope. For example, if the monitoring environment is complex and the corresponding noise intensity range is wide, the amount of monitoring equipment will increase, and the corresponding main area storage items will also increase accordingly. At the same time, the noise intensity storage items in the sub-area will also increase. Finally, combined with the specific values ​​corresponding to the noise monitoring data source items, the traversal order of each main area and sub-area storage item is divided, which is the initial traversal order; After completing the planning of the initial traversal sequence, the next step is to store and locate the collected noise monitoring data. First, in order to facilitate data comparison, it is necessary to disassemble the specific values ​​corresponding to the source items of the noise monitoring data, and obtain the values ​​of single source items, that is, disassemble them into equipment number, monitoring time, noise intensity, data memory and monitoring area, and obtain the corresponding values. According to the divided main area traversal order, the matching single source item values ​​are compared to obtain the main area storage item to which the current single source item value belongs. Finally, according to the sub-area traversal order and the sub-area storage items in the main area storage items that have been matched, the location of each single source item value is located.

[0027] In addition, if Figure 2 As shown, the traversal path storage module 50 includes a unit path division unit 510 and a path occurrence rate calculation unit 520. The unit path division unit 510 combines each noise monitoring data storage path and divides the unit storage path according to the matching storage items. The path occurrence rate calculation unit 520 is used to calculate the occurrence rate of each unit storage path.

[0028] Further, such as Figure 5 As shown, the method for calculating the occurrence rate of each unit storage path in the path occurrence rate calculation unit 520 includes the following steps: S5201. Set unit statistical time and obtain the total number of unit storage paths counted within the unit statistical time. ; S5202: Obtain the occurrence of each unit storage path within the unit statistical time , calculate the corresponding occurrence rate = occurrence amount / Total number of unit storage paths ; S5203. Sequentially divide each unit storage path according to the occurrence rate.

[0029] In specific use, in order to establish an adaptive storage database later, the traversal order of each storage item is reasonably arranged according to the storage path. First, the unit path division unit 510 is combined with each noise monitoring data storage path, and the unit storage path is divided according to the matching storage item, and the occurrence rate of each unit storage path is calculated by the path occurrence rate calculation unit 520, that is, the unit statistical time is formulated, and the total number of unit storage paths counted within the unit statistical time is obtained. , which is the total amount of noise monitoring data stored, and then the occurrence of each unit storage path is obtained within the unit statistical time , calculate the corresponding appearance rate. The higher the appearance rate, the better the adaptation effect of the unit storage path. When used as the initial traversal order of the new storage database, the corresponding traversal path will be shortened, and the corresponding storage matching time will be reduced. Finally, the unit storage paths are divided in order according to the appearance rate, which serves as a reference for selecting the initial traversal order later.

[0030] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments. The above embodiments and descriptions are only preferred examples of the present invention and are not intended to limit the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A noise monitoring data management system based on ShardingJDBC dynamic database and table construction, comprising a noise monitoring data acquisition module (10) and a noise monitoring data classification processing module (20), wherein the noise monitoring data acquisition module (10) is used to collect noise data acquired by different devices, and the noise monitoring data classification processing module (20) combines the acquired noise data and classifies and processes different noise monitoring data according to the source items of the noise monitoring data, and is characterized in that: It also includes a dynamic sharding rule configuration module (30), a traversal storage allocation module (40), a traversal path storage module (50), and a dynamic database and table building module (60); The dynamic sharding rule configuration module (30) establishes a partition storage database in ShardingJDBC according to the noise monitoring data source project, and marks the storage project of each partition storage database; The traversal storage allocation module (40) traverses each partition storage database according to the source items of the collected noise monitoring data to locate the noise monitoring data storage location; The traversal path storage module (50) is used to obtain the traversal path in each noise monitoring data storage process, and calculate the occurrence probability of each traversal path; The dynamic database and table building module (60) combines the occurrence probability of each traversal path and the corresponding noise monitoring data source item to obtain the association between the traversal path and the noise monitoring data source item, and builds a new database or data table according to the association.

2. The noise monitoring data management system based on ShardingJDBC dynamic database and table construction according to claim 1 is characterized in that: The noise monitoring data source items in the noise monitoring data classification processing module (20) include equipment number, monitoring time, noise intensity, data memory and monitoring area; Wherein, the equipment number is the equipment number of the noise monitoring instrument; The monitoring time is the time taken by the noise monitoring instrument to obtain the noise monitoring data; The noise intensity is the noise intensity range in the noise monitoring data; The data memory is the memory size occupied by the noise monitoring data acquired by the noise monitoring instrument; The monitoring area is the monitoring coverage of the noise monitoring instrument.

3. The noise monitoring data management system based on ShardingJDBC dynamic database and table construction according to claim 1 is characterized in that: The dynamic sharding rule configuration module (30) establishes a partition storage database in ShardingJDBC and adopts the dynamic sharding rule configuration of ShardingJDBC.

4. The noise monitoring data management system based on ShardingJDBC dynamic database and table construction according to claim 3 is characterized in that: The configuration method of the dynamic sharding rule configuration of ShardingJDBC includes the following steps: S301, establish a main and secondary area coverage relationship strategy, and divide the storage items of the main area and the secondary area according to the source items of noise monitoring data; S302: Based on the specific values ​​corresponding to the source items of the noise monitoring data, the traversal order of the storage items in each main area and sub-area is divided.

5. The noise monitoring data management system based on ShardingJDBC dynamic database and table construction according to claim 4 is characterized in that: The method for traversing the storage allocation module (40) and locating the noise monitoring data storage location comprises the following steps: S401, disassemble the specific values ​​corresponding to the source items of the noise monitoring data to obtain the values ​​of the single source items; S402, comparing the matched single source item values ​​according to the main area traversal order divided in S302, and obtaining the main area storage item to which the current single source item value belongs; S403, according to the sub-area traversal order and the sub-area storage items in the main area storage items that have been matched, locate the position of the value of each single source item.

6. The noise monitoring data management system based on ShardingJDBC dynamic database and table construction according to claim 1 is characterized in that: The traversal path storage module (50) comprises a unit path division unit (510) and a path occurrence rate calculation unit (520), wherein the unit path division unit (510) combines each noise monitoring data storage path and divides the unit storage path according to the matching storage items, and the path occurrence rate calculation unit (520) is used to calculate the occurrence rate of each unit storage path.

7. The noise monitoring data management system based on ShardingJDBC dynamic database and table construction according to claim 6 is characterized in that: The method for calculating the occurrence rate of each unit storage path in the path occurrence rate calculation unit (520) comprises the following steps: S5201. Set unit statistical time and obtain the total number of unit storage paths counted within the unit statistical time. ; S5202: Obtain the occurrence of each unit storage path within the unit statistical time , calculate the corresponding occurrence rate = occurrence amount / Total number of unit storage paths ; S5203. Sequentially divide each unit storage path according to the occurrence rate.

Images