Intelligent traffic information acquisition method and system based on big data, medium and equipment

Abstract

The invention discloses a smart traffic information acquisition method, system, medium and equipment based on big data, and relates to the technical field of data acquisition. The method comprises the following steps: acquiring historical acquisition records of acquisition devices in a target traffic network, and grouping the acquisition devices based on the historical acquisition records to obtain a plurality of acquisition device groups; acquiring working state information of each acquisition device in the acquisition device group; determining complementary acquisition equipment groups of the acquisition equipment groups; and determining a data acquisition strategy of each acquisition equipment group based on the working state information of each acquisition equipment and the complementary acquisition equipment group. By implementing the technical scheme provided by the invention, the transmission timeliness of the traffic information is improved, and reasonable scheduling of the collected resources is realized.

Description

Technical Field

[0001] The present application relates to the field of data collection technology, and specifically to methods, systems, media and equipment for collecting smart traffic information based on big data. Background Art

[0002] With the acceleration of urbanization, urban traffic congestion is becoming increasingly prominent. As an important means to alleviate traffic congestion, the core of the intelligent transportation system lies in the accurate and timely collection of traffic information. At present, urban traffic information collection mainly relies on various collection devices distributed in the road network, such as vehicle detectors, video surveillance, electronic tags, etc.

[0003] In the prior art, traffic information collection equipment usually adopts a fixed collection strategy for data collection. Each collection device works independently and performs collection tasks according to the preset collection frequency and collection parameters. This fixed collection mode has certain limitations in practical applications. Especially in the case of drastic fluctuations in traffic flow or sudden changes in equipment load, a single collection device may have problems such as reduced data quality and insufficient collection capacity, resulting in low timeliness of traffic information transmission. Summary of the invention

[0004] This application provides a method, system, medium and equipment for collecting intelligent traffic information based on big data, which improves the timeliness of traffic information transmission and realizes the rational scheduling of collection resources.

[0005] In a first aspect, the present application provides a method for collecting intelligent traffic information based on big data, the method comprising: Acquire historical collection records of each collection device in the target traffic network, and group each of the collection devices based on the historical collection records to obtain multiple collection device groups; Obtaining working status information of each acquisition device in the acquisition device group; Determining a complementary collection device group for each of the collection device groups; Based on the working status information of each acquisition device and the complementary acquisition device group, a data acquisition strategy for each acquisition device group is determined.

[0006] By adopting the above technical solution, the collection equipment is grouped based on historical collection records, so that equipment with similar collection characteristics are divided into the same group, which is convenient for subsequent overall scheduling; secondly, by determining the complementary collection equipment group, when an abnormality occurs in a certain collection equipment group, the corresponding complementary equipment can be enabled in time for collaborative collection; thirdly, based on the real-time working status information of each collection equipment and the relationship between complementary collection equipment groups, the data collection strategy is dynamically adjusted to avoid the problem of data quality degradation caused by the use of fixed collection strategies. Even in the case of drastic fluctuations in traffic flow or sudden changes in equipment load, a high data collection quality can be maintained, which improves the timeliness of traffic information transmission and realizes the reasonable scheduling of collection resources.

[0007] In a second aspect of the present application, a smart traffic information collection system based on big data is provided, the system comprising: A collection device grouping module is used to obtain the historical collection records of each collection device in the target traffic network, and group each of the collection devices based on the historical collection records to obtain multiple collection device groups; A device data acquisition module, used to acquire the working status information of each acquisition device in the acquisition device group; A complementary device determination module, used to determine a complementary acquisition device group for each of the acquisition device groups; The acquisition strategy determination module is used to determine the data acquisition strategy of each acquisition device group based on the working status information of each acquisition device and the complementary acquisition device group.

[0008] In a third aspect of the present application, a computer storage medium is provided, wherein the computer storage medium stores a plurality of instructions, wherein the instructions are suitable for being loaded by a processor and executing the above method steps.

[0009] In a fourth aspect of the present application, an electronic device is provided, comprising: a processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the above-mentioned method steps.

[0010] In summary, one or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages: This application groups the collection devices based on historical collection records, so that devices with similar collection characteristics are divided into the same group, which is convenient for subsequent overall scheduling; secondly, by determining the complementary collection device group, when an abnormality occurs in a certain collection device group, the corresponding complementary device can be enabled in time for collaborative collection; thirdly, based on the real-time working status information of each collection device and the relationship between the complementary collection device groups, the data collection strategy is dynamically adjusted to avoid the problem of data quality degradation caused by the use of fixed collection strategies. Even in the case of drastic fluctuations in traffic flow or sudden changes in equipment load, a high data collection quality can be maintained, which improves the timeliness of traffic information transmission and realizes the reasonable scheduling of collection resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 It is a flowchart of a method for collecting intelligent traffic information based on big data provided by an embodiment of the present application; Figure 2 It is a module schematic diagram of a smart traffic information collection system based on big data provided in an embodiment of the present application; Figure 3 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application.

[0012] Description of reference numerals: 300, electronic device; 301, processor; 302, communication bus; 303, user interface; 304, network interface; 305, memory. DETAILED DESCRIPTION

[0013] In order to enable technicians in this field to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments.

[0014] In the description of the embodiments of the present application, words such as "for example" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "for example" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "for example" or "for example" is intended to present related concepts in a specific way.

[0015] In the description of the embodiments of the present application, the meaning of the term "multiple" refers to two or more. For example, multiple systems refer to two or more systems, and multiple screen terminals refer to two or more screen terminals. In addition, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. The terms "include", "comprise", "have" and their variations all mean "including but not limited to", unless otherwise specifically emphasized.

[0016] The following will provide a clear and complete description of the technical solutions in the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments.

[0017] Please refer to Figure 1 , a flowchart of a method for collecting intelligent traffic information based on big data is proposed. The method can be implemented by a computer program, a single-chip microcomputer, or run on an intelligent traffic information collection system based on big data. The computer program can be integrated into a computer device or run as an independent tool application. Specifically, the method includes steps 10 to 40, and the steps are as follows: Step 10: Obtain the historical collection records of each collection device in the target traffic network, and group each collection device based on the historical collection records to obtain multiple collection device groups.

[0018] The target traffic road network in the embodiment of the present application refers to the road network area where traffic information collection is required, which can be a road network in a certain area of ​​a city, a trunk road network connecting multiple areas, or a highway network.

[0019] In the embodiment of the present application, the collection equipment refers to equipment installed in the target traffic network for collecting traffic information, including but not limited to vehicle detectors, video surveillance equipment, electronic tag readers, GPS signal receivers, traffic flow monitors and other equipment that can collect traffic-related data.

[0020] In the embodiment of the present application, historical collection records refer to traffic data records collected by each collection device within a preset time period, including but not limited to traffic flow data, vehicle speed data, vehicle type data, traffic event data, etc., as well as equipment work records generated during the collection process, such as data collection time, collection frequency, data transmission records and other information.

[0021] In the embodiment of the present application, a collection device group refers to a collection device set grouped according to data features and geographic location information in historical collection records. Devices in the same collection device group have similar data collection features and adjacent geographic location distribution.

[0022] Specifically, the system can obtain the historical collection records of each collection device in the target traffic network in the past period of time (for example, the last 30 days). Based on these historical collection records, the data integrity of each collection device is calculated, that is, the ratio of the amount of effective data actually obtained during the collection cycle to the amount of data that should be obtained theoretically. Subsequently, the system divides each collection device into different data integrity levels according to the preset integrity interval (for example, 90%-100% is high integrity, 70%-90% is medium integrity, and below 70% is low integrity). On this basis, the geographical location information of each collection device is further obtained, and the collection devices with the same data integrity level and adjacent geographical locations (for example, the straight-line distance is less than 500 meters) are divided into the same collection device group. Through this grouping method based on data integrity and geographical location, the collection devices in the same group have similar data collection characteristics and spatial distribution characteristics, which provides a basis for the subsequent formulation of collection strategies based on the overall characteristics of the equipment group, and also facilitates the rapid positioning of possible backup devices when an abnormality occurs in the collection equipment in a certain area.

[0023] Based on the above embodiment, as an optional embodiment, the step of grouping the collection devices based on the historical collection records to obtain multiple collection device groups may also include the following steps: Step 101: Obtain the data integrity of each collection device in the historical collection records.

[0024] Specifically, before grouping the collection devices, it is necessary to first evaluate the data collection quality of each collection device. The system can obtain the historical collection records of each collection device within a preset time period (for example, the last 30 days) and calculate its data integrity. For each collection device, the system first determines the amount of data N that should be collected theoretically, which can be calculated based on the collection frequency of the device (for example, once per minute) and the statistical time length (for example, 30 days). Then, the amount of valid data M actually obtained by the collection device during this period is counted, where valid data refers to data records that meet preset quality standards, such as correct data format, values ​​within a reasonable range, and complete timestamps. Data integrity is the ratio of the actual amount of valid data obtained to the theoretical amount of data that should be collected, that is, M / N.

[0025] Step 102: Determine the data integrity level of each acquisition device according to the data integrity.

[0026] Specifically, after obtaining the data integrity of each acquisition device, the acquisition device will be graded according to the preset integrity range. For example, when the data integrity is greater than 90%, the acquisition device is classified as Class A; when the data integrity is between 70%-90%, the acquisition device is classified as Class B; when the data integrity is less than 70%, the acquisition device is classified as Class C. In this way, the system can quantitatively evaluate the data acquisition capabilities of each acquisition device, provide a reliable quality basis for subsequent device grouping, and help to group devices with similar acquisition performance into the same group, so as to facilitate the subsequent formulation of targeted data acquisition strategies.

[0027] Step 103: Acquire geographic location information of each collection device, and group collection devices with the same data integrity level and adjacent geographic locations into the same collection device group, thereby obtaining multiple collection device groups.

[0028] Specifically, the geographic location information of each acquisition device is obtained, including its longitude and latitude coordinates. These geographic location information can be fixed coordinates recorded when the device is installed, or can be obtained in real time through the built-in GPS module of the device. Then, the acquisition devices of the same integrity level are grouped by spatial clustering. Specifically, firstly, an ungrouped acquisition device is selected as the initial device, and the straight-line distance between it and other ungrouped devices of the same level is calculated. When the distance between two devices is less than a preset spatial threshold (for example, 500 meters), the two devices are considered to be geographically adjacent. The system divides all devices of the same level that are geographically adjacent to the initial device into the same acquisition device group. Repeat the above process until all acquisition devices are divided into corresponding groups. For example, the system may obtain multiple A-level device groups (A1 group, A2 group, etc.), and the devices in each A-level device group have more than 90% data integrity and are geographically adjacent. This grouping method based on data integrity level and geographical location enables the acquisition devices in the same group to have not only similar data acquisition quality, but also spatial correlation. Such grouping results are conducive to quickly calling nearby devices of the same level for collaborative collection when data collection anomalies occur in a certain area in the future, thereby improving the system's fault tolerance and the reliability of data collection.

[0029] Step 20: Obtain the working status information of each acquisition device in the acquisition device group.

[0030] In the embodiment of the present application, the working status information refers to the real-time operating status parameters of the acquisition device, including but not limited to: the current data acquisition frequency of the device, CPU usage, memory occupancy, data cache capacity, network transmission bandwidth occupancy, device power (for battery-powered devices), device temperature, data collection success rate in the most recent period (for example, the past 1 hour), and other real-time information reflecting the working status of the device.

[0031] Specifically, in order to grasp the operating status of each acquisition device in real time and promptly discover and handle possible abnormal situations, it is necessary to regularly obtain the working status information of each acquisition device in the acquisition device group. Through the data communication link established with each acquisition device, a status query instruction is sent to the acquisition device at preset time intervals (for example, every 5 minutes). After receiving the query instruction, the acquisition device will collect the current working status parameters and package them back. These working status parameters include: the device's CPU usage rate (used to evaluate processing load), memory occupancy rate (used to evaluate data processing capabilities), data cache capacity (used to evaluate data storage status), network transmission bandwidth occupancy rate (used to evaluate data transmission capabilities), current data collection frequency (used to confirm whether it meets the preset collection requirements), device temperature (used to monitor the device operating environment) and the data collection success rate of the most recent statistical period (for example, the past 1 hour) (used to evaluate collection stability). For battery-powered acquisition devices, the current power information will also be returned. After receiving this working status information, the system stores it in the database and compares it with the preset normal working parameter range. This mechanism of regularly obtaining working status information enables the system to grasp the operating status of each acquisition device in a timely manner, Step 30: Determine the complementary acquisition device group of each acquisition device group.

[0032] In the embodiment of the present application, a complementary acquisition device group refers to a combination relationship of acquisition device groups that have complementary properties in terms of data acquisition capability index, spatial distribution range, etc. Specifically, when two acquisition device groups meet the following conditions, they are complementary acquisition device groups: the two device groups have a high degree of capability complementarity (that is, their data acquisition capability indexes can match and support each other), and their spatial distribution ranges overlap or are adjacent (that is, the spatial distance is within the preset threshold range). This complementary relationship ensures that when an acquisition device group is abnormal, its complementary acquisition device group can effectively undertake the data acquisition tasks of the original device group in terms of performance assurance and spatial coverage.

[0033] Specifically, in order to build a highly reliable data acquisition system, it is necessary to establish a complementary mechanism between acquisition equipment groups, that is, to determine for each acquisition equipment group a complementary acquisition equipment group that can support each other in terms of performance and spatial dimensions. First, the data acquisition capability index of each acquisition equipment group is calculated. This index is obtained by comprehensively evaluating the working status parameters of each acquisition equipment in the equipment group, such as CPU usage, memory occupancy, data cache capacity, and network transmission bandwidth occupancy, and is obtained by weighted calculation in combination with the number of equipment and data integrity level. Subsequently, the system obtains the spatial distribution range of each acquisition equipment group, and calculates the minimum circumscribed polygon representing the coverage area of ​​the equipment group by analyzing the geographical location coordinates of all acquisition equipment in the equipment group. On this basis, the capability complementarity between any two acquisition equipment groups is calculated, which reflects the degree of matching between the two equipment groups in terms of data acquisition capability. When the data acquisition capability indexes of the two equipment groups are close, they have a high degree of capability complementarity; conversely, when the capability indexes differ greatly, the complementarity is low. Finally, the complementary acquisition equipment group is determined by comprehensively considering the capability complementarity and spatial distribution characteristics. If the capability complementarity of two equipment groups exceeds a preset threshold (e.g. 0.8), and their spatial distribution ranges overlap or the distance is less than a preset value (e.g. 500 meters), the two equipment groups are identified as each other's complementary collection equipment groups. This complementary mechanism based on data collection capability and spatial distribution ensures that when an abnormality occurs in a collection equipment group, its complementary collection equipment group can effectively undertake the data collection task of the original equipment group from the two dimensions of performance guarantee and spatial coverage, thereby maintaining the stable operation of the system and the continuity of data collection.

[0034] Based on the above embodiment, as another optional embodiment, the step of determining the complementary acquisition device group of each acquisition device group may further include the following steps: Step 301: Calculate the data acquisition capability index of each acquisition device group.

[0035] Specifically, when calculating the data collection capability index, the system obtains the real-time working status information of each collection device in the collection device group, including basic parameters such as CPU usage, memory occupancy, data cache capacity, and network transmission bandwidth occupancy. For each collection device group, the system calculates the average CPU usage and average memory occupancy of all devices in it, and considers the total number of devices in the device group and the data integrity level. The specific calculation formula for the data collection capability index is: I=α×(1-average CPU usage)+β×(1-average memory occupancy)+γ×number of devices×integrity level coefficient, where α=0.4, β=0.3, and γ=0.3 are preset weight coefficients.

[0036] Step 302: Obtain the spatial distribution range of each acquisition device group.

[0037] Specifically, the geographic location coordinates of each acquisition device in each acquisition device group are read, and the convex hull algorithm is used to calculate the minimum circumscribed polygon containing all device coordinate points. The Graham scanning algorithm can be used to first find the point in the lower left corner as the starting point, and then sort the other points according to the polar angle with the starting point, and connect them in sequence to form a convex polygon, which is the spatial distribution range of the acquisition device group. For example, there are 4 acquisition devices in a certain acquisition device group, and their coordinates are (1,1), (1,4), (3,2), and (4,3). The convex hull algorithm can be used to obtain a quadrilateral area as the spatial distribution range of the device group. In this way, the system not only obtains the data acquisition capability index that reflects the performance level of the device group, but also obtains the spatial distribution range that characterizes the geographical coverage characteristics of the device group, which provides an important evaluation basis for the subsequent determination of complementary acquisition device groups.

[0038] Step 303: Calculate the capability complementarity between the acquisition equipment groups, where the capability complementarity is determined based on the data acquisition capability index.

[0039] Specifically, the capability complementarity between any two acquisition equipment groups is calculated. This indicator reflects the degree of matching between the two equipment groups in data acquisition capabilities. For acquisition equipment groups A and B, assuming that their data acquisition capability indexes are Ia and Ib respectively, the capability complementarity calculation formula between them is: C=1-|Ia-Ib| / (Ia+Ib). This calculation method ensures that the capability complementarity value range is between 0 and 1, and the closer the data acquisition capability indexes of the two equipment groups are, the closer their capability complementarity is to 1.

[0040] For example, if the data collection capability index of equipment group A is 1.81 and the data collection capability index of equipment group B is 1.65, then the capability complementarity between them is C = 1-|1.81-1.65| / (1.81+1.65) = 1-0.16 / 3.46 = 0.954.

[0041] Step 304: Determine a complementary acquisition device group based on the capability complementarity and the spatial distribution range.

[0042] Specifically, first determine whether the capability complementarity of the two equipment groups exceeds a preset threshold (e.g., 0.8). If the condition is met, further calculate the relationship between their spatial distribution ranges. The system uses a polygon intersection algorithm to determine whether the spatial distribution ranges of the two equipment groups overlap. If not, calculate the shortest distance between the two polygons. When the distance is less than the preset distance threshold (e.g., 100 meters), it is considered that the two equipment groups have the possibility of spatial complementarity. Finally, if the two acquisition equipment groups meet both the capability complementarity threshold and the spatial distribution requirements, they are mutually determined as each other's complementary acquisition equipment groups. This complementary mechanism based on capability complementarity and spatial characteristics not only ensures that the standby equipment group has sufficient data acquisition capabilities to undertake the tasks of the original equipment group, but also ensures the geographical proximity of the complementary equipment group, so that it can quickly switch and maintain the continuity and integrity of data acquisition when an abnormality occurs in the system.

[0043] Based on the above embodiment, as another optional embodiment, the step of calculating the data acquisition capability index of each acquisition device group may further include the following steps: Step 3011: Obtain the data collection success rate of each collection device group within a preset time period, and count the number of data types supported by each collection device group.

[0044] Specifically, the data collection success rate of each collection device group in a preset time period (for example, the past 24 hours) is obtained, which is calculated by counting the total number of data collection requests and the number of successes of each collection device group in the time period. For example, a collection device group has initiated 1,000 data collection requests in the past 24 hours, of which 950 were successfully completed, and its data collection success rate is 95%. At the same time, the system counts the number of data types supported by each collection device group, which include but are not limited to temperature data, humidity data, pressure data, vibration data, etc.

[0045] Step 3012: Calculate the data collection amount per unit time based on the collection frequency of each collection device group.

[0046] Specifically, the collection frequency refers to the number of times each data type is sampled per unit time, and the specific calculation formula is: data collection volume = Σ(collection frequency of each type of data × data volume collected in a single time).

[0047] Step 3013: Obtain the data collection capability index of each collection device group based on a weighted combination of the data collection success rate, the number of data types, and the amount of data collected.

[0048] Specifically, the three indicators of data collection success rate, number of data types and data collection volume are weighted and combined to calculate the data collection capability index. The weight of the data collection success rate is set to α (for example, 0.4), the weight of the number of data types is set to β (for example, 0.3), and the weight of the data collection volume is set to γ ​​(for example, 0.3). The calculation formula of the data collection capability index is: I = α × data collection success rate + β × (number of data types / number of benchmark types) + γ × (data collection volume / benchmark collection volume). Among them, the number of benchmark types and the benchmark collection volume are standard values ​​preset by the system. Continuing with the above example, assuming that the number of benchmark types is 5 and the benchmark collection volume is 50KB / minute, the data collection capability index of the collection device group is I = 0.4 × 0.95 + 0.3 × (3 / 5) + 0.3 × (31 / 50) = 0.38 + 0.18 + 0.186 = 0.746. Through this multi-dimensional weighted calculation method, the data collection capability index obtained by the system not only reflects the collection reliability of the equipment group, but also reflects the comprehensiveness and efficiency of its data collection, providing a scientific evaluation basis for the subsequent determination of complementary collection equipment groups.

[0049] Based on the above embodiment, as another optional embodiment, the step of determining the complementary acquisition device group based on the capability complementarity and the spatial distribution range may further include the following steps: Step 3041: Calculate the overlapping coverage between each acquisition device group based on the spatial distribution range.

[0050] Specifically, based on the acquired spatial distribution range, the overlapping coverage rate between any two acquisition device groups is calculated. For acquisition device groups A and B, assuming that their spatial distribution ranges are represented by polygons A and B, respectively, the system uses a polygon intersection algorithm to calculate the area Soverlap of the overlapping area, and calculates the areas SA and SB of polygons A and B, respectively. The overlapping coverage rate calculation formula between the two device groups is: R=Soverlap / min(SA, SB). For example, if the area of ​​polygon A is 100 square meters, the area of ​​polygon B is 120 square meters, and their overlapping area is 60 square meters, then the overlapping coverage rate R=60 / 100=0.6. Through this calculation method, the overlapping coverage rate obtained by the system can accurately reflect the degree of overlap between the two device groups in spatial distribution.

[0051] Step 3042: Filter out acquisition device pairs whose overlapping coverage ratio is greater than the overlapping coverage ratio threshold.

[0052] Specifically, the calculated overlap coverage is compared with a preset overlap coverage threshold (e.g., 0.3) to screen out acquisition device pairs with an overlap coverage greater than the threshold. In the above example, since the overlap coverage of 0.6 is greater than the threshold of 0.3, the device pair consisting of device groups A and B will be retained for subsequent processing.

[0053] Step 3043: sorting the capability complementarity of each acquisition device pair, and determining the acquisition device pair whose capability complementarity is greater than the capability complementarity threshold as a complementary acquisition device group.

[0054] Specifically, the capability complementarity of the screened acquisition equipment pairs is sorted in descending order, and the capability complementarity is calculated using the aforementioned method, i.e., C=1-|Ia-Ib| / (Ia+Ib), where Ia and Ib are the data acquisition capability indexes of equipment groups A and B, respectively. For example, if the data acquisition capability index of equipment group A is 0.746 and the data acquisition capability index of equipment group B is 0.682, then the capability complementarity between them is C=1-|0.746-0.682| / (0.746+0.682)=1-0.064 / 1.428=0.955. Finally, the system compares the sorted equipment pairs with a preset capability complementarity threshold (e.g., 0.85), and determines the acquisition equipment pairs with a capability complementarity greater than the threshold as complementary acquisition equipment groups. In the above example, since the capability complementarity of 0.955 is greater than the threshold of 0.85, equipment groups A and B are finally determined as complementary acquisition equipment groups. This dual screening mechanism based on spatial overlapping coverage and capability complementarity ensures that the complementary collection equipment group has sufficient overlapping coverage in terms of geographical location and a high degree of matching in terms of data collection capabilities, so that it can effectively undertake its data collection tasks when an abnormality occurs in the original equipment group, thereby ensuring the reliability of the system and the continuity of data collection.

[0055] Step 40: Based on the working status information of each collection device and the complementary collection device group, determine the data collection strategy of each collection device group.

[0056] Specifically, in order to achieve efficient collaboration and optimal resource allocation of the acquisition equipment group, the system needs to formulate a reasonable data acquisition strategy based on the working status information of each acquisition device and the characteristics of the complementary acquisition equipment group. First, the real-time working status information of each acquisition device in each acquisition equipment group is obtained, including basic operating parameters such as CPU usage, memory occupancy, data cache capacity, and network transmission bandwidth occupancy. Specifically, the system uses the CPU usage and memory occupancy of the acquisition device as key indicators for evaluating the device load. For example, when the CPU usage of a certain acquisition device is 75% and the memory occupancy is 80%, it can be determined that the device is in a high load state. At the same time, the system calculates the overall working status index of each acquisition device group, which is obtained by weighted averaging the working status parameters of all acquisition devices in the device group. The calculation formula is: W=δ×average CPU usage+ε×average memory occupancy+ζ×average bandwidth occupancy, where δ, ε, ζ are weight coefficients, and δ+ε+ζ=1. Then, based on the calculated overall working status index and combined with the information of the complementary acquisition device group, a data acquisition strategy is formulated for each acquisition device group. When the overall working status index of a certain collection equipment group exceeds the preset high load threshold (for example, 0.8), the system will allocate part of the data collection tasks to its complementary collection equipment group. The specific task allocation method is: first calculate the amount of data collection tasks to be allocated, the calculation formula is: T = (W-0.8) × the total current task, where W is the overall working status index; then according to the data collection capability index of the complementary collection equipment group, the task volume T is allocated to each complementary collection equipment group according to the proportion of the capability index.

[0057] Based on the above embodiment, as another optional embodiment, the step of determining the data collection strategy of each collection device group based on the working status information of each collection device and the complementary collection device group may also include the following steps: Step 401: Calculate the resource pressure index of each set of equipment based on the calculated resource occupancy rate.

[0058] Specifically, it is achieved by monitoring the CPU usage and memory occupancy of the collection device. The calculation formula of the resource pressure index is: Ir=λ1×CPU usage+λ2×memory occupancy, where λ1 and λ2 are weight coefficients and λ1+λ2=1. For example, the CPU usage of a collection device is 85%, and the memory occupancy is 75%. If λ1=0.6 and λ2=0.4, then its resource pressure index Ir=0.6×0.85+0.4×0.75=0.81. When the resource pressure index exceeds the preset threshold (such as 0.8), it indicates that the computing resource load of the device is heavy.

[0059] Step 402: Calculate the storage pressure index of each acquisition device based on the remaining storage capacity.

[0060] Specifically, the system calculates the storage pressure index of each acquisition device based on the remaining storage capacity. The calculation formula is: Is=1-(remaining storage capacity / total storage capacity)^μ, where μ is an adjustment coefficient (for example, a value of 1.5), which is used to speed up the growth rate of the storage pressure index when the storage space is insufficient. For example, if the total storage capacity of a certain acquisition device is 1000GB and the remaining storage capacity is 200GB, then its storage pressure index Is=1-(200 / 1000)^1.5=0.911. This calculation method ensures that when the remaining storage space is small, the storage pressure index will increase rapidly, reminding the system to clean up or dump data in time.

[0061] Step 403: Calculate the transmission pressure index of each acquisition device based on the transmission queue length.

[0062] Specifically, the system calculates the transmission pressure index of each acquisition device based on the transmission queue length. The calculation formula is: It = current queue length / (maximum queue length × η), where η is the buffer coefficient (for example, 0.8), which is used to reserve a safety margin for data transmission. For example, the maximum transmission queue length of a certain acquisition device is 1000 data records, the current queue length is 600, and η = 0.8, then its transmission pressure index It = 600 / (1000 × 0.8) = 0.75. This calculation method can reflect the data transmission load of the device. When the transmission pressure index is close to or exceeds 1, it indicates that the data transmission capacity of the device is close to saturation.

[0063] Step 404: When there is an abnormal collection device group whose resource pressure index and / or storage pressure index and / or transmission pressure index exceeds the corresponding threshold, determine the collaborative task of the complementary collection device group corresponding to the abnormal collection device group.

[0064] Specifically, the system identifies abnormal conditions by comparing the three types of pressure indexes of each collection device group with the corresponding thresholds. The resource pressure index threshold is set to 0.8, the storage pressure index threshold is set to 0.9, and the transmission pressure index threshold is set to 0.85. When any pressure index exceeds the corresponding threshold, the system marks the collection device group as abnormal. For example, the resource pressure index of a collection device group is 0.81, the storage pressure index is 0.911, and the transmission pressure index is 0.75. Since both the resource pressure index and the storage pressure index exceed their respective thresholds, the device group is judged to be in an abnormal state.

[0065] Furthermore, for abnormal collection equipment groups, corresponding task allocation strategies are formulated based on different types of pressure exceeding conditions. When the resource pressure index exceeds the limit, the system calculates the amount of computing tasks that need to be allocated, and the calculation formula is: Tr=(Ir-0.8)×the current total amount of computing tasks, where Ir is the resource pressure index; when the storage pressure index exceeds the limit, the system calculates the amount of data storage that needs to be transferred, and the calculation formula is: Ts=(Is-0.9)×the current total amount of stored data, where Is is the storage pressure index; when the transmission pressure index exceeds the limit, the system calculates the amount of data transmission that needs to be diverted, and the calculation formula is: Tt=(It-0.85)×the current transmission queue length, where It is the transmission pressure index. For example, a certain abnormal collection equipment group is currently responsible for the computing task of 1000 collection points / minute, and its resource pressure index is 0.85, then the amount of computing tasks that need to be allocated is (0.85-0.8)×1000=50 collection points / minute. Then, the system determines a specific collaborative task allocation plan for the abnormal collection equipment group. The system first checks the current pressure state of the complementary acquisition device groups of the device group, excludes the complementary device groups that are also in a high pressure state, and then allocates tasks in proportion to the data acquisition capability index of the remaining complementary device groups. Specifically, if an abnormal acquisition device group has two available complementary acquisition device groups A and B, and their data acquisition capability indexes are 0.746 and 0.682 respectively, then for the calculation task of 50 acquisition points / minute that needs to be allocated, the task amount allocated to A is 50×0.746 / (0.746+0.682)≈26 acquisition points / minute, and the task amount allocated to B is 50×0.682 / (0.746+0.682)≈24 acquisition points / minute.

[0066] Step 405: Generate a data collection strategy including collection tasks and / or collaborative tasks of each collection device group.

[0067] Specifically, the system finally generates a complete data collection strategy, including the original collection tasks and collaborative tasks of each collection equipment group. For the collection equipment group operating normally, its original collection tasks remain unchanged; for the abnormal collection equipment group, its collection task volume is adjusted, and the excess part is allocated as a collaborative task to the corresponding complementary collection equipment group; for the complementary collection equipment group that undertakes the collaborative task, the corresponding collaborative task is added on the basis of its original collection task. Through this dynamic task adjustment and allocation mechanism, the system can effectively alleviate the pressure of the abnormal collection equipment group, ensure the continuity and reliability of data collection work, and at the same time, through the reasonable allocation of collaborative tasks, avoid the generation of new pressure bottlenecks and improve the operating efficiency of the entire data collection system.

[0068] For example, suppose there are two cameras (A1, A2) and a traffic flow detector (B1) at a busy intersection. The collection ranges of A1 and A2 overlap and can be divided into Group A. The collection range of B1 partially overlaps with Group A, which constitutes a complementary area. The system can formulate a strategy: A1 works normally and A2 is on standby; when A1 data is abnormal, A2 is automatically enabled to collect data, and the data of B1 is referenced for verification.

[0069] Based on the above embodiment, as another optional embodiment, a method for collecting intelligent traffic information based on big data may also include the following process: Specifically, in order to ensure the reliability and accuracy of the collected data, the system needs to comprehensively evaluate and promptly correct the data quality of each collection device group. First, obtain the data quality indicators of each collection device group, including data completeness, data accuracy, and data timeliness. Among them, the data completeness is determined by calculating the ratio of the number of data points actually obtained to the number of data points that should be obtained theoretically, and the calculation formula is: Rc=actual number of data points / theoretical number of data points. For example, a certain collection device group should collect 3600 data points in the past hour, and actually collected 3420 data points, then its data completeness is 3420 / 3600=0.95. The data accuracy is evaluated by comparing the collected data with the preset reasonable value range, and the calculation formula is: Ra=valid data points / total data points, where valid data points refer to data points whose values ​​fall within a reasonable range. For example, 96 of the 100 data points collected by a temperature sensor fall within the preset reasonable range of -30℃ to 50℃, then its data accuracy is 0.96. Data timeliness is measured by calculating the ratio of the time required for data collection to be available to the preset maximum allowable delay time, and the calculation formula is: Rt=1-min(actual delay time / maximum allowable delay time, 1). For example, when the actual processing delay of a data point is 200 milliseconds and the maximum allowable delay is 500 milliseconds, its timeliness is 1-200 / 500=0.6.

[0070] The system calculates the data quality score of each acquisition device group based on these three indicators, and the calculation formula is: Q=ω1×Rc+ω2×Ra+ω3×Rt, where ω1, ω2, and ω3 are weight coefficients. For example, if ω1=0.3, ω2=0.4, and ω3=0.3, the data quality score of the above acquisition device group is Q=0.3×0.95+0.4×0.96+0.3×0.6=0.849. When the calculated data quality score is lower than the preset quality threshold (for example, 0.85), the system marks the acquisition device group as a quality abnormality state. In this case, the system obtains calibration data from the complementary acquisition device group corresponding to the quality abnormality acquisition device group, and uses these data to correct the quality abnormality data. The specific correction process includes: first, the acquisition data of the complementary acquisition device group in the same time period and the same location is obtained as calibration data; then, the system calculates the deviation between the calibration data and the abnormal data, including the systematic deviation and the random deviation; finally, the system corrects the abnormal data according to the identified deviation. For example, if the data collected by a temperature sensor is continuously 2°C higher and shows a random fluctuation of ±0.5°C, the system will first eliminate the 2°C system deviation, and then smooth the random fluctuation through algorithms such as Kalman filtering, and finally obtain the corrected collected data. Through this mechanism based on data quality assessment and complementary equipment calibration, the system can promptly detect and correct anomalies in the collected data, ensure the reliability and accuracy of the data, and provide more valuable data support for subsequent data analysis and decision-making.

[0071] See also Figure 2 , is a module diagram of a smart traffic information collection system based on big data provided in an embodiment of the present application, wherein the system includes: A collection device grouping module is used to obtain the historical collection records of each collection device in the target traffic network, and group each of the collection devices based on the historical collection records to obtain multiple collection device groups; A device data acquisition module, used to acquire the working status information of each acquisition device in the acquisition device group; A complementary device determination module, used to determine a complementary acquisition device group for each of the acquisition device groups; The acquisition strategy determination module is used to determine the data acquisition strategy of each acquisition device group based on the working status information of each acquisition device and the complementary acquisition device group.

[0072] Optionally, the collection device grouping module is further used to obtain the data integrity of each collection device in the historical collection record; Determining the data integrity level of each of the acquisition devices according to the data integrity; The geographical location information of each of the acquisition devices is obtained, and the acquisition devices with the same data integrity level and adjacent geographical locations are divided into the same acquisition device group to obtain multiple acquisition device groups.

[0073] Optionally, the complementary device determination module is further used to calculate the data acquisition capability index of each of the acquisition device groups; Obtaining the spatial distribution range of each of the acquisition device groups; Calculating the capability complementarity between the acquisition equipment groups, wherein the capability complementarity is determined according to the data acquisition capability index; Based on the capability complementarity and the spatial distribution range, a complementary acquisition device group is determined.

[0074] Optionally, the complementary device determination module is further used to obtain the data collection success rate of each of the collection device groups within a preset time period, and to count the number of data types supported by each of the collection device groups; Calculating the amount of data collected per unit time based on the collection frequency of each of the collection device groups; The data collection capability index of each collection device group is obtained according to a weighted combination of the data collection success rate, the number of data types and the data collection amount.

[0075] Optionally, the complementary device determination module is further used to calculate the overlapping coverage between each acquisition device group based on the spatial distribution range; Screening out acquisition device pairs whose overlapping coverage ratio is greater than an overlapping coverage ratio threshold; The capability complementarity of each of the acquisition device pairs is sorted, and the acquisition device pairs whose capability complementarity is greater than a capability complementarity threshold are determined as complementary acquisition device groups.

[0076] Optionally, the acquisition strategy determination module is further used to calculate the overlapping coverage between each acquisition device group based on the spatial distribution range; Screening out acquisition device pairs whose overlapping coverage ratio is greater than an overlapping coverage ratio threshold; The capability complementarity of each of the acquisition device pairs is sorted, and the acquisition device pairs whose capability complementarity is greater than a capability complementarity threshold are determined as complementary acquisition device groups.

[0077] Optionally, the acquisition strategy determination module is further used to obtain data quality indicators of each of the acquisition device groups, wherein the data quality indicators include data completeness, data accuracy and data timeliness; Calculate the data quality score of each of the acquisition device groups based on the data quality index; When the data quality score is lower than a preset quality threshold, determining a collection device group with abnormal quality; Calibration data is acquired from the complementary acquisition device group corresponding to the acquisition device group with abnormal quality, and the abnormal quality data is corrected based on the calibration data to obtain corrected acquisition data.

[0078] It should be noted that: when the system provided in the above embodiment realizes its functions, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the system and method embodiments provided in the above embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0079] An embodiment of the present application also provides a computer storage medium, which can store multiple instructions. The instructions are suitable for being loaded by a processor and executing a method for collecting smart traffic information based on big data in the above embodiment. The specific execution process can be found in the specific description of the above embodiment, which will not be repeated here.

[0080] Please refer to Figure 3 , the application also discloses an electronic device. Figure 3 The electronic device 300 may include: at least one processor 301 , at least one network interface 304 , a user interface 303 , a memory 305 , and at least one communication bus 302 .

[0081] The communication bus 302 is used to realize the connection and communication between these components.

[0082] The user interface 303 may include a display screen (Display) and a camera (Camera). Optionally, the user interface 303 may also include a standard wired interface and a wireless interface.

[0083] The network interface 304 may optionally include a standard wired interface or a wireless interface (such as a WI-FI interface).

[0084] Among them, the processor 301 may include one or more processing cores. The processor 301 uses various interfaces and lines to connect various parts in the entire server, and executes various functions of the server and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory 305, and calling data stored in the memory 305. Optionally, the processor 301 can be implemented in at least one hardware form of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), and programmable logic array (Programmable Logic Array, PLA). The processor 301 can integrate one or a combination of a central processing unit (Central Processing Unit, CPU), a graphics processing unit (Graphics Processing Unit, GPU) and a modem. Among them, the CPU mainly processes the operating system, user interface and application programs; the GPU is responsible for rendering and drawing the content to be displayed on the display screen; the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 301, and it can be implemented separately through a chip.

[0085] Among them, the memory 305 may include a random access memory (Random Access Memory, RAM) and may also include a read-only memory (Read-Only Memory). Optionally, the memory 305 includes a non-transitory computer-readable storage medium. The memory 305 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 305 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned method embodiments, etc.; the data storage area may store data involved in the above-mentioned method embodiments, etc. The memory 305 may optionally be at least one storage device located away from the aforementioned processor 301. Refer to Figure 3 , as a computer storage medium, the memory 305 may include an operating system, a network communication module, a user interface module, and an application program of a smart traffic information collection method based on big data.

[0086] exist Figure 3In the electronic device 300 shown, the user interface 303 is mainly used to provide an input interface for the user and obtain the data input by the user; and the processor 301 can be used to call the application program stored in the memory 305 for a smart traffic information collection method based on big data. When executed by one or more processors 301, the electronic device 300 executes one or more of the methods described in the above embodiments. It should be noted that for the aforementioned method embodiments, for the sake of simple description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited to the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required for the present application.

[0087] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0088] In the several embodiments provided in this application, it should be understood that the disclosed devices can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some service interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0089] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0090] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0091] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a memory and includes several instructions for a computer device (which can be a personal computer, server or network device, etc.) to execute all or part of the steps of the various embodiments of the present application. The aforementioned memory includes: various media that can store program codes, such as USB flash drives, mobile hard drives, magnetic disks or optical disks.

[0092] The above is only an exemplary embodiment of the present disclosure and cannot be used to limit the scope of the present disclosure. That is, any equivalent changes and modifications made according to the teachings of the present disclosure are still within the scope of the present disclosure. After considering the disclosure of the specification and the truth of practice, those skilled in the art will easily think of other embodiments of the present disclosure.

[0093] This application is intended to cover any variation, use or adaptation of the present disclosure, which follows the general principles of the present disclosure and includes common knowledge or customary technical means in the art not described in the present disclosure. The description and examples are to be regarded as exemplary only, and the scope and spirit of the present disclosure are defined by the claims.

Claims

1. A method for collecting intelligent traffic information based on big data, characterized in that: The method comprises: Acquire historical collection records of each collection device in the target traffic network, and group each of the collection devices based on the historical collection records to obtain multiple collection device groups; Obtaining working status information of each acquisition device in the acquisition device group; Determining a complementary collection device group for each of the collection device groups; Based on the working status information of each acquisition device and the complementary acquisition device group, a data acquisition strategy for each acquisition device group is determined.

2. The method for collecting intelligent traffic information based on big data according to claim 1 is characterized in that: The collecting devices are grouped based on the historical collecting records to obtain a plurality of collecting device groups, including: Obtaining the data integrity of each of the acquisition devices in the historical acquisition records; Determining the data integrity level of each of the acquisition devices according to the data integrity; The geographical location information of each of the acquisition devices is obtained, and the acquisition devices with the same data integrity level and adjacent geographical locations are divided into the same acquisition device group to obtain multiple acquisition device groups.

3. The method for collecting intelligent traffic information based on big data according to claim 1 is characterized in that: The step of determining a complementary acquisition device group for each of the acquisition device groups comprises: Calculating the data collection capability index of each of the collection equipment groups; Obtaining the spatial distribution range of each of the acquisition device groups; Calculating the capability complementarity between the acquisition equipment groups, wherein the capability complementarity is determined according to the data acquisition capability index; Based on the capability complementarity and the spatial distribution range, a complementary acquisition device group is determined.

4. The method for collecting intelligent traffic information based on big data according to claim 3 is characterized in that: The calculating of the data acquisition capability index of each of the acquisition device groups comprises: Obtaining the data collection success rate of each of the collection device groups within a preset time period, and counting the number of data types supported by each of the collection device groups; Calculating the amount of data collected per unit time based on the collection frequency of each of the collection device groups; The data collection capability index of each collection device group is obtained according to a weighted combination of the data collection success rate, the number of data types and the data collection amount.

5. The method for collecting intelligent traffic information based on big data according to claim 3 is characterized in that: The determining of a complementary acquisition device group based on the capability complementarity and the spatial distribution range includes: Calculate the overlapping coverage between each collection device group based on the spatial distribution range; Screening out acquisition device pairs whose overlapping coverage ratio is greater than an overlapping coverage ratio threshold; The capability complementarity of each of the acquisition device pairs is sorted, and the acquisition device pairs whose capability complementarity is greater than a capability complementarity threshold are determined as complementary acquisition device groups.

6. The method for collecting intelligent traffic information based on big data according to claim 1 is characterized in that: The working status information includes computing resource occupancy rate, remaining storage capacity and transmission queue length. The determining of the data collection strategy of each collection device group based on the working status information of each collection device and the complementary collection device group includes: Calculating a resource pressure index of each of the acquisition devices based on the computing resource occupancy rate; Calculating a storage pressure index of each of the acquisition devices based on the remaining storage capacity; Calculate the transmission pressure index of each of the acquisition devices based on the transmission queue length; When there is an abnormal collection device group whose resource pressure index and / or storage pressure index and / or transmission pressure index exceeds corresponding thresholds, determining a collaborative task of the abnormal collection device group corresponding to the complementary collection device group; A data collection strategy including collection tasks and / or collaborative tasks of each of the collection device groups is generated.

7. The method for collecting intelligent traffic information based on big data according to claim 1 is characterized in that: The method further comprises: Acquire data quality indicators of each of the acquisition device groups, wherein the data quality indicators include data completeness, data accuracy and data timeliness; Calculate the data quality score of each of the acquisition device groups based on the data quality index; When the data quality score is lower than a preset quality threshold, determining a collection device group with abnormal quality; Calibration data is acquired from the complementary acquisition device group corresponding to the acquisition device group with abnormal quality, and the abnormal quality data is corrected based on the calibration data to obtain corrected acquisition data.

8. A smart traffic information collection system based on big data, characterized in that: The system comprises: A collection device grouping module is used to obtain the historical collection records of each collection device in the target traffic network, and group each of the collection devices based on the historical collection records to obtain multiple collection device groups; A device data acquisition module, used to acquire the working status information of each acquisition device in the acquisition device group; A complementary device determination module, used to determine a complementary acquisition device group for each of the acquisition device groups; The acquisition strategy determination module is used to determine the data acquisition strategy of each acquisition device group based on the working status information of each acquisition device and the complementary acquisition device group.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor and executing the method according to any one of claims 1 to 7.

10. An electronic device, characterized in that: It includes a processor, a memory, a user interface and a network interface, the memory is used to store instructions, the user interface and the network interface are used to communicate with other devices, and the processor is used to execute the instructions stored in the memory so that the electronic device executes the method as described in any one of claims 1-7.

Images