Road perception integrated system based on internet of things

Abstract

The application relates to the technical field of road network sensing, in particular to a highway sensing integrated system based on Internet of Things. The system comprises a server and various types of road network sensing devices, the road network sensing devices comprise laser radars, millimeter wave radars, high-definition cameras, environment detection units and infrared cameras arranged at intervals, the server comprises a weather identification module, which is used for acquiring video images captured by the high-definition cameras, and judging whether the road environment exists in a severe environment according to the video of the high-definition cameras; a device management module, which is used for controlling the high-definition cameras to be closed and the infrared cameras to be started when the weather identification module judges that the current road environment exists in a severe environment; an environment identification module, which is used for identifying the atmospheric transmittance of the current road according to the data measured by the environment detection unit; and the device management module is further used for controlling the laser radars to be started when the atmospheric transmittance meets preset conditions, and controlling the millimeter wave radars to be started when the atmospheric transmittance does not meet the preset conditions.

Description

Technical Field

[0001] This invention relates to the field of road network sensing technology, and specifically to an integrated highway sensing system based on the Internet of Things. Background Technology

[0002] Traffic sensors primarily collect data on road conditions, providing raw data for roadside sensing networks. In recent years, traffic sensors have undergone a series of technological developments, including coils, geomagnetic sensors, video sensors, and radar sensors. It is understood that existing traffic sensors are mainly divided into two categories: the first category is traditional sensors, including induction coils, cross-sectional radar, and geomagnetic nails. These devices are typically used to obtain vehicle speed information for a specific cross-section or instantaneous moment within a vehicle's lane. The second category is traffic target sensors, mainly including AI cameras, LiDAR, and traffic millimeter-scale radar.

[0003] AI cameras are primarily used to detect illegal parking, vehicles driving in the wrong direction, traffic congestion, pedestrians on the roadside, and littering on the road. However, their performance in vehicle trajectory tracking, vehicle position detection, and speed detection is relatively weak, they are easily affected by weather and environmental factors, and their detection range is relatively small.

[0004] Radar can be categorized by its operating frequency band into over-the-horizon radar, microwave radar, millimeter-wave radar, and lidar. The advantages of radar include its ability to detect distant targets both day and night, its immunity to fog, clouds, and rain, making it suitable for all weather conditions and times, and it possesses a certain degree of penetration capability.

[0005] Compared with ordinary microwave radar, millimeter-wave radar is characterized by its small size, easy integration, and high spatial resolution. It also has strong penetration capabilities through fog, smoke, and dust, strong anti-interference capabilities, and all-weather (except for heavy rain) and all-time characteristics.

[0006] Compared to conventional microwave radar, lidar offers advantages such as high resolution, strong resistance to active interference, small size, and light weight. However, it is greatly affected by weather and atmospheric conditions. In heavy rain, dense smoke, or dense fog, the attenuation increases sharply, significantly impacting the propagation distance. Furthermore, it has a narrower target search and acquisition range and relatively higher costs.

[0007] Infrared thermal imaging technology has the characteristics of penetrating fog and haze, resisting rain and snow, and resisting glare. In severe conditions such as no light or low light, fog and haze, rain and snow, and glare, ordinary visible light cameras basically lose their monitoring capabilities, while infrared thermal imagers are not affected by interference and can monitor real-time road conditions, vehicle conditions, and the safety status of public facilities on highways.

[0008] Therefore, each type of sensor on highways has its own characteristics and limitations. A single sensor cannot achieve all-weather, high-precision sensing; multiple sensors need to complement each other and cooperate to build a sensing system adaptable to different weather conditions and environments. However, there is overlap between the sensing data of various sensors. If all sensors operate together, there will be a lot of redundant information, which will increase operating costs. Therefore, how to achieve optimal coordination between various sensors is an urgent problem to be solved. Summary of the Invention

[0009] The technical problem solved by this invention is to provide an integrated highway perception system based on the Internet of Things, which can adapt to different weather conditions and different occasions to perform highway network perception.

[0010] The basic solution provided by this invention is an integrated highway perception system based on the Internet of Things, including a server and various types of road network perception devices. The road network perception devices include laser radar, millimeter-wave radar, high-definition cameras, environmental detection units, and infrared cameras deployed at intervals. The server includes a meteorological recognition module, an environmental recognition module, and a data fusion module.

[0011] The weather recognition module is used to acquire video images captured by a high-definition camera and determine whether there are adverse conditions on the road based on the video from the high-definition camera. The adverse conditions include no light, low light, fog, rain, snow, and glare.

[0012] The equipment management module is used to control the high-definition camera to shut down and the infrared camera to start when the weather recognition module determines that the current road environment is adverse.

[0013] The environmental recognition module is used to identify the atmospheric transmittance of the current road based on the data measured by the environmental detection unit.

[0014] The equipment management module is also used to control the lidar to turn on when the atmospheric transmittance meets the preset conditions, and to control the millimeter-wave radar to turn on when the preset conditions are not met.

[0015] The data fusion module is used to fuse the perception data acquired by the activated devices.

[0016] The principle and advantages of this invention are as follows: By identifying meteorological and environmental conditions, and focusing on those significantly affected by environmental and meteorological factors, the invention determines whether the current environment and weather are suitable for the device's operation. If the current environment is suitable, the device is activated and road network information is collected. If the current weather and environment are unsuitable, the device is deactivated, and other devices are activated. For example, for a high-definition camera, the presence of adverse environmental conditions is identified by capturing images of the road surface. Since high-definition cameras are significantly affected by environmental factors, if adverse environmental conditions are detected affecting their operation, the high-definition camera is deactivated, and an infrared camera is activated instead. The specific method for identifying adverse environmental conditions can be achieved using existing image recognition technology. LiDAR is primarily affected by atmospheric transmittance; particulate matter and water vapor in the atmosphere affect laser transmittance. Therefore, an environmental detection unit measures the particulate matter and water vapor in the atmosphere to determine the current atmospheric transmittance, thereby determining whether the LiDAR is usable in the current environment. If usable, the LiDAR is activated; otherwise, the millimeter-wave radar is activated.

[0017] Furthermore, the data fusion module includes a data acquisition module and a first fusion module;

[0018] The data acquisition module is used to identify the currently activated road network sensing devices and acquire their sensing information;

[0019] The first fusion module is used when the currently activated road network sensing devices are LiDAR and HD cameras:

[0020] The system acquires vehicle trajectory information using LiDAR and vehicle information using video footage captured by a high-definition camera. It then binds the vehicle trajectory information with the vehicle information to generate binding information and tracks the vehicle based on this binding information.

[0021] Furthermore, the data fusion module also includes a second fusion module;

[0022] The second fusion module is used when the currently activated road network sensing devices are millimeter-wave radar and high-definition cameras:

[0023] The system acquires vehicle location information using millimeter-wave radar and vehicle information using video footage captured by high-definition cameras. Based on preset time intervals and the location and vehicle information at different intervals, it determines the movement trajectory of each vehicle, binds the movement trajectory with the vehicle information to generate binding information, and tracks each vehicle based on the binding information.

[0024] Furthermore, the data fusion module also includes a third fusion module;

[0025] The third fusion module is used when the currently activated road network sensing devices are LiDAR and infrared cameras:

[0026] Vehicle trajectories are acquired using LiDAR, vehicle outlines are obtained from images captured by infrared cameras, individual vehicles are identified based on the vehicle outlines, individual vehicles are bound to vehicle trajectories to generate binding information, and vehicles are tracked based on the binding information.

[0027] Furthermore, the data fusion module also includes a fourth fusion module;

[0028] The fourth fusion module is used when the currently activated road network sensing devices are millimeter-wave radar and infrared cameras:

[0029] The system acquires vehicle location information using millimeter-wave radar, obtains vehicle outlines using images captured by infrared cameras, identifies individual vehicles based on these outlines, determines the movement trajectory of each vehicle based on preset time intervals and location and vehicle information at different time intervals, binds individual vehicles to their trajectories to generate binding information, and tracks vehicles based on this binding information.

[0030] High-definition cameras capture video images that can identify license plates and vehicle models, as well as determine vehicle information. Infrared cameras, however, have lower image resolution and cannot identify license plates or other vehicle information. Therefore, when using infrared cameras, vehicle outlines are identified in the image to determine the individual vehicle, and then trajectory or location information is linked to the individual vehicle for tracking. LiDAR can directly acquire and scan vehicle point clouds to obtain trajectory information, while millimeter-wave radar determines vehicle location information by reading the OBU / CPC card via radio frequency. Therefore, when using millimeter-wave radar, vehicle trajectory information is identified by periodic vehicle positioning.

[0031] Furthermore, the data fusion module also includes a data verification module, which includes a road segment identification module, an overlap identification module, and an information identification module;

[0032] The road segment identification module is used to identify different road segments when adjacent and activated road network sensing devices exist, and to mark the road segments as identified road segments.

[0033] The overlap identification module is used to acquire the overlapping part within the identification range of the upstream and downstream network sensing devices in the identified road segment;

[0034] The information recognition module is used to mark the binding information generated by the previous road network sensing device for the vehicle as the first binding information and the binding information generated by the next road network sensing device for the vehicle as the second binding information when the vehicle enters the overlapping area. It also identifies the differences between the first and second binding information. When the difference reaches a preset standard and the number of different vehicles that meet the preset standard exceeds a threshold, the module determines the subject of the difference based on the different road network sensing devices activated and the environment of the identified road segment. If the subject of the difference is the previous road network sensing device, the binding information uploaded by the previous road network sensing device is corrected. If the subject of the difference is the next road network sensing device, the device activated by the next road network sensing device is adjusted to be consistent with the previous road network sensing device.

[0035] Because road network sensing devices are deployed at intervals, and due to differences in road conditions and environmental changes, the sensing devices activated in one road segment may differ from those activated in the next. When two adjacent road segments have different activated sensing devices, vehicles will be tracked using different sensing methods upon entering the next segment. Therefore, it is necessary to verify the feasibility of switching between these methods. Since sensing is performed using different devices, there will inherent differences. For example, the vehicle trajectory drawn from location information and the vehicle trajectory obtained from laser scanning point clouds will differ. However, if both measured data are accurate, it indicates that the data measured by the sensing devices on both road segments are accurate. If the difference is significant, it indicates that one of the data is inaccurate, thus requiring identification of the source of the discrepancy. Based on the current road environment, since the two road segments are adjacent, even if the environments differ, they are similar. It's possible that the environment is within the critical range for switching sensing devices. For example, the atmospheric transmittance required for LiDAR might be 0.6, the median for the previous segment is 0.55, and for the next segment it's 0.6. If the previous segment doesn't have LiDAR enabled, while the next segment does, the next segment has just reached the critical point and still has measurement inaccuracies. Therefore, the sensing device in the next segment is the main difference. In this case, the sensing device enabled in the next segment should be adjusted to match the previous segment, the LiDAR should be turned off, and the millimeter-wave radar should be enabled. Conversely, if the previous segment has LiDAR enabled, and the next segment doesn't, then the sensing device in the previous segment is the main difference, and the uploaded binding information contains errors, which need to be corrected. This method verifies the binding information and adjusts improperly enabled sensing devices. Attached Figure Description

[0036] Figure 1 This is a logic block diagram of an embodiment of the Internet of Things-based integrated highway sensing system of the present invention. Detailed Implementation

[0037] The following detailed description illustrates the specific implementation method:

[0038] The basic implementation examples are as follows: Figure 1 As shown:

[0039] The Internet of Things-based integrated highway sensing system includes a server and various types of road network sensing devices. The road network sensing devices include spaced-out lidar, millimeter-wave radar, high-definition cameras, environmental detection units, and infrared cameras. The server includes a meteorological recognition module, an environmental recognition module, and a data fusion module.

[0040] The weather recognition module is used to acquire video images captured by a high-definition camera and determine whether there are adverse conditions on the road based on the video from the high-definition camera. The adverse conditions include no light, low light, fog, rain, snow, and glare.

[0041] The equipment management module is used to control the high-definition camera to shut down and the infrared camera to start when the weather recognition module determines that the current road environment is adverse.

[0042] The environmental recognition module is used to identify the atmospheric transmittance of the current road based on the data measured by the environmental detection unit.

[0043] The equipment management module is also used to control the lidar to turn on when the atmospheric transmittance meets the preset conditions, and to control the millimeter-wave radar to turn on when the preset conditions are not met.

[0044] The data fusion module is used to fuse the perception data acquired by the activated devices.

[0045] The principle and advantages of this invention are as follows: By identifying meteorological and environmental conditions, and focusing on those significantly affected by environmental and meteorological factors, the invention determines whether the current environment and weather are suitable for the device's operation. If the current environment is suitable, the device is activated and road network information is collected. If the current weather and environment are unsuitable, the device is deactivated, and other devices are activated. For example, for a high-definition camera, the presence of adverse environmental conditions is identified by capturing images of the road surface. Since high-definition cameras are significantly affected by environmental factors, if adverse environmental conditions are detected affecting their operation, the high-definition camera is deactivated, and an infrared camera is activated instead. The specific method for identifying adverse environmental conditions can be achieved using existing image recognition technology. LiDAR is primarily affected by atmospheric transmittance; particulate matter and water vapor in the atmosphere affect laser transmittance. Therefore, an environmental detection unit measures the particulate matter and water vapor in the atmosphere to determine the current atmospheric transmittance, thereby determining whether the LiDAR is usable in the current environment. If usable, the LiDAR is activated; otherwise, the millimeter-wave radar is activated.

[0046] The data fusion module includes a data acquisition module and a first fusion module;

[0047] The data acquisition module is used to identify the currently activated road network sensing devices and acquire their sensing information;

[0048] The first fusion module is used when the currently activated road network sensing devices are LiDAR and HD cameras:

[0049] The system acquires vehicle trajectory information using LiDAR and vehicle information using video footage captured by a high-definition camera. It then binds the vehicle trajectory information with the vehicle information to generate binding information and tracks the vehicle based on this binding information.

[0050] The data fusion module further includes a second fusion module;

[0051] The second fusion module is used when the currently activated road network sensing devices are millimeter-wave radar and high-definition cameras:

[0052] The system acquires vehicle location information using millimeter-wave radar and vehicle information using video footage captured by high-definition cameras. Based on preset time intervals and the location and vehicle information at different intervals, it determines the movement trajectory of each vehicle, binds the movement trajectory with the vehicle information to generate binding information, and tracks each vehicle based on the binding information.

[0053] The data fusion module also includes a third fusion module;

[0054] The third fusion module is used when the currently activated road network sensing devices are LiDAR and infrared cameras:

[0055] Vehicle trajectories are acquired using LiDAR, vehicle outlines are obtained from images captured by infrared cameras, individual vehicles are identified based on the vehicle outlines, individual vehicles are bound to vehicle trajectories to generate binding information, and vehicles are tracked based on the binding information.

[0056] The data fusion module also includes a fourth fusion module;

[0057] The fourth fusion module is used when the currently activated road network sensing devices are millimeter-wave radar and infrared cameras:

[0058] The system acquires vehicle location information using millimeter-wave radar, obtains vehicle outlines using images captured by infrared cameras, identifies individual vehicles based on these outlines, determines the movement trajectory of each vehicle based on preset time intervals and location and vehicle information at different time intervals, binds individual vehicles to their trajectories to generate binding information, and tracks vehicles based on this binding information.

[0059] High-definition cameras capture video images that can identify license plates and vehicle models, as well as determine vehicle information. Infrared cameras, however, have lower image resolution and cannot identify license plates or other vehicle information. Therefore, when using infrared cameras, vehicle outlines are identified in the image to determine the individual vehicle, and then trajectory or location information is linked to the individual vehicle for tracking. LiDAR can directly acquire and scan vehicle point clouds to obtain trajectory information, while millimeter-wave radar determines vehicle location information by reading the OBU / CPC card via radio frequency. Therefore, when using millimeter-wave radar, vehicle trajectory information is identified by periodic vehicle positioning.

[0060] The data fusion module also includes a data verification module, which includes a road segment identification module, an overlap identification module, and an information identification module.

[0061] The road segment identification module is used to identify different road segments when adjacent and activated road network sensing devices exist, and to mark the road segments as identified road segments.

[0062] The overlap identification module is used to acquire the overlapping part within the identification range of the upstream and downstream network sensing devices in the identified road segment;

[0063] The information recognition module is used to mark the binding information generated by the previous road network sensing device for the vehicle as the first binding information and the binding information generated by the next road network sensing device for the vehicle as the second binding information when the vehicle enters the overlapping area. It also identifies the differences between the first and second binding information. When the difference reaches a preset standard and the number of different vehicles that meet the preset standard exceeds a threshold, the module determines the subject of the difference based on the different road network sensing devices activated and the environment of the identified road segment. If the subject of the difference is the previous road network sensing device, the binding information uploaded by the previous road network sensing device is corrected. If the subject of the difference is the next road network sensing device, the device activated by the next road network sensing device is adjusted to be consistent with the previous road network sensing device.

[0064] Because road network sensing devices are deployed at intervals, and due to differences in road conditions and environmental changes, the sensing devices activated in one road segment may differ from those activated in the next. When two adjacent road segments have different activated sensing devices, vehicles will be tracked using different sensing methods upon entering the next segment. Therefore, it is necessary to verify the feasibility of switching between these methods. Since sensing is performed using different devices, there will inherent differences. For example, the vehicle trajectory drawn from location information and the vehicle trajectory obtained from laser scanning point clouds will differ. However, if both measured data are accurate, it indicates that the data measured by the sensing devices on both road segments are accurate. If the difference is significant, it indicates that one of the data is inaccurate, thus requiring identification of the source of the discrepancy. Based on the current road environment, since the two road segments are adjacent, their environments, even if different, are similar. It's possible the environment is within the critical range for switching sensing devices. For example, the atmospheric transmittance required for LiDAR might be 0.6, the median for the previous segment is 0.55, and for the next segment it's 0.6. If the previous segment didn't have LiDAR enabled, while the next segment did, the next segment has just reached the critical point and still has measurement inaccuracies. Therefore, the sensing device in the next segment is the main difference. In this case, the sensing device enabled in the next segment should be adjusted to match the previous segment, the LiDAR should be turned off, and the millimeter-wave radar should be enabled. Conversely, if the previous segment has LiDAR enabled, but the next segment doesn't, then the sensing device in the previous segment is the main difference, and the uploaded binding information contains errors, which need to be corrected.

[0065] The above are merely embodiments of the present invention. Commonly known structures and characteristics are not described in detail here. Those skilled in the art are aware of all common technical knowledge in the field prior to the application date or priority date, are aware of all existing technologies in that field, and have the ability to apply conventional experimental methods prior to that date. Those skilled in the art can, under the guidance of this application, improve and implement this solution in combination with their own capabilities. Some typical known structures or methods should not be obstacles for those skilled in the art to implement this application. It should be noted that those skilled in the art can make several modifications and improvements without departing from the structure of the present invention. These should also be considered within the scope of protection of the present invention, and will not affect the effectiveness of the implementation of the present invention or the practicality of the patent. The scope of protection claimed in this application should be determined by the content of its claims, and the specific embodiments described in the specification can be used to interpret the content of the claims.

Claims

1. An integrated highway sensing system based on the Internet of Things, characterized in that: It includes a server and various types of road network sensing devices, including spaced-out lidar, millimeter-wave radar, high-definition cameras, environmental detection units, and infrared cameras. The server includes a meteorological recognition module, an environmental recognition module, and a data fusion module. The weather recognition module is used to acquire video images captured by a high-definition camera and determine whether there are adverse conditions on the road based on the video from the high-definition camera. The adverse conditions include no light, low light, fog, rain, snow, and glare. The equipment management module is used to control the high-definition camera to shut down and the infrared camera to start when the weather recognition module determines that the current road environment is adverse. The environmental recognition module is used to identify the atmospheric transmittance of the current road based on the data measured by the environmental detection unit. The equipment management module is also used to control the lidar to turn on when the atmospheric transmittance meets the preset conditions, and to control the millimeter-wave radar to turn on when the preset conditions are not met. The data fusion module is used to fuse the sensing information acquired by the activated devices to obtain binding information for tracking road vehicles; The data fusion module also includes a data verification module, which includes a road segment identification module, an overlap identification module, and an information identification module. The road segment identification module is used to identify different road segments when adjacent and activated road network sensing devices exist, and to mark the road segments as identified road segments. The overlap identification module is used to acquire the overlapping part within the identification range of the upstream and downstream network sensing devices in the identified road segment; The information recognition module is used to mark the binding information generated by the previous road network sensing device for the vehicle as the first binding information and the binding information generated by the next road network sensing device for the vehicle as the second binding information when the vehicle enters the overlapping area. It also identifies the differences between the first and second binding information. When the difference reaches a preset standard and the number of different vehicles that meet the preset standard exceeds a threshold, the module determines the subject of the difference based on the different road network sensing devices activated and the environment of the identified road segment. If the subject of the difference is the previous road network sensing device, the binding information uploaded by the previous road network sensing device is corrected. If the subject of the difference is the next road network sensing device, the device activated by the next road network sensing device is adjusted to be consistent with the previous road network sensing device.

2. The integrated highway sensing system based on the Internet of Things according to claim 1, characterized in that: The data fusion module includes a data acquisition module and a first fusion module; The data acquisition module is used to identify the currently activated road network sensing devices and acquire their sensing information; The first fusion module is used when the currently activated road network sensing devices are LiDAR and HD cameras: The system acquires vehicle trajectory information using LiDAR and vehicle information using video footage captured by a high-definition camera. It then binds the vehicle trajectory information with the vehicle information to generate binding information and tracks the vehicle based on this binding information.

3. The integrated highway sensing system based on the Internet of Things according to claim 2, characterized in that: The data fusion module further includes a second fusion module; The second fusion module is used when the currently activated road network sensing devices are millimeter-wave radar and high-definition cameras: The system acquires vehicle location information using millimeter-wave radar and vehicle information using video footage captured by high-definition cameras. Based on preset time intervals and the location and vehicle information at different intervals, it determines the movement trajectory of each vehicle, binds the movement trajectory with the vehicle information to generate binding information, and tracks each vehicle based on the binding information.

4. The integrated highway sensing system based on the Internet of Things according to claim 3, characterized in that: The data fusion module also includes a third fusion module; The third fusion module is used when the currently activated road network sensing devices are LiDAR and infrared cameras: Vehicle trajectories are acquired using LiDAR, vehicle outlines are obtained from images captured by infrared cameras, individual vehicles are identified based on the vehicle outlines, individual vehicles are bound to vehicle trajectories to generate binding information, and vehicles are tracked based on the binding information.

5. The integrated highway sensing system based on the Internet of Things according to claim 4, characterized in that: The data fusion module also includes a fourth fusion module; The fourth fusion module is used when the currently activated road network sensing devices are millimeter-wave radar and infrared cameras: The system acquires vehicle location information using millimeter-wave radar, obtains vehicle outlines using images captured by infrared cameras, identifies individual vehicles based on these outlines, determines the movement trajectory of each vehicle based on preset time intervals and location and vehicle information at different time intervals, binds individual vehicles to their trajectories to generate binding information, and tracks vehicles based on this binding information.

Images