Road perception integrated system based on Internet of Things

Abstract

The invention relates to the technical field of road network perception, in particular to a road perception integrated system based on the Internet of Things. Comprising a server and various types of road network sensing devices, each road network sensing device comprises a laser radar, a millimeter-wave radar, a high-definition camera, an environment detection unit and an infrared camera which are arranged at intervals, and the server comprises a meteorological recognition module which is used for obtaining video images shot by the high-definition camera and sending the video images to the server; judging whether the road environment has a severe environment or not according to the video of the high-definition camera; the equipment management module is used for controlling the high-definition camera to be turned off and turning on the infrared camera when the weather identification module judges that the current road environment has a severe environment; the environment identification module is used for identifying the atmospheric transmittance of the current road according to the data detected by the environment detection unit; and the equipment management module is also used for controlling the laser radar to be turned on when the atmospheric transmissivity meets a preset condition, and controlling the millimeter wave radar to be turned on when the atmospheric transmissivity does not meet the preset condition.

Description

Technical Field

[0001] The present invention relates to the field of road network perception technology, and in particular to an integrated highway perception system based on the Internet of Things. Background Art

[0002] Traffic sensors are mainly used to collect data about road conditions and provide raw data for roadside perception networks. In recent years, traffic sensors have gone through a series of technological developments, including coils, geomagnetism, video, radar, etc. It is reported that existing traffic sensors are mainly divided into two categories: the first category is traditional sensors, including induction coils, cross-sectional radars, and geomagnetic spikes, etc. These devices are usually used to obtain vehicle speed information of a certain cross section or instantaneous lane; the second category is traffic target sensors, mainly including AI cameras, lidars, and traffic millimeter radars.

[0003] AI cameras are mainly used to detect illegal parking, vehicles driving in the wrong direction, road congestion, roadside pedestrians, road spills, etc. However, the accuracy of vehicle trajectory tracking, vehicle position and speed detection is relatively weak, and it is easily affected by weather and environmental factors, and the detection range is relatively small.

[0004] Radars can be divided into over-the-horizon radars, microwave radars, millimeter-wave radars, and laser radars according to their operating frequency bands. The advantage of radars is that they can detect distant targets both day and night, and are basically not blocked by fog, clouds, and rain. They have the characteristics of all-weather and all-day, and have certain penetration capabilities.

[0005] Compared with ordinary microwave radar, millimeter wave radar has the characteristics of small size, easy integration and high spatial resolution. It has strong ability to penetrate fog, smoke and dust, strong anti-interference ability, and can be used all day and all day (except heavy rainy days).

[0006] Compared with ordinary microwave radar, laser radar has the advantages of high resolution, strong anti-active interference ability, small size and light weight. However, it is greatly affected by weather and atmosphere. In heavy rain, thick smoke, fog and other weather conditions, the attenuation increases sharply, the propagation distance is greatly affected, and the search and capture target range is narrow, and the cost is relatively high.

[0007] Infrared thermal imaging technology has the characteristics of penetrating haze, overcoming rain and snow, and resisting glare. In severe conditions such as no light or weak light, haze, rain and snow, and glare, ordinary visible light cameras basically lose their monitoring capabilities, while infrared thermal imagers are not subject to interference and can monitor the real-time road conditions, vehicle conditions, and safety of public facilities on highways.

[0008] Therefore, all types of sensors on highways have their own characteristics and limitations. A single sensor cannot achieve all-weather, high-precision perception. Multiple sensors need to complement each other and work together to build a perception system that can adapt to different weather and different occasions. However, the perception data of various sensors overlap. If all sensors operate together, there will be more redundant information and higher operating costs. Therefore, how to achieve the best coordination between sensors is an urgent problem to be solved. Summary of the invention

[0009] The technical problem solved by the present invention is to provide an integrated highway perception system based on the Internet of Things, which can adapt to different weather and different occasions to perform highway road network perception.

[0010] The basic solution provided by the present invention is: an integrated highway perception system based on the Internet of Things, including a server and various types of road network perception equipment, wherein the road network perception equipment includes laser radars, millimeter wave radars, high-definition cameras, environmental detection units and infrared cameras arranged at intervals, and the server includes a meteorological recognition module, an environmental recognition module and a data fusion module; A weather recognition module is used to obtain video images taken by a high-definition camera and determine whether the road environment is in a harsh environment based on the video of the high-definition camera. The harsh environment includes no light, weak light, haze, rain, snow, and glare. The equipment management module is used to control the high-definition camera to shut down and start the infrared camera when the weather recognition module determines that the current road environment is in a bad environment; An environment recognition module, used to identify the atmospheric transmittance of the current road based on the data measured by the environment detection unit; The device management module is also used to control the laser radar to turn on when the atmospheric transmittance meets the preset conditions, and to control the millimeter wave radar to turn on when the preset conditions are not met; The data fusion module is used to fuse the perceptions obtained by the turned-on devices.

[0011] The principle and advantage of the present invention are: by identifying the weather and the environment, identifying the environment and the weather, identifying whether the current environment and weather are suitable for its work, if the current environment is suitable for its work, then control this type of equipment to start, and collect road network information, if the current weather and environment are not suitable for this type of equipment to work, then control this type of equipment to shut down, and make other equipment work. For example, for a high-definition camera, the road surface is photographed by a high-definition camera to identify whether there is a bad environment at present. Since the high-definition camera is greatly affected by environmental factors, when it is identified that there is a bad environment, which affects the work of the high-definition camera, the high-definition camera is controlled to shut down and the infrared camera is activated instead. The specific identification method of the bad environment can be completed by the existing image recognition technology. The laser radar is mainly affected by the atmospheric transmittance. The particles and water vapor in the atmosphere will affect the transmittance of the laser. Therefore, the particles and water vapor in the current atmosphere are measured by the environmental detection unit to determine the current atmospheric transmittance, and then determine whether the laser radar is available in the current environment. If it is available, the laser radar is started. If it is not available, the millimeter wave radar is controlled to start.

[0012] Further, the data fusion module includes a data acquisition module and a first fusion module; The data acquisition module is used to identify the currently activated road network perception device and obtain its perception information; The first fusion module is used when the currently activated road network perception device is a laser radar and a high-definition camera: The vehicle's trajectory information is obtained through laser radar, and the vehicle information of the vehicle is obtained through video images taken by a high-definition camera. The vehicle's trajectory information is bound to the vehicle information to generate binding information, and the vehicle is tracked based on the binding information.

[0013] Further, the data fusion module also includes a second fusion module; The second fusion module is used when the currently activated road network perception device is a millimeter wave radar and a high-definition camera: The vehicle's positioning information is obtained through millimeter-wave radar, and the vehicle information is obtained through video images taken by high-definition cameras. The movement trajectory of each vehicle is determined based on the positioning information and vehicle information at different time intervals according to preset time intervals. The movement trajectory is bound to the vehicle information to generate binding information, and each vehicle is tracked based on the binding information.

[0014] Further, the data fusion module also includes a third fusion module; The third fusion module is used when the currently activated road network perception device is a laser radar and an infrared camera: The vehicle trajectory is obtained through the laser radar, the vehicle outline is obtained through the shooting picture of the infrared camera, the individual vehicle is determined based on the vehicle outline, the individual vehicle is bound to the vehicle trajectory to generate binding information, and the vehicle is tracked based on the binding information.

[0015] Further, the data fusion module also includes a fourth fusion module; The fourth fusion module is used when the currently activated road network perception equipment is a millimeter wave radar and an infrared camera: The vehicle's positioning information is obtained through millimeter-wave radar, and the vehicle's outline is obtained through the shooting image of the infrared camera. The individual vehicle is determined based on the vehicle outline. According to the preset time interval, the positioning information and vehicle information at different time intervals are used to determine the movement trajectory of each vehicle. The individual vehicle is bound to the vehicle trajectory to generate binding information, and the vehicle is tracked based on the binding information.

[0016] The video images captured by high-definition cameras can identify license plates and vehicle models, as well as determine vehicle information, while infrared cameras have low image accuracy and cannot identify vehicle information such as license plates. Therefore, when using infrared cameras, the individual vehicle is identified by identifying the vehicle outline in the image, and then the trajectory information or positioning information is bound to the individual vehicle to track the vehicle. LiDAR can directly obtain trajectory information by scanning the vehicle point cloud, while millimeter-wave radar determines the vehicle's positioning information by reading the OBU / CPC card through radio frequency. Therefore, when using millimeter-wave radar, the vehicle's trajectory information is identified by locating the vehicle at intervals.

[0017] Furthermore, the data fusion module further includes a data verification module, and the data verification module includes a road section identification module, an overlap identification module, and an information identification module; A road section identification module is used to obtain different road sections where adjacent and enabled road network sensing devices exist, and mark the road section as an identified road section; An overlapping recognition module is used to obtain the overlapping part within the recognition range of the previous road network sensing device and the next road network sensing device in the recognition section; The information identification module is used to mark the binding information generated by the previous road network perception device for the vehicle as the first binding information, mark the binding information generated by the next road network perception device for the vehicle as the second binding information, and identify the difference between the first binding information and the second binding information when the difference reaches a preset standard and the number of different vehicle identifications that meet the preset standard exceeds a threshold number of times, determine the difference subject according to the different road network perception devices started and the environment of the identified road section, if the difference subject is the previous road network perception device, correct the binding information uploaded by the previous road network perception device, if the difference subject is the next road network perception device, adjust the device turned on by the next road network perception device to be consistent with the previous road network perception device.

[0018] Since the road network sensing devices are arranged at intervals, the conditions of each road section are different, the environment changes, and other factors, the sensing devices turned on in the previous section may be different from the sensing devices turned on in the next section. When the sensing devices turned on in two adjacent sections are different, that is, after the vehicle enters the next section, it will be tracked through different sensing methods, so it is necessary to verify the feasibility of switching at this time. Since perception is performed through different devices, there will be certain differences. For example, there will be certain differences between the vehicle trajectory drawn according to the positioning information and the vehicle trajectory obtained by the laser scanning point cloud. However, if both measured data are accurate, it means that the data measured by the sensing devices on the two sections are accurate. If the difference is large, it means that one of the data is inaccurate, so it is necessary to identify which one is the main difference. According to the current road environment, since the two sections are adjacent, the environments are similar even if they are different. This may be because the environment is in the critical interval for switching perception devices. For example, the atmospheric transmittance required by the laser radar is 0.6, the median of the previous section is 0.55, and the next section is 0.6. The previous section does not have the laser radar turned on, while the next section has the laser radar turned on. At this time, the next section has just reached the critical point and there is still an inaccurate measurement problem. Therefore, the perception device of the next section is the difference subject. At this time, the perception device turned on in the next section is adjusted to be consistent with the previous section, the laser radar is turned off, and the millimeter wave radar is turned on. On the contrary, if the laser radar is turned on in the previous section and the laser radar is not turned on in the next section, the perception device of the previous section is the difference subject, and there is an error in the uploaded binding information, which needs to be corrected. In this way, while verifying the binding information, the unreasonable perception device can also be adjusted. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a logical block diagram of an embodiment of the highway perception integrated system based on the Internet of Things of the present invention. DETAILED DESCRIPTION

[0020] The following is further described in detail through specific implementation methods: The embodiment is basically as shown in the attached Figure 1 As shown: The highway perception integrated system based on the Internet of Things includes a server and various types of road network perception equipment, wherein the road network perception equipment includes laser radars, millimeter wave radars, high-definition cameras, environmental detection units and infrared cameras arranged at intervals, and the server includes a meteorological recognition module, an environmental recognition module and a data fusion module; A weather recognition module is used to obtain video images taken by a high-definition camera and determine whether the road environment is in a harsh environment based on the video of the high-definition camera. The harsh environment includes no light, weak light, haze, rain, snow, and glare. The equipment management module is used to control the high-definition camera to shut down and start the infrared camera when the weather recognition module determines that the current road environment is in a bad environment; An environment recognition module, used to identify the atmospheric transmittance of the current road based on the data measured by the environment detection unit; The device management module is also used to control the laser radar to turn on when the atmospheric transmittance meets the preset conditions, and to control the millimeter wave radar to turn on when the preset conditions are not met; The data fusion module is used to fuse the perceptions obtained by the turned-on devices.

[0021] The principle and advantage of the present invention are: by identifying the weather and the environment, identifying the environment and the weather, identifying whether the current environment and weather are suitable for its work, if the current environment is suitable for its work, then control this type of equipment to start, and collect road network information, if the current weather and environment are not suitable for this type of equipment to work, then control this type of equipment to shut down, and make other equipment work. For example, for a high-definition camera, the road surface is photographed by a high-definition camera to identify whether there is a bad environment at present. Since the high-definition camera is greatly affected by environmental factors, when it is identified that there is a bad environment, which affects the work of the high-definition camera, the high-definition camera is controlled to shut down and the infrared camera is activated instead. The specific identification method of the bad environment can be completed by the existing image recognition technology. The laser radar is mainly affected by the atmospheric transmittance. The particles and water vapor in the atmosphere will affect the transmittance of the laser. Therefore, the particles and water vapor in the current atmosphere are measured by the environmental detection unit to determine the current atmospheric transmittance, and then determine whether the laser radar is available in the current environment. If it is available, the laser radar is started. If it is not available, the millimeter wave radar is controlled to start.

[0022] The data fusion module includes a data acquisition module and a first fusion module; The data acquisition module is used to identify the currently activated road network perception device and obtain its perception information; The first fusion module is used when the currently activated road network perception device is a laser radar and a high-definition camera: The vehicle's trajectory information is obtained through laser radar, and the vehicle information of the vehicle is obtained through video images taken by a high-definition camera. The vehicle's trajectory information is bound to the vehicle information to generate binding information, and the vehicle is tracked based on the binding information.

[0023] The data fusion module also includes a second fusion module; The second fusion module is used when the currently activated road network perception device is a millimeter wave radar and a high-definition camera: The vehicle's positioning information is obtained through millimeter-wave radar, and the vehicle information is obtained through video images taken by high-definition cameras. The movement trajectory of each vehicle is determined based on the positioning information and vehicle information at different time intervals according to preset time intervals. The movement trajectory is bound to the vehicle information to generate binding information, and each vehicle is tracked based on the binding information.

[0024] The data fusion module also includes a third fusion module; The third fusion module is used when the currently activated road network perception device is a laser radar and an infrared camera: The vehicle trajectory is obtained through the laser radar, the vehicle outline is obtained through the shooting picture of the infrared camera, the individual vehicle is determined based on the vehicle outline, the individual vehicle is bound to the vehicle trajectory to generate binding information, and the vehicle is tracked based on the binding information.

[0025] The data fusion module also includes a fourth fusion module; The fourth fusion module is used when the currently activated road network perception equipment is a millimeter wave radar and an infrared camera: The vehicle's positioning information is obtained through millimeter-wave radar, and the vehicle's outline is obtained through the shooting image of the infrared camera. The individual vehicle is determined based on the vehicle outline. According to the preset time interval, the positioning information and vehicle information at different time intervals are used to determine the movement trajectory of each vehicle. The individual vehicle is bound to the vehicle trajectory to generate binding information, and the vehicle is tracked based on the binding information.

[0026] The video images captured by high-definition cameras can identify license plates and vehicle models, as well as determine vehicle information, while infrared cameras have low image accuracy and cannot identify vehicle information such as license plates. Therefore, when using infrared cameras, the individual vehicle is identified by identifying the vehicle outline in the image, and then the trajectory information or positioning information is bound to the individual vehicle to track the vehicle. LiDAR can directly obtain trajectory information by scanning the vehicle point cloud, while millimeter-wave radar determines the vehicle's positioning information by reading the OBU / CPC card through radio frequency. Therefore, when using millimeter-wave radar, the vehicle's trajectory information is identified by locating the vehicle at intervals.

[0027] The data fusion module also includes a data verification module, which includes a road section identification module, an overlap identification module, and an information identification module; A road section identification module is used to obtain different road sections where adjacent and enabled road network sensing devices exist, and mark the road section as an identified road section; An overlapping recognition module is used to obtain the overlapping part within the recognition range of the previous road network sensing device and the next road network sensing device in the recognition section; The information identification module is used to mark the binding information generated by the previous road network perception device for the vehicle as the first binding information, mark the binding information generated by the next road network perception device for the vehicle as the second binding information, and identify the difference between the first binding information and the second binding information when the difference reaches a preset standard and the number of different vehicle identifications that meet the preset standard exceeds a threshold number of times, determine the difference subject according to the different road network perception devices started and the environment of the identified road section, if the difference subject is the previous road network perception device, correct the binding information uploaded by the previous road network perception device, if the difference subject is the next road network perception device, adjust the device turned on by the next road network perception device to be consistent with the previous road network perception device.

[0028] Since the road network sensing devices are arranged at intervals, the conditions of each road section are different, the environment changes, and other factors, the sensing devices turned on in the previous section may be different from the sensing devices turned on in the next section. When the sensing devices turned on in two adjacent sections are different, that is, after the vehicle enters the next section, it will be tracked through different sensing methods, so it is necessary to verify the feasibility of switching at this time. Since perception is performed through different devices, there will be certain differences. For example, there will be certain differences between the vehicle trajectory drawn according to the positioning information and the vehicle trajectory obtained by the laser scanning point cloud. However, if both measured data are accurate, it means that the data measured by the sensing devices on the two sections are accurate. If the difference is large, it means that one of the data is inaccurate, so it is necessary to identify which one is the main difference. According to the current road environment, since the two sections are adjacent, the environments are similar even if they are different. This may be because the environment is in the critical interval for switching perception devices. For example, the atmospheric transmittance required by the laser radar is 0.6, the median of the previous section is 0.55, and the next section is 0.6. The previous section does not have the laser radar turned on, while the next section has the laser radar turned on. At this time, the next section has just reached the critical point and there is still an inaccurate measurement problem. Therefore, the perception device of the next section is the difference subject. At this time, the perception device turned on in the next section is adjusted to be consistent with the previous section, the laser radar is turned off, and the millimeter wave radar is turned on. On the contrary, if the laser radar is turned on in the previous section and the laser radar is not turned on in the next section, the perception device of the previous section is the difference subject, and there is an error in the uploaded binding information, which needs to be corrected.

[0029] The above are only embodiments of the present invention. Common knowledge such as the known specific structures and characteristics in the scheme is not described in detail here. Ordinary technicians in the relevant field know all the common technical knowledge in the technical field to which the invention belongs before the application date or priority date, can obtain all the existing technologies in the field, and have the ability to apply conventional experimental means before that date. Ordinary technicians in the relevant field can improve and implement this scheme in combination with their own abilities under the enlightenment given by this application. Some typical known structures or known methods should not become obstacles for ordinary technicians in the relevant field to implement this application. It should be pointed out that for those skilled in the art, without departing from the structure of the present invention, several deformations and improvements can be made, which should also be regarded as the scope of protection of the present invention, and these will not affect the effect of the implementation of the present invention and the practicality of the patent. The scope of protection required by this application shall be based on the content of its claims, and the specific implementation methods and other records in the specification can be used to interpret the content of the claims.

Claims

1. The highway perception integrated system based on the Internet of Things is characterized by: It includes a server and various types of road network sensing equipment, wherein the road network sensing equipment includes laser radars, millimeter wave radars, high-definition cameras, environmental detection units and infrared cameras arranged at intervals, and the server includes a meteorological recognition module, an environmental recognition module and a data fusion module; A weather recognition module is used to obtain video images taken by a high-definition camera and determine whether the road environment is in a harsh environment based on the video of the high-definition camera. The harsh environment includes no light, weak light, haze, rain, snow, and glare. The equipment management module is used to control the high-definition camera to shut down and start the infrared camera when the weather recognition module determines that the current road environment is in a bad environment; An environment recognition module, used to identify the atmospheric transmittance of the current road based on the data measured by the environment detection unit; The device management module is also used to control the laser radar to turn on when the atmospheric transmittance meets the preset conditions, and to control the millimeter wave radar to turn on when the preset conditions are not met; The data fusion module is used to fuse the perception information obtained by the turned-on devices to obtain binding information and track road vehicles; The data fusion module also includes a data verification module, which includes a road section identification module, an overlap identification module, and an information identification module; A road section identification module is used to obtain different road sections where adjacent and enabled road network sensing devices exist, and mark the road section as an identified road section; An overlapping recognition module is used to obtain the overlapping part within the recognition range of the previous road network sensing device and the next road network sensing device in the recognition section; The information identification module is used to mark the binding information generated by the previous road network perception device for the vehicle as the first binding information, mark the binding information generated by the next road network perception device for the vehicle as the second binding information, and identify the difference between the first binding information and the second binding information when the difference reaches a preset standard and the number of different vehicle identifications that meet the preset standard exceeds a threshold number of times, determine the difference subject according to the different road network perception devices started and the environment of the identified road section, if the difference subject is the previous road network perception device, correct the binding information uploaded by the previous road network perception device, if the difference subject is the next road network perception device, adjust the device turned on by the next road network perception device to be consistent with the previous road network perception device.

2. The highway perception integrated system based on the Internet of Things according to claim 1 is characterized in that: The data fusion module includes a data acquisition module and a first fusion module; The data acquisition module is used to identify the currently activated road network perception device and obtain its perception information; The first fusion module is used when the currently activated road network perception device is a laser radar and a high-definition camera: The vehicle's trajectory information is obtained through laser radar, and the vehicle information of the vehicle is obtained through video images taken by a high-definition camera. The vehicle's trajectory information is bound to the vehicle information to generate binding information, and the vehicle is tracked based on the binding information.

3. The highway perception integrated system based on the Internet of Things according to claim 2 is characterized in that: The data fusion module also includes a second fusion module; The second fusion module is used when the currently activated road network perception equipment is a millimeter wave radar and a high-definition camera: The vehicle's positioning information is obtained through millimeter-wave radar, and the vehicle information is obtained through video images taken by high-definition cameras. The movement trajectory of each vehicle is determined based on the positioning information and vehicle information at different time intervals according to preset time intervals. The movement trajectory is bound to the vehicle information to generate binding information, and each vehicle is tracked based on the binding information.

4. The highway perception integrated system based on the Internet of Things according to claim 3 is characterized in that: The data fusion module also includes a third fusion module; The third fusion module is used when the currently activated road network perception device is a laser radar and an infrared camera: The vehicle trajectory is obtained through the laser radar, the vehicle outline is obtained through the shooting picture of the infrared camera, the individual vehicle is determined based on the vehicle outline, the individual vehicle is bound to the vehicle trajectory to generate binding information, and the vehicle is tracked based on the binding information.

5. The highway perception integrated system based on the Internet of Things according to claim 4 is characterized in that: The data fusion module also includes a fourth fusion module; The fourth fusion module is used when the currently activated road network perception equipment is a millimeter wave radar and an infrared camera: The vehicle's positioning information is obtained through millimeter-wave radar, and the vehicle's outline is obtained through the shooting image of the infrared camera. The individual vehicle is determined based on the vehicle outline. According to the preset time interval, the positioning information and vehicle information at different time intervals are used to determine the movement trajectory of each vehicle. The individual vehicle is bound to the vehicle trajectory to generate binding information, and the vehicle is tracked based on the binding information.

Images