A pavement foreign object detection system

Abstract

The present invention discloses a pavement foreign object detection system, which includes a monitoring layer, a network layer, a communication layer and a device layer. The monitoring layer includes at least one monitor, a client and a server; the network layer includes a switch that is network-connected to the above-mentioned monitoring layer; the communication layer includes at least one industrial computer and a positioning module that are connected to the above-mentioned network layer; the device layer includes a detection unit corresponding to the industrial computer, and the detection unit includes a high-speed multi-spectral 3D camera, a color multi-spectral camera, a multi-spectral or polarizing lens and a light source. The present invention realizes high-precision and fast pavement foreign object detection and recognition through optoelectronic detection and machine vision recognition technologies.

Description

Technical Field

[0001] The present invention relates to the technical field of foreign object detection, and particularly to a road surface foreign object detection system. Background Art

[0002] Vehicles move very fast on highways. Therefore, foreign objects on the road surface can easily cause traffic accidents. In order to ensure the safety of vehicle driving, it is necessary to inspect the highway road surface to determine whether there are foreign objects or defects and deal with them in time. At the same time, airport runway safety is a major aviation safety issue. Foreign objects on airport runways have always been a killer to runway safety. Foreign Object Debris (FOD) on airport runways generally refers to some foreign substances that may damage aircraft or systems. There are quite a variety of FOD, such as aircraft and engine connectors (nuts, screws, washers, fuses, etc.), mechanical tools, flying objects (nails, personal documents, pens, pencils, etc.), wild animals, leaves, stones and sand, road surface materials, wooden blocks, plastic or polyethylene materials, paper products, ice debris in the operating area, and so on.

[0003] Currently, the detection of foreign objects on highways or airport runways often adopts the method of manual inspection. At fixed times, airport personnel drive cars to patrol on the road surface. However, this manual inspection method has a large human factor, low efficiency, and a very high probability of missing foreign objects. It cannot be carried out in an environment with poor visibility at night or in a bad weather environment, and there is no sufficient guarantee for flight safety. For the research and development of FOD detection systems, the main technical approach is to combine W-band detection radar and video systems, use the high-precision characteristics of this frequency band radar for different detection methods, and confirm through the corresponding video system to achieve real-time monitoring of FOD on airport runways. However, on the one hand, the above method has a huge cost; on the other hand, the use of electromagnetic wave radar is restricted in special environments, making it difficult to popularize. The present invention precisely solves the above problems and realizes high-precision and fast detection and identification of road surface foreign objects by adopting optoelectronic detection and machine vision recognition technologies. Summary of the Invention

[0004] In order to overcome the problems of high cost, low efficiency of manual inspection, and low precision of the existing road surface foreign object detection system, the present invention provides a road surface foreign object detection system. The specific technical solution is as follows:

[0005] It includes a monitoring layer, a network layer, a communication layer, and a device layer. Among them,

[0006] The monitoring layer includes at least one monitor, a client, and a server;

[0007] The network layer includes a switch that is network-connected to the monitoring layer;

[0008] The communication layer includes at least one industrial control computer and a positioning module connected to the network layer;

[0009] The device layer includes one or more groups of detection units corresponding to the industrial control computer, and the detection unit includes a high-speed multi-spectral 3D camera, a color multi-spectral area array camera, a multi-spectral or polarizing lens, and a light source.

[0010] Further, the industrial control computer is in 4 groups or 10 groups; the industrial control computer receives data collected by the high-speed multi-spectral 3D camera and the color multi-spectral area array camera through network cables and controls the high-speed multi-spectral 3D camera and the color multi-spectral area array camera.

[0011] Further, the detection unit further includes an encoder, a frequency divider, and a frequency-down conversion frequency division module;

[0012] Further, the high-speed multi-spectral 3D camera, the color multi-spectral area array camera, and the light source use digital high and low levels as external triggers.

[0013] Further, the positioning module is a GNSS global satellite positioning and navigation system and an inertial navigation system ISN.

[0014] Further, the positioning module has two working modes, namely the normal mode and the emergency mode. The normal working mode is the combination of the GNSS global satellite positioning and navigation system and the inertial navigation system ISN; the emergency mode is that after the GNSS global satellite positioning and navigation system fails, the inertial navigation system ISN works independently.

[0015] Further, the GNSS global satellite positioning and navigation system internally uses real-time kinematic carrier phase differential technology for positioning and calculation.

[0016] Further, the inertial navigation system ISN uses the DR (Dead Reckoning) algorithm for positioning and calculation.

[0017] Further, the time difference of the trigger level synchronization between the high-speed multi-spectral 3D cameras in the multiple groups of detection units is less than 50 ns; the time difference of the trigger level synchronization between the color multi-spectral area array camera and the light source is less than 50 ns; the level triggering the positioning module is synchronized with the level triggering the color multi-spectral area array camera, and the time difference is less than 50 ns

[0018] Further, the server and the at least one industrial control computer are connected by a switch with a bandwidth of not less than gigabit; the at least one industrial control computer and the corresponding detection unit are connected by a switch with a bandwidth of not less than gigabit.

[0019] Compared with the prior art, the present invention realizes high-precision and rapid detection and recognition of road surface foreign objects by adopting optoelectronic detection and machine vision recognition technologies. Description of the Drawings

[0020] Figure 1 It is a schematic diagram of the composition of the road surface foreign object detection system of the present invention;

[0021] Figure 2 It is a schematic diagram of the external trigger signal connection of the road surface foreign object detection system of the present invention; Detailed Embodiments

[0022] To make the technical means, creative features, achieved purposes and effects of the present invention easy to understand, the present invention will be further described below in conjunction with specific embodiments.

[0023] As Figure 1-2 shown, this embodiment provides a road surface foreign object detection system, including a monitoring layer, a network layer, a communication layer and a device layer. The monitoring layer includes at least one monitor, a client and a server; the network layer includes a switch network-connected to the monitoring layer; the communication layer includes a plurality of industrial computers and a positioning module connected to the network layer. In this embodiment, a positioning module is adopted, such as the GNSS global satellite positioning and navigation system and the inertial navigation system ISN, or the Beidou navigation module, the GPS navigation module, and the Galileo navigation module. When working in the normal mode, the global satellite positioning and navigation GNSS is combined with the inertial navigation INS; in the emergency mode, when there is no signal for navigation and positioning, the inertial navigation module is simply used for positioning. If the navigation module is not used, only a fixed route can be followed, and the efficiency is relatively low.

[0024] The above normal working mode adopts a mode of combining satellite positioning and navigation with encoder navigation. The working principle of satellite positioning and navigation is that the ground master control station collects the observation data and meteorological information of each monitoring station, calculates the ephemeris of each satellite and the satellite clock correction number, edits the navigation message in a specified format, and injects this information into the navigation satellite through the ground injection station. When measuring and positioning, the user can obtain the approximate positions of each satellite by using the stored ephemeris of the receiver. Based on these data and its own position, the computer selects four satellites with a relatively large included angle between the satellite and the user connection as the observation objects. During observation, the receiver performs correlation processing on the information generated by the code generator and the signal received by the satellite, and measures the pseudo-range between the user and the satellite according to the time stamp and sub-frame count of the navigation message. By listing three observation equations with the corrected pseudo-range, the input initial data and the observation values of the four satellites, the position of the receiver can be solved and the required coordinate system can be converted to achieve the positioning purpose.

[0025] The above inertial navigation module INS usually consists of inertial sensors, accelerometers, gyroscopes, and other components (magnetometers, pressure sensors, etc.). Usually, each INS device contains three sets of gyroscopes and accelerometers to measure the angular acceleration and linear acceleration in three degrees of freedom respectively. The gyroscope is used to obtain the angular velocity of the moving object and measure its angular change, and the accelerometer is used to obtain the linear acceleration of the moving object and measure its velocity change. The inertial navigation solution software calculates the velocity, position, and attitude of the vehicle in the geographical coordinate system through methods such as integral operation and attitude matrix calculation.

[0026] In the normal working mode, the combination of the global satellite positioning and navigation GNSS and the inertial navigation INS can provide the maximum navigation and positioning accuracy. The implementation method is as follows: There is a set of subsystem algorithms inside GNSS, and there is a set of subsystem algorithms inside INS, which are distributed as follows: Algorithms inside the GNSS subsystem: The single-point positioning accuracy of the GNSS system is at the meter level. The RTK (Real-time kinematic) carrier phase differential technology enhancement technology needs to be used to achieve centimeter-level positioning accuracy. Due to factors such as the actual environment and communication conditions, the accuracy of the high-precision positioning algorithm will be affected, and it depends on long-term accumulation to achieve algorithm iteration. The algorithm inside the INS subsystem is mainly the DR (Dead Reckoning) algorithm, which means that given the navigation state (state, velocity, and position) at the previous moment, the navigation state at the next moment is inferred based on the sensor observations. The DR algorithm includes two parts: attitude arrangement and position arrangement.

[0027] The two navigation systems are tightly coupled and superimposed. The superimposing process is as follows: RTK positioning results + raw GNSS data + raw INS data are used. Compared with loose coupling, tight coupling does not require a complete GNSS result, and the INS error can be calculated based on partial data provided by GNSS. Therefore, when GNSS is subject to certain interference and the number of detected satellites is less than 4, the obtained information can still be used as the basis for the filter calculation after being resolved. Using the tight coupling algorithm can avoid problems such as inaccurate positioning in scenarios with no satellite signals at all, such as tunnels and underground garages, which is comparable to tight coupling. However, in scenarios with satellite signals but the signals are blocked, such as urban canyons, the positioning is inaccurate.

[0028] In the emergency mode: Only the inertial navigation INS is applicable. It is a navigation device that uses an inertial measurement unit to measure the specific force and angular velocity information of the carrier, combines the given initial motion conditions, and thus calculates parameters such as velocity, position, and attitude in real time, and can output complete six-degree-of-freedom data. This method does not radiate energy to the outside and does not depend on external signals, so it has strong autonomy.

[0029] The above industrial control computer is directly powered by the vehicle power supply, with a power supply of DC12V and a built-in adapter AC220 / DC12V. The server is directly powered by a vehicle power strip, with a power supply of AC220. The display screen is directly powered by a cockpit power strip, with a power supply of AC220 and 35W. The data display cable is a DP cable. The control cable is a USB to square port cable. The display screen can be a touch screen. In this embodiment, 2 monitors are used. In other embodiments, one or more monitors can be used, which is determined according to the actual installation space and requirements.

[0030] The device layer includes multiple groups of detection units corresponding to multiple industrial control computers. In this embodiment, 4 industrial control computers correspond to 4 detection units, and the 4 industrial control computers operate independently without interfering with each other in the network. The detection unit includes a high-speed multi-spectral 3D camera, a laser, a color multi-spectral area array camera, a lens, and a light source. Among them, the high-speed multi-spectral 3D camera is a 3D structured light measurement integrated imaging component, which can collect high-precision 3D point cloud data of the road surface at high speed. For example, millimeter-level 3D point cloud data can be used to quickly and accurately obtain road surface foreign object information. The color multi-spectral area array camera includes a camera module, an imaging module, and an illumination module. The imaging module is the main functional module of the color multi-spectral area array camera module, which completes the acquisition of the two-dimensional color image of the road surface. The imaging module mainly includes a lens and a camera. The lens projects the road surface information onto the camera sensing surface; the camera receives an external trigger, starts exposure and image processing, and outputs the color image information of the road surface. The color multi-spectral area array camera is used in cooperation with the high-speed multi-spectral 3D camera module, and during the working process, it outputs high-resolution color image information of the detected road surface in real time to assist the user in accurately positioning and identifying road surface foreign objects. The illumination module is an auxiliary part of the imaging module. In a low-illumination environment, the illumination module receives a trigger signal to start illumination and fills light for the collected road surface target background to ensure that the imaging module can collect clear-quality images in both day and night environments.

[0031] In this embodiment, the power supply range of the high-speed multi-spectral 3D camera device is DC11V - 13V, and the maximum power is 50W. It can be set to two triggering methods, internal trigger and external trigger, through configuration software. When driving without a driver, it can be an internal trigger. If the trigger variable is set to an internal trigger, the shooting frequency is directly given a fixed value by the configuration software. If the camera trigger is set to an external trigger, digital high and low triggers (TTL+, TTL-) are performed by an external physical connection. The physical wiring signal range is the camera line trigger signal, 5V, TTL level. The data acquisition of the high-speed multi-spectral 3D camera is transmitted through a network cable, and it is required that the acquisition device is not lower than a gigabit network port. In this embodiment, the high-speed multi-spectral 3D camera is a line array camera, with an accuracy requirement of 1mm. The line frequency of the high-speed multi-spectral 3D camera needs to reach 4Khz. Therefore, the frequency requirement for the TTL level generating source can reach 4Khz.

[0032] The internal trigger working mode is as follows: when the vehicle is in driverless mode, the software inside the vehicle can preset the starting position of the high-speed multi-spectral 3D camera in advance. Then, based on the navigation and positioning information provided by its own positioning module, when the vehicle reaches the preset position, the high-speed multi-spectral 3D camera automatically starts to take pictures.

[0033] The external trigger working mode is as follows: when the vehicle is manned or there is a problem with the positioning module not working, the external trigger working mode is used. The external trigger is an externally installed starting power supply, an external starting switch, and an artificial starting method is adopted. When reaching the section that needs to be inspected, the external starting power supply is manually started. After the external starting power supply is started, it can output high and low levels. The starting power supply is connected to the Doppler color camera and the light source, and the starting power supply drives the camera to start, realizing external triggering.

[0034] The power supply range of the color multi-spectral area array camera is DC11V - 13V, and the maximum power is 10W. It can be set to internal trigger and external trigger. When set to external trigger, the received signal is 5V, TTL level. The power supply electrical interface of the light source has a power supply range of DC24V and a minimum power of 30W. The edge trigger mode is adopted, and the response time is within 1us. The falling edge trigger (TTL) is adopted. The light source can be triggered by high level or low level; it can be triggered by both digital quantity signals and TTL levels. The flash time is 50us - 700us, and the transient power is 3000 watts.

[0035] In the embodiment of the present invention, the server is connected to the above-mentioned multiple groups of industrial control computers through a gigabit switch. The data exchange delay between the industrial control computer and the server does not exceed 1s. Each industrial control computer reserves a serial port to receive the serial port data actively returned by the navigation and positioning module, and the data format is determined by both parties. Multiple groups of industrial control computers are connected to the corresponding detection units through a gigabit switch. The industrial control computer receives the data collected by the high-speed multi-spectral 3D camera and the color multi-spectral area array camera through the network cable at the same time and controls the high-speed multi-spectral 3D camera and the color multi-spectral area array camera. The detection unit also includes an encoder and a frequency divider. In this embodiment, a high-precision encoder is used, specifically. In this embodiment, according to the frequency divider interface circuit, the encoder selects the common NPN type, 600P / turn, 5-wire system, inner hole installation, and the inner diameter is φ30. The power supply input of the frequency divider is: DC24V, and it distributes 1-way pulse into 8-way 5vTTL pulse outputs, which is commonly used for the simultaneous triggering of multiple Basler cameras. In addition, it also includes a frequency reduction and frequency division module, the power supply input is DC 24v, and the frequency division number can be switched correspondingly by a DIP switch (subsequent is digital). The supported frequency division numbers in this embodiment are: 2 / 4 / 5 / 10 / 20 / 50 / 100 / 500.

[0036] In this embodiment, the high-speed multi-spectral 3D camera, the color multi-spectral area array camera, and the light source use TTL level as external trigger, as Figure 2 shown. The trigger level synchronization time difference between the high-speed multi-spectral 3D cameras in multiple groups of detection units is less than 10 ns. The trigger level synchronization time difference between the color multi-spectral area array camera and the light source in multiple groups of detection units is less than 10 ns. The navigation and positioning module includes an antenna and a frequency divider. The level triggering the navigation and positioning module is synchronized with the level triggering the color multi-spectral area array camera, and the time difference is less than 10 ns.

[0037] The road surface foreign object detection system of the present invention realizes high-precision and fast road surface foreign object detection and recognition by adopting optoelectronic detection and machine vision recognition technologies, and can be used in scenarios with high requirements for road surface foreign objects such as highways or airport runways.

[0038] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited by the above embodiments. What is described in the above embodiments and the specification only illustrates the principle of the present invention. Without departing from the spirit and scope of the present invention, the present invention will have various changes and improvements, and these changes and improvements all fall within the scope of the present invention claimed. The scope of protection claimed by the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for detecting foreign objects on a road surface with multiple trigger working modes, characterized in that, Specifically, it includes the following steps: Drive a vehicle with a detection unit on the road surface to be measured. The detection unit includes a high-speed multi-spectral 3D camera, a color multi-spectral area array camera, a multi-spectral or polarizing lens and a light source, a frequency divider, and a frequency-down conversion module. The high-speed multi-spectral 3D camera collects millimeter-level 3D point cloud data of the road to obtain information on road surface foreign objects; the color multi-spectral area array camera receives an external trigger, starts exposure and image processing, and outputs color image information of the road surface. Set the high-speed multi-spectral 3D camera to two modes: internal trigger and external trigger through configuration software. The color multi-spectral area array camera is set to internal trigger and external trigger. The color multi-spectral area array camera is used in conjunction with the high-speed multi-spectral 3D camera to output high-resolution color image information of the detected road surface in real time during the working process. When the vehicle is driverless, the software inside the vehicle presets the starting position of the high-speed multi-spectral 3D camera. The vehicle, based on the navigation and positioning information given by its own positioning module, when the vehicle reaches the preset starting position, the high-speed multi-spectral 3D camera automatically starts shooting, and the shooting frequency is directly given a fixed value by the configuration software; the power supply range of the high-speed multi-spectral 3D camera device is DC11V - 13V, and the maximum power is 50W. When it is driverless, it is in the internal trigger mode, and the shooting frequency is directly given a fixed value by the configuration software; the high-speed multi-spectral 3D camera is a line array camera with an accuracy requirement of 1mm, and the line frequency of the high-speed multi-spectral 3D camera needs to reach 4Khz. Therefore, the frequency requirement for the TTL level generation source can reach 4Khz. When the vehicle is manned or the positioning module fails to work, use the external trigger working mode. The external trigger adopts a manual start method. When reaching the section that needs to be inspected, manually start the external start power supply. The external start power supply can output high and low levels after startup, and the external start power supply drives the camera to start, realizing the external trigger; the external trigger is digitally triggered by high and low levels through an external physical connection. The physical wiring signal range is the external trigger signal executed by the high-speed multi-spectral 3D camera, which is 5V, TTL level. Among them, The detection unit is multiple groups of detection units. The server in the monitoring layer is connected to multiple industrial computers through a gigabit switch. The data exchange delay between the industrial computer and the server does not exceed 1s. Each industrial computer reserves a serial port to receive the serial data actively returned by the positioning module. Multiple industrial computers are connected to the corresponding detection units through a gigabit switch; when the high-speed multi-spectral 3D camera is in the external trigger working mode, the positioning module is connected to the color multi-spectral area array camera via a frequency-down conversion module, and the positioning module is connected to the high-speed multi-spectral camera via a frequency-down conversion module and a frequency division module in sequence; the positioning module is a GNSS global satellite positioning and navigation system and an inertial navigation system ISN. When GNSS is subject to certain interference and the number of detected satellites is less than 4, the obtained information is still used as the basis for calculation by the filter after being resolved.

2. The method for detecting foreign objects on the road surface with multiple trigger working modes according to claim 1, wherein, The detection unit also includes an encoder, a frequency divider, and a frequency-down conversion module.

3. The method for detecting foreign objects on the road surface with multiple trigger working modes according to claim 1, wherein, The positioning module has two working modes, namely the normal mode and the emergency mode. In the normal mode, the GNSS global satellite positioning and navigation system works in combination with the inertial navigation system ISN. In the emergency mode, after the GNSS global satellite positioning and navigation system fails, the inertial navigation system ISN works independently.

4. A method for detecting foreign objects on a road surface with multiple trigger working modes according to claim 3, characterized in that, The GNSS global satellite positioning and navigation system internally uses real-time kinematic carrier phase differential technology for positioning and calculation.

5. A method for detecting foreign objects on a road surface with multiple trigger working modes according to claim 3, characterized in that, The inertial navigation system ISN uses a dead reckoning algorithm for positioning and calculation.

6. A method for detecting foreign objects on a road surface with multiple trigger working modes according to claim 1, characterized in that The detection unit is a multi-group detection unit. The trigger level synchronization time difference between the high-speed multi-spectral 3D cameras in the multi-group detection units is less than 50 ns. The trigger level synchronization time difference between the color multi-spectral area array camera and the light source is less than 50 ns. The level triggering the positioning module is synchronized with the level triggering the color multi-spectral area array camera, and the time difference is less than 50 ns.

Images