A multi-sensor fusion ground speed measurement method and system for maglev trains

Abstract

This application discloses a multi-sensor fusion method and system for measuring the ground speed of a maglev train. The method comprises the following steps: S1: fusing the detection results of two ultrasonic speed radars installed on the ground to obtain radar speed data of the maglev train; S2: fusing image data of the maglev train collected by a binocular camera installed on the ground to obtain camera speed data of the maglev train; and S3: fusing the radar speed data and camera speed data using complementary filtering to obtain the actual speed of the maglev train. This application utilizes a lower-precision, lower-frequency speed measurement device to collect raw data, and achieves higher-precision, higher-frequency speed acquisition through multiple data fusion methods, thus meeting system requirements. This method eliminates the need to install additional devices on the maglev train, make any modifications to the maglev train, or transmit data back to the ground via wireless communication or other means, thereby ensuring the safe operation of the maglev train.

Description

Technical Field

[0001] The present application relates to the technical field of train speed measurement, and in particular, to a multi-sensor fusion ground speed measurement method and system for a maglev train. Background Art

[0002] The 360-degree image inspection system for maglev trains primarily consists of a gantry, a linear array camera, a laser light source, a speed measurement unit, a master control unit, and analysis and service units. It enables 360-degree image inspection of maglev train bodies and identifies surface faults. The gantry, equipped with multiple linear array cameras and laser light sources, stands above the maglev track. As a train passes through the gantry, the linear array cameras and laser light sources activate, capturing 360-degree images of the train's roof, sides, and underbody in real time. Simultaneously, the speed measurement unit collects the train's speed as it passes through the gantry, providing speed information for the analysis and service units to reconstruct train images.

[0003] Since maglev trains inevitably experience speed changes as they pass through the gantry, images captured by line array cameras are inevitably stretched and compressed. Accurately restoring train images requires highly accurate and dense real-time train speed readings. Therefore, the real-time speed measurement system for maglev trains is a key component of the 360-degree image detection system for maglev trains.

[0004] Currently, there are several ways to measure the speed of medium and low-speed maglev trains:

[0005] 1. This approach involves installing onboard coils on the train's underside, induction loops on the maglev track, and a ground control unit, enabling train speed measurement through train-to-ground communication. A representative application is the Beijing S1 Line. This approach offers the advantage of accurate speed measurement across the train's speed range. However, it requires induction loops throughout the entire line, resulting in high costs.

[0006] 2. Speed measurement units (modules consisting of multiple electromagnetic induction switches spaced at regular intervals) are installed on the underside of trains, measuring train speed by counting sleepers. A representative application is the Changsha Maglev Express. This method offers the advantage of low cost, but its disadvantage is that speed measurement accuracy decreases significantly when train speeds fall below 10 km / h.

[0007] 3. Train speed measurement can be achieved by installing GPS and Beidou positioning systems on trains and BTM transponders on the ground. This method has the advantages of low cost and continuous and accurate measurement. However, its disadvantage is that positioning accuracy may be poor or even ineffective due to signal obstruction.

[0008] In the field of high-speed maglev, the Shanghai High-Speed Maglev mainly measures train speed by installing relative position sensors on the maglev train suspension frame to collect pulse coding data from the tooth-shaped grooves on the surface of the long stator on the track.

[0009] The above-mentioned speed measurement methods all achieve real-time speed measurement and positioning of the train by installing a speed measuring unit on the train. However, in the 360-degree image detection system on the trackside of the maglev train, the detection system is installed on the ground. No modifications can be made to the existing maglev vehicles. The real-time speed of the train cannot be obtained from the train through wireless communication between the train and the ground. The train speed can only be measured by installing a speed measuring device on the ground. It is also impossible to assist in ground speed measurement by adding other components (including targets, etc.) to the vehicle.

[0010] Currently, there are no 360-degree image detection system products in the field of medium and low-speed maglev. Therefore, there are no relevant patent references for the scheme of using ground devices to realize the speed measurement of maglev trains. The main system similar to this system is the subway 360-degree image detection system. Due to different application scenarios, the subway 360-degree image detection system train speed measurement is not difficult.

[0011] In the maglev 360-degree image detection system, no modifications can be made to the maglev train. Therefore, many speed measurement methods, such as installing tags, RFID modules, and QR codes on the train, are not feasible. Non-contact speed measurement is the only option. The main methods are as follows:

[0012] 1. Laser speed measurement requires sticking reflectors on the surface of the train, but maglev vehicles are not allowed to install them, so it is not feasible.

[0013] 2. Radar speed measurement, although it can collect train speed in real time, has low detection accuracy and low detection frequency, which makes it difficult to meet system requirements.

[0014] 3. Camera speed measurement has not yet been maturely applied in the field of rail transit. Its detection accuracy and frequency depend on whether the target has obvious feature points and whether the environment changes drastically.

[0015] 4. Ground magnetic induction speed measurement is currently the most accurate solution for measuring train speed. However, in order to measure the speed of the train in the entire section, ground magnetic induction devices need to be deployed along the entire line, which is costly. At the same time, the magnetic induction devices cannot be deployed very close together, so their speed frequency is relatively low. Summary of the Invention

[0016] On one hand, the present application provides a multi-sensor fusion ground speed measurement method for maglev trains to solve the technical problem that existing train speed measurement methods cannot perform high-frequency and high-precision tests on maglev trains.

[0017] The technical solutions adopted in this application are as follows:

[0018] A multi-sensor fusion ground speed measurement method for a maglev train comprises the following steps:

[0019] S1. Fusion of detection results from two ultrasonic speed radars installed on the ground to obtain radar speed data of the maglev train. During the detection process, one ultrasonic speed radar detects the direction of the oncoming maglev train, while the other ultrasonic speed radar detects the direction of the maglev train.

[0020] S2, fusing the maglev train image data collected by the binocular camera installed on the ground to obtain the camera speed data of the maglev train;

[0021] S3. Fusing the radar speed data and the camera speed data using a complementary filtering method to obtain the actual speed of the maglev train.

[0022] Furthermore, the step S1 specifically includes the steps of:

[0023] S11. During the detection of the maglev train, the ultrasonic speed measuring radar facing the oncoming direction of the maglev train is set as the primary speed measuring radar to obtain primary radar speed measurement data, and another ultrasonic speed measuring radar facing the traveling direction of the train is set as the auxiliary speed measuring radar to obtain auxiliary radar speed measurement data;

[0024] S12. After the train completely passes through the measurement midpoint between the two ultrasonic speed measuring radars, the primary speed measuring radar and the auxiliary speed measuring radar are swapped to obtain primary radar speed measuring data and auxiliary radar speed measuring data;

[0025] S13. Fusing the main radar speed measurement data and the auxiliary radar speed measurement data to obtain the radar speed measurement data of the maglev train.

[0026] Furthermore, the step S13 specifically includes the following steps:

[0027] S131. Sort the radar speed measurement data of the two ultrasonic speed measurement radars in chronological order.

[0028] S132, inserting the auxiliary radar speed measurement data into the main radar speed measurement data in chronological order;

[0029] S133. Use the Lombard integration algorithm to interpolate the speed data according to the data interval of the set time length to generate a sufficient amount of radar speed measurement data of the maglev train.

[0030] Furthermore, the step S131 further includes the following steps during the sorting process:

[0031] All radar speed measurement data are compared with the auxiliary radar speed measurement data based on the main radar speed measurement data. If the difference between the auxiliary radar speed measurement data and the two primary radar speed measurement data adjacent to it in time is greater than 1 km / h, or the difference between any of the two primary radar speed measurement data adjacent to it in time is greater than 1.5 km / h, the auxiliary radar speed measurement data is regarded as erroneous data and is directly filtered out.

[0032] Furthermore, the step S2 specifically includes the steps of:

[0033] S21. When the maglev train passes through the location photographed by the binocular camera, image depth information is calculated in real time to obtain image position information of the train at a fixed depth position;

[0034] S22. Calculate the train motion optical flow field using deep learning using the two preceding and following frames of images from one of the single cameras. Then, select the optical flow field data corresponding to the train position based on the depth information in the three-dimensional data to obtain the current camera speed data captured by the binocular camera.

[0035] S23, the speed obtained from the previous and subsequent multiple frames of images is corrected using Kalman filtering.

[0036] Furthermore, the step S21 specifically includes the following steps:

[0037] S211, calibrate the binocular cameras to obtain the internal and external parameters and homography matrix of the two cameras;

[0038] S212. When the maglev train passes through the location photographed by the binocular camera, the original image is corrected according to the calibration result, and the image depth information is calculated in real time to obtain the three-dimensional position information of the image where the train is located with a fixed depth position.

[0039] Furthermore, step S22 specifically includes the following steps:

[0040] S221. Based on the image captured by one of the cameras, two consecutive frame images are input. Features are extracted from each image using a CNN convolutional neural network. The two feature maps are compared by multiplication blocks. After the feature map matching is completed, the relevant results are forward propagated to the subsequent convolutional layer to further extract the top-level features. The convolutional architecture consists of 6 residual layers. The inner product of all feature map pairs is calculated to obtain a four-dimensional tensor. The last two dimensions of the four-dimensional tensor are pooled using kernels of sizes 1, 2, 4, and 8 respectively to form a correlation pyramid. Then, a multi-scale image similarity feature is established.

[0041] S222, using a gated recurrent unit sequence to update the optical flow value calculated from the initial value f0=0, generating an update direction Δf for each iteration. Since the train's forward direction is unique, some erroneous calculations can be directly filtered out to obtain the image optical flow field information;

[0042] S223. Determine the position of the maglev train based on the depth information of the converted three-dimensional coordinates, obtain the image optical flow information of the corresponding position of the maglev train, cluster the displacement information in the optical flow information to remove abnormal points, and then perform mean filtering to obtain the current camera speed data captured by the binocular camera.

[0043] Furthermore, the step S3 specifically includes the steps of:

[0044] S31, performing linear interpolation on the radar speed measurement data to obtain a radar speed measurement data column, wherein 0 is correspondingly inserted between data in the radar speed measurement data obtained by the ultrasonic speed measurement radar that are separated by more than a set time length;

[0045] S32. Adjust the proportional coefficients α and β corresponding to the radar speed measurement data and the binocular speed measurement data according to the actual results;

[0046] S33, calculate the actual speed V of the final maglev train at time k based on the weighted radar speed measurement data and camera speed measurement data k * :

[0047]

[0048] in, is the radar speed measurement data at time k, is the camera speed measurement data at time k.

[0049] Furthermore, the step S32 specifically includes the following steps:

[0050] During the day, the proportional coefficient β of the binocular test data is greater than the proportional coefficient α of the radar speed measurement data. At the same time, when the speed of the radar speed measurement data and the camera speed measurement data is less than 1 km / h, α is 0.05 and β is 0.95; when the train speed is greater than 1 km / h and less than 10 km / h, α is 0.35 and β is 0.65; when the train speed is greater than 10 km / h, α is 0.45 and β is 0.55;

[0051] At night, the proportional coefficient β of the binocular test data is smaller than the proportional coefficient α of the radar speed measurement data. When the speed of the radar speed measurement data and the camera speed measurement data is less than 1 km / h, α is 0.55 and β is 0.45; when the train speed is greater than 1 km / h and less than 10 km / h, α is 0.65 and β is 0.35; when the train speed is greater than 10 km / h, α is 0.75 and β is 0.25.

[0052] On the other hand, the present application also provides a multi-sensor fusion maglev train ground speed measurement system, comprising:

[0053] Two ultrasonic speed radars are installed on the ground to detect the speed of the maglev train. During the detection process, one ultrasonic speed radar detects the direction of the maglev train, and the other ultrasonic speed radar detects the direction of the maglev train.

[0054] Binocular cameras, installed obliquely and symmetrically on the side of the maglev track, are used to collect image data of the maglev train;

[0055] An oncoming train detection device, used to detect a train arrival signal when a train passes through the oncoming train detection device;

[0056] The control center is respectively connected to the two ultrasonic speed measuring radars, the binocular camera, and the vehicle detection device for signal transmission, so as to realize the multi-sensor fusion maglev train ground speed measurement method.

[0057] On the other hand, the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the multi-sensor fusion maglev train ground speed measurement method are implemented.

[0058] On the other hand, the present application further provides a storage medium, which includes a stored program, and when the program is running, controls the device where the storage medium is located to execute the steps of the multi-sensor fusion maglev train ground speed measurement method.

[0059] Compared with the existing technology, this application has the following beneficial effects:

[0060] (1) This application collects raw data by using lower-precision, lower-frequency speed measuring devices, such as ultrasonic speed measuring radar and binocular camera, and achieves higher-precision, higher-frequency speed collection through multiple data fusion methods to meet system requirements.

[0061] (2) This application does not require the installation of other devices on the maglev train, does not require any modifications to the maglev train, and does not require the maglev train to transmit data back to the ground through wireless communication or other means, thereby ensuring the safe operation of the maglev train.

[0062] In addition to the above-described purposes, features and advantages, the present application has other purposes, features and advantages. The present application will be further described in detail below with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] The accompanying drawings, which constitute part of this application, are intended to provide a further understanding of this application. The exemplary embodiments and descriptions of this application are intended to explain this application and do not constitute an improper limitation on this application. In the accompanying drawings:

[0064] Figure 1It is a flow chart of a multi-sensor fusion maglev train ground speed measurement method according to a preferred embodiment of the present application.

[0065] Figure 2 It is a schematic diagram of the composition of the multi-sensor fusion maglev train ground speed measurement system of the preferred embodiment of the present application.

[0066] Figure 3 It is a schematic block diagram of an electronic device entity of a preferred embodiment of the present application.

[0067] Figure 4 It is a diagram of the internal structure of a computer device according to a preferred embodiment of the present application. DETAILED DESCRIPTION

[0068] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in this application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0069] Reference Figure 1 The preferred embodiment of the present application provides a multi-sensor fusion maglev train ground speed measurement method, comprising the steps of:

[0070] S1. Fusion of detection results of two ground-mounted ultrasonic speed radars on the maglev train to obtain radar speed data of the maglev train. During the detection process, one ultrasonic speed radar detects in the direction of the oncoming maglev train, while the other ultrasonic speed radar detects in the direction of the maglev train.

[0071] S2, fusing the image data of the maglev train collected by binocular cameras installed obliquely symmetrically on the side of the maglev track to obtain the camera speed data of the maglev train;

[0072] S3. Fusing the radar speed data and the camera speed data using a complementary filtering method to obtain the actual speed of the maglev train.

[0073] The multi-sensor fusion maglev train ground speed measurement method provided in this embodiment has the following advantages:

[0074] (1) This application collects raw data by using lower-precision, lower-frequency speed measuring devices, such as ultrasonic speed measuring radar and binocular camera, and achieves higher-precision, higher-frequency speed collection through multiple data fusion methods to meet system requirements.

[0075] (2) This application does not require the installation of other devices on the maglev train, does not require any modifications to the maglev train, and does not require the maglev train to transmit data back to the ground through wireless communication or other means, thereby ensuring the safe operation of the maglev train.

[0076] Specifically, the step S1 includes the following steps:

[0077] S11. During the detection of the maglev train, the ultrasonic speed measuring radar facing the oncoming direction of the maglev train is set as the primary speed measuring radar to obtain primary radar speed measurement data, and another ultrasonic speed measuring radar facing the traveling direction of the train is set as the auxiliary speed measuring radar to obtain auxiliary radar speed measurement data;

[0078] S12. After the train completely passes through the measurement midpoint between the two ultrasonic speed measuring radars, the primary speed measuring radar and the auxiliary speed measuring radar are swapped to obtain primary radar speed measuring data and auxiliary radar speed measuring data;

[0079] S13. Fusing the main radar speed measurement data and the auxiliary radar speed measurement data to obtain the radar speed measurement data of the maglev train.

[0080] In this embodiment, the data detected by the two ultrasonic speed radars are divided into primary radar speed data and auxiliary radar speed data according to the direction of the train. At the same time, the primary radar speed data and the auxiliary radar speed data are switched after the train passes a designated position. This is because speed radar speed measurement is based on the principle of the Doppler effect: when the target approaches the speed radar, the frequency of the reflected signal will be higher than the transmitter frequency. Conversely, when the target moves away from the speed radar, the frequency of the reflected signal will be lower than the transmitter frequency. The body of a maglev train is generally long and consists of multiple body sections connected in series. When a maglev train passes, the distance between the ultrasonic speed radar facing the direction of the maglev train and the train body continues to decrease. During this process, as the target continues to approach the speed radar, the frequency of the reflected signal returned from the body will continue to increase and become higher than the transmitter frequency, which is conducive to measuring the radar speed data based on the principle of the Doppler effect. However, after the signal emitted by the ultrasonic speed radar in the train's direction of travel reaches the vehicle body and is reflected back from it, the distance between the target and the speed radar initially remains constant due to the long vehicle body, making it impossible to obtain speed data using the Doppler effect. However, after the train completely passes the measurement midpoint between the two ultrasonic speed radars, the signal emitted by the ultrasonic speed radar in the train's direction of travel reaches the rear of the vehicle and is reflected back. As the rear of the vehicle gradually moves away from the speed radars, the frequency of the reflected signal returning from the rear of the vehicle decreases and eventually falls below the transmitter frequency. In other words, to obtain more accurate radar speed data, this embodiment, based on the characteristics of the maglev train's vehicle body, the installation and transmission directions of the two ultrasonic speed radars, and the current position of the maglev train, alternately uses the ultrasonic speed radar facing the oncoming maglev train and the ultrasonic speed radar in the train's direction of travel as the primary speed radar and the auxiliary speed radar, respectively. Primary and auxiliary radar speed data are then obtained accordingly. Finally, the primary and auxiliary radar speed data are fused to obtain more accurate radar speed data, thereby improving the accuracy of radar testing.

[0081] Specifically, the step S13 includes the following steps:

[0082] S131. Sort the radar speed measurement data of the two ultrasonic speed measurement radars in chronological order.

[0083] S132, inserting the auxiliary radar speed measurement data into the main radar speed measurement data in chronological order;

[0084] S133. Use the Lombard integration algorithm to interpolate the speed data according to the data interval of the set time length to generate a sufficient amount of radar speed measurement data of the maglev train.

[0085] After sorting in chronological order, this embodiment inserts the auxiliary radar speed measurement data into the main radar speed measurement data in chronological order, and finally interpolates the speed data through the Lombard integral algorithm to generate a sufficient number of radar speed measurement data of the maglev train. The benefits include: the speed data obtained is more accurate and close to the actual speed, the more speed data obtained, the smaller the error generated, and the data obtained after fusion with the camera speed measurement data is more accurate.

[0086] Specifically, the step S131 further includes the following steps during the sorting process:

[0087] All radar speed measurement data are compared with the auxiliary radar speed measurement data based on the main radar speed measurement data. If the difference between the auxiliary radar speed measurement data and the two primary radar speed measurement data adjacent to it in time is greater than 1 km / h, or the difference between any of the two primary radar speed measurement data adjacent to it in time is greater than 1.5 km / h, the auxiliary radar speed measurement data is regarded as erroneous data and is directly filtered out.

[0088] Specifically, the step S2 includes the following steps:

[0089] S21. When the maglev train passes through the location photographed by the binocular camera, image depth information is calculated in real time to obtain image position information of the train at a fixed depth position;

[0090] S22. Calculate the train motion optical flow field using deep learning using the two preceding and following frames of images from one of the single cameras. Then, select the optical flow field data corresponding to the train position based on the depth information in the three-dimensional data to obtain the current camera speed data captured by the binocular camera.

[0091] S23, the speed obtained from the previous and subsequent multiple frames of images is corrected using Kalman filtering.

[0092] Specifically, the step S21 includes the following steps:

[0093] S211, calibrate the binocular cameras to obtain the internal and external parameters and homography matrix of the two cameras;

[0094] S212. When the maglev train passes through the location photographed by the binocular camera, the original image is corrected according to the calibration result, and the image depth information is calculated in real time to obtain the three-dimensional position information of the image where the train is located with a fixed depth position.

[0095] Specifically, step S22 includes the following steps:

[0096] S221. Based on the image captured by one of the cameras, two consecutive frame images are input. Features are extracted from each image using a CNN convolutional neural network. The two feature maps are compared by multiplication blocks. After the feature map matching is completed, the relevant results are forward propagated to the subsequent convolutional layer to further extract the top-level features. The convolutional architecture consists of 6 residual layers. The inner product of all feature map pairs is calculated to obtain a four-dimensional tensor. The last two dimensions of the four-dimensional tensor are pooled using kernels of sizes 1, 2, 4, and 8 respectively to form a correlation pyramid. Then, a multi-scale image similarity feature is established.

[0097] S222, using a gated recurrent unit sequence to update the optical flow value calculated from the initial value f0=0, generating an update direction Δf for each iteration. Since the train's forward direction is unique, some erroneous calculations can be directly filtered out to obtain the image optical flow field information;

[0098] S223. Determine the position of the maglev train based on the depth information of the converted three-dimensional coordinates, obtain the image optical flow information of the corresponding position of the maglev train, cluster the displacement information in the optical flow information to remove abnormal points, and then perform mean filtering to obtain the current camera speed data captured by the binocular camera.

[0099] This embodiment uses a binocular camera to collect image data and uses algorithms such as optical flow to obtain a large amount of train speed data.

[0100] Specifically, the step S3 includes the following steps:

[0101] S31, performing linear interpolation on the radar speed measurement data to obtain a radar speed measurement data column, wherein 0 is correspondingly inserted between data in the radar speed measurement data obtained by the ultrasonic speed measurement radar that are separated by more than a set time length;

[0102] S32. Adjust the proportional coefficients α and β corresponding to the radar speed measurement data and the binocular speed measurement data according to the actual results;

[0103] S33, calculate the actual speed V of the final maglev train at time k based on the weighted radar speed measurement data and camera speed measurement data k * :

[0104]

[0105] in, is the radar speed measurement data at time k, is the camera speed measurement data at time k.

[0106] When calculating the final actual speed of the maglev train, this embodiment comprehensively considers the advantages and disadvantages of radar speed measurement and camera speed measurement, weights the radar speed measurement data and the camera speed measurement data accordingly, and uses a filtering algorithm to fuse the camera speed measurement data obtained by two monocular cameras and the radar speed measurement data obtained by two ultrasonic radars. Ultimately, a high-precision, high-frequency actual speed of the maglev train is obtained, thereby improving the reliability, stability, adaptability, and robustness of the speed measurement and reducing speed measurement errors.

[0107] Specifically, the step S32 includes the following steps:

[0108] During the day, the proportional coefficient β of the binocular test data is greater than the proportional coefficient α of the radar speed measurement data. At the same time, when the speed of the radar speed measurement data and the camera speed measurement data is less than 1 km / h, α is 0.05 and β is 0.95; when the train speed is greater than 1 km / h and less than 10 km / h, α is 0.35 and β is 0.65; when the train speed is greater than 10 km / h, α is 0.45 and β is 0.55;

[0109] At night, the proportional coefficient β of the binocular test data is smaller than the proportional coefficient α of the radar speed measurement data. When the speed of the radar speed measurement data and the camera speed measurement data is less than 1 km / h, α is 0.55 and β is 0.45; when the train speed is greater than 1 km / h and less than 10 km / h, α is 0.65 and β is 0.35; when the train speed is greater than 10 km / h, α is 0.75 and β is 0.25.

[0110] This embodiment specifically takes into account the respective advantages and disadvantages of radar speed measurement and camera speed measurement, and organically combines different lighting environments and non-stop speeds. When calculating the actual speed of the maglev train based on the integration of radar speed measurement data and camera speed measurement data, the weights of the radar speed measurement data and the camera speed measurement data are scientifically and reasonably set, thereby ensuring that the present application can obtain the accurate actual speed of the maglev train under different working conditions (including different light intensities and different speeds), thereby improving the reliability, stability, adaptability and robustness of the speed measurement, and avoiding large speed measurement errors under different working conditions.

[0111] like Figure 2 As shown, the present application also provides a multi-sensor fusion maglev train ground speed measurement system, including:

[0112] Two ultrasonic speed radars are installed on the ground to detect the speed of the maglev train. During the detection process, one ultrasonic speed radar detects the direction of the maglev train, and the other ultrasonic speed radar detects the direction of the maglev train.

[0113] Binocular cameras, installed obliquely and symmetrically on the side of the maglev track, are used to collect image data of the maglev train;

[0114] An oncoming train detection device, used to detect a train arrival signal when a train passes through the oncoming train detection device;

[0115] The control center is respectively connected to the two ultrasonic speed measuring radars, the binocular camera, and the vehicle detection device for signal transmission, so as to realize the multi-sensor fusion maglev train ground speed measurement method.

[0116] When a train passes through the oncoming vehicle detection unit, it transmits an arrival signal to the control center, which then activates two radar speed measurement units and a binocular camera. The binocular camera, consisting of two area array cameras, captures images at a rate of 100 frames per second after defining a region of interest (ROI). The captured images are transmitted in real time via Ethernet to the control center for image processing. Because the ultrasonic speed radar lacks data caching, its measured speed is transmitted to the control center in real time via the 485 bus.

[0117] The data obtained by the ultrasonic speed radar is a discrete, low-precision speed value, and the measurement is inaccurate when the train speed is low. At the same time, it is transmitted to the control center using the 485 protocol, and the speed does not have timestamp information. When the control center receives the data packet, it assigns a real-time timestamp to each speed value.

[0118] Two monocular cameras form a binocular camera to jointly shoot the maglev train suspension frame area. This area has a complex structure and many feature points, which is conducive to extracting feature points for matching.

[0119] It can be seen that the above embodiments have the following characteristics:

[0120] (1) Two opposing ultrasonic speed measurement radars and a binocular camera (a monocular camera measures speed, while a binocular camera only calculates depth information) are used to complement each other to measure the speed of maglev trains traveling in different directions.

[0121] (2) A single area array camera in the binocular camera is used to calculate the optical flow of the maglev train through images using deep learning. The binocular camera realizes the depth calculation of the train, and then extracts the corresponding optical flow through the depth corresponding to the train, directly eliminating the optical flow field data with matching errors, and realizing accurate speed measurement of the maglev train.

[0122] (3) The control center fuses the speed data collected by the ultrasonic speed radar and the monocular camera to obtain high-precision, high-frequency speed data and uploads it.

[0123] like Figure 3As shown, a preferred embodiment of the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the multi-sensor fusion maglev train ground speed measurement method in the above-mentioned embodiment are implemented.

[0124] like Figure 4 As shown, the preferred embodiment of the present application further provides a computer device, which can be a terminal or a liveness detection server, and its internal structure diagram can be as shown in FIG. Figure 4 As shown. The computer device includes a processor, a memory, and a network interface connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with other external computer devices via a network connection. When the computer program is executed by the processor, the steps of the above-mentioned multi-sensor fusion maglev train ground speed measurement method are implemented.

[0125] Those skilled in the art will understand that Figure 4 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0126] A preferred embodiment of the present application further provides a storage medium, which includes a stored program. When the program is running, the device where the storage medium is located is controlled to execute the steps of the multi-sensor fusion maglev train ground speed measurement method in the above embodiment.

[0127] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0128] If the functions described in the method of this embodiment are implemented in the form of a software functional unit and sold or used as an independent product, they can be stored in a storage medium readable by one or more computing devices. Based on this understanding, the part of the embodiment of the present application that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes a number of instructions for enabling a computing device (which can be a personal computer, server, mobile computing device or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code.

[0129] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code. The scheme in the embodiment of the present application can be implemented in various computer languages, for example, object-oriented programming language Java and literal translation scripting language JavaScript, etc.

[0130] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0131] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1The function specified in one or more boxes.

[0132] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The steps for the function specified in one or more boxes.

[0133] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present application.

[0134] Obviously, those skilled in the art may make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalents, this application is intended to include these modifications and variations.

Claims

1. A multi-sensor fusion method for measuring the ground speed of a maglev train, characterized in that: Including steps: S1. Fusion of detection results of two ultrasonic speed measuring radars installed on the ground on a maglev train to obtain radar speed data of the maglev train. During the detection process, one of the ultrasonic speed measuring radars is detecting in the direction of the oncoming maglev train, and the other ultrasonic speed measuring radar is detecting in the direction of the traveling maglev train. The method specifically includes the following steps: S11. During the detection process of the maglev train, the ultrasonic speed measuring radar facing the direction of the oncoming maglev train is set as the primary speed measuring radar to obtain primary radar speed data, and the other ultrasonic speed measuring radar in the direction of the traveling train is set as the auxiliary speed measuring radar to obtain auxiliary radar speed data; S12. After the train completely passes through the measurement midpoint between the two ultrasonic speed measuring radars, the primary speed measuring radar and the auxiliary speed measuring radar are swapped to obtain primary radar speed data and auxiliary radar speed data; S13. Fusion of the primary radar speed data and the auxiliary radar speed data to obtain radar speed data of the maglev train; S2, fusing the maglev train image data collected by the binocular camera installed on the ground to obtain the camera speed data of the maglev train; S3. Fusing the radar speed data and the camera speed data using a complementary filtering method to obtain the actual speed of the maglev train.

2. The multi-sensor fusion maglev train ground speed measurement method according to claim 1, characterized in that: The step S13 specifically includes the following steps: S131. Sort the radar speed measurement data of the two ultrasonic speed measurement radars in chronological order. S132, inserting the auxiliary radar speed measurement data into the main radar speed measurement data in chronological order; S133. Use the Lombard integration algorithm to interpolate the speed data according to the data interval of the set time length to generate a sufficient amount of radar speed measurement data of the maglev train.

3. The multi-sensor fusion maglev train ground speed measurement method according to claim 2, characterized in that: The step S131 further includes the following steps during the sorting process: All radar speed measurement data are compared with the auxiliary radar speed measurement data based on the main radar speed measurement data. If the difference between the auxiliary radar speed measurement data and the two primary radar speed measurement data adjacent to it in time is greater than 1 km / h, or the difference between any of the two primary radar speed measurement data adjacent to it in time is greater than 1.5 km / h, the auxiliary radar speed measurement data is regarded as erroneous data and is directly filtered out.

4. The multi-sensor fusion maglev train ground speed measurement method according to claim 1, characterized in that: The step S2 specifically includes the following steps: S21. When the maglev train passes through the location photographed by the binocular camera, image depth information is calculated in real time to obtain image position information of the train at a fixed depth position; S22. Calculate the train motion optical flow field using deep learning using the two preceding and following frames of images from one of the single cameras. Then, select the optical flow field data corresponding to the train position based on the depth information in the three-dimensional data to obtain the current camera speed data captured by the binocular camera. S23, the speed obtained from the previous and subsequent multiple frames of images is corrected using Kalman filtering.

5. The multi-sensor fusion maglev train ground speed measurement method according to claim 4, characterized in that: The step S21 specifically includes the following steps: S211, calibrate the binocular cameras to obtain the internal and external parameters and homography matrix of the two cameras; S212. When the maglev train passes through the location photographed by the binocular camera, the original image is corrected according to the calibration result, and the image depth information is calculated in real time to obtain the three-dimensional position information of the image where the train is located with a fixed depth position.

6. The multi-sensor fusion ground speed measurement method for maglev train according to claim 4, characterized in that: Step S22 specifically includes the following steps: S221. Based on the image captured by one of the cameras, two consecutive frame images are input. Features are extracted from each image using a CNN convolutional neural network. The two feature maps are compared by multiplication blocks. After the feature map matching is completed, the relevant results are forward propagated to the subsequent convolutional layer to further extract the top-level features. The convolutional architecture consists of 6 residual layers. The inner product of all feature map pairs is calculated to obtain a four-dimensional tensor. The last two dimensions of the four-dimensional tensor are pooled using kernels of sizes 1, 2, 4, and 8 respectively to form a correlation pyramid. Then, a multi-scale image similarity feature is established. S222, using a gated recurrent unit sequence to update the optical flow value calculated from the initial value f0=0, generating an update direction Δf for each iteration. Since the train's forward direction is unique, some erroneous calculations can be directly filtered out to obtain the image optical flow field information; S223. Determine the position of the maglev train based on the depth information of the converted three-dimensional coordinates, obtain the image optical flow information of the corresponding position of the maglev train, cluster the displacement information in the optical flow information to remove abnormal points, and then perform mean filtering to obtain the current camera speed data captured by the binocular camera.

7. The multi-sensor fusion ground speed measurement method for maglev train according to claim 1, characterized in that: The step S3 specifically includes the following steps: S31, performing linear interpolation on the radar speed measurement data to obtain a radar speed measurement data column, wherein 0 is correspondingly inserted between data in the radar speed measurement data obtained by the ultrasonic speed measurement radar that are separated by more than a set time length; S32. Adjust the proportional coefficients α and β corresponding to the radar speed measurement data and the binocular speed measurement data according to the actual results; S33, calculate the actual speed V of the final maglev train at time k based on the weighted radar speed measurement data and camera speed measurement data k * : in, is the radar speed measurement data at time k, is the camera speed measurement data at time k.

8. The multi-sensor fusion ground speed measurement method for maglev train according to claim 7, characterized in that: The step S32 specifically includes the following steps: During the day, the proportional coefficient β of the binocular test data is greater than the proportional coefficient α of the radar speed measurement data. At the same time, when the speed of the radar speed measurement data and the camera speed measurement data is less than 1 km / h, α is 0.05 and β is 0.95; when the train speed is greater than 1 km / h and less than 10 km / h, α is 0.35 and β is 0.65; when the train speed is greater than 10 km / h, α is 0.45 and β is 0.55; At night, the proportional coefficient β of the binocular test data is smaller than the proportional coefficient α of the radar speed measurement data. When the speed of the radar speed measurement data and the camera speed measurement data is less than 1 km / h, α is 0.55 and β is 0.45; when the train speed is greater than 1 km / h and less than 10 km / h, α is 0.65 and β is 0.35; when the train speed is greater than 10 km / h, α is 0.75 and β is 0.

25.

9. A multi-sensor fusion maglev train ground speed measurement system, characterized in that: include: Two ultrasonic speed radars are installed on the ground to detect the speed of the maglev train. During the detection process, one ultrasonic speed radar detects the direction of the maglev train, and the other ultrasonic speed radar detects the direction of the maglev train. Binocular cameras, installed obliquely and symmetrically on the side of the maglev track, are used to collect image data of the maglev train; An oncoming train detection device, used to detect a train arrival signal when a train passes through the oncoming train detection device; The control center is respectively connected to the two ultrasonic speed measurement radars, the binocular camera, and the vehicle detection device for signal connection, so as to implement the multi-sensor fusion maglev train ground speed measurement method according to any one of claims 1 to 8.

Images