Multi-modal fusion sensing method and device for vehicle position in limited space and medium

Abstract

The invention discloses a multi-modal fusion sensing method and device for a vehicle position in a limited space and a medium, and the method comprises the steps: building a one-dimensional coordinate system OX in the limited space, and obtaining a set of all millimeter wave radar coverage areas; the method comprises the following steps: acquiring positioning data of UWB card readers on vehicles through the Ethernet, and calculating to obtain position information of k vehicles in a coordinate system OX to form a set A; acquiring vehicle data of the millimeter wave radar through a CAN bus, and calculating to obtain position information and speed information of n vehicles in a coordinate system OX to form a set B; performing fitting adjustment on the error of the vehicle position; dynamically diagnosing whether the millimeter wave radar is abnormal; and dynamically diagnosing whether UWB positioning is abnormal or not. According to the method, the millimeter wave radar with accurate local distance measurement is used, the position sensing result is dynamically fitted and corrected, and even if interference factors change after equipment installation, the vehicle position sensing effect can be improved; a multi-source sensing mode is adopted, diagnosis of the sensing technology state can be perfected, and abnormity or faults can be found in time.

Description

Technical Field

[0001] The present invention relates to the technical field of underground vehicle positioning, and in particular to a multi-modal fusion sensing method, device and medium for vehicle position in a confined space. Background Art

[0002] Pickup trucks and other auxiliary transport vehicles are an important means of transportation in mining production. When driving in confined spaces underground, considering factors such as limited driver vision, it is usually necessary to realize vehicle position perception at key sections such as forks in the road, complete vehicle traffic instructions, and ensure driving safety. At present, common vehicle perception technologies in confined spaces underground mines include: regional positioning based on RFID / TOA and other technologies, fixed-point positioning based on infrared beam / ground sensing coils, and positioning based on UWB technology.

[0003] Regional positioning based on RFID / TOA and other technologies: RFID / TOA readers are installed on the side walls of the tunnel, with a coverage range of 5 to 20 meters; RFID / TOA identification cards are installed on vehicles. When a vehicle passes through the sensing area, the reader obtains vehicle information and senses the direction of the vehicle through the data sequence of two adjacent readers. It has the following defects and shortcomings: (1) When multiple vehicles pass through at the same time, there is a phenomenon of missing cards due to signal obstruction; (2) When the vehicle enters the sensing system at the beginning, the direction of the vehicle cannot be known; (3) There are many blind spots in the sensing area, and the number of vehicles in the area cannot be effectively obtained; (4) When the sensing fails, there is a lack of self-diagnosis method.

[0004] Fixed-point positioning based on ground sensing coils: At each positioning point, two sets of ground sensing coils are pre-buried on the tunnel ground about 5 meters apart. When a vehicle passes over the coils, the controller senses the vehicle based on the change in the coil magnetic field; the direction of the vehicle is sensed by the change sequence of the two sets of coils. It has the following defects and shortcomings: (1) When multiple vehicles pass at the same time, the number and direction of vehicles cannot be sensed; (2) When the sensing fails, there is a lack of self-diagnosis method.

[0005] Positioning based on UWB technology: Install a UWB reader on the side wall of the tunnel, and lead out two antennas at a certain distance to be installed on the top of the tunnel. The coverage range is usually 300 to 400 meters; install a UWB identification card on the vehicle. When the vehicle is driving in the sensing area, the card reader obtains two distance values ​​of the vehicle relative to the two antennas and determines the vehicle position based on the distance values. It has the following defects and shortcomings: (1) Due to the bending and ups and downs of the tunnel, there are blind spots and non-line-of-sight transmission problems; (2) Due to the limited installation position of the identification card, there is signal blocking, causing missed cards and signal loss problems; (3) Since the number of identification cards that can respond in the same period is limited, the installation density of the card reader is limited, and blind spots cannot be filled at will; (4) When perception fails, there is a lack of self-diagnosis methods. Summary of the invention

[0006] To solve the problems existing in the prior art, the present invention provides a multi-modal fusion perception method, device and medium for vehicle position in a restricted space, which can, in a restricted space, fit and correct UWB positioning data vulnerable to interference such as non-line-of-sight transmission and multipath reflection with segmented, small-range and highly accurate data; and at the same time, use different perception technologies to mutually verify to complete the diagnosis of the perception state.

[0007] The technical solution adopted by the present invention to solve its technical problems is: a multi-modal fusion perception method for vehicle position in a restricted space, including:

[0008] S1. Establish a one-dimensional coordinate system OX in the restricted space to obtain a set of coverage areas of each millimeter-wave radar;

[0009] S2. Obtain the positioning data of the UWB card reader on the vehicle through Ethernet, and calculate the position information of k vehicles in the coordinate system OX to form a set A; obtain the vehicle data of the millimeter-wave radar through the CAN bus, and calculate the position information and speed information of n vehicles in the coordinate system OX to form a set B;

[0010] S3. Fit and adjust the error of the vehicle position;

[0011] S4. Dynamically diagnose whether the millimeter-wave radar is abnormal;

[0012] S5. Dynamically diagnose whether the UWB positioning is abnormal.

[0013] Further, the set of coverage areas of each millimeter-wave radar is Fh_1 = {x|m_1 < x < n_1},..., Fh_j = {x|m_j < x < n_j}, where j represents the number of millimeter-wave radars, x represents the signal propagation distance of the millimeter-wave radar, m represents the shortest distance covered by the millimeter-wave radar signal, and n represents the longest distance covered by the millimeter-wave radar signal.

[0014] Further, according to the characteristics of the millimeter-wave radar, construct a weighted function between the credibility p and the radar signal propagation distance x, x0 represents the position where the millimeter radar is installed.

[0015] Further, step S3 specifically includes:

[0016] S31. When card(A) > 1 or card(B) > 1, where card represents the number of elements in the set, when the number of elements in set A or B is greater than 1, it indicates that multiple vehicles pass through the same area at the same time; let set G = φ, where φ represents an empty set;

[0017] S32, set the vehicle process data set G = {(xu, xh, vh) | card(A) = 1, card(B) = 1, xu∈A, xh∈B, vh∈B}; xu represents the vehicle position information obtained through Ethernet, xh represents the vehicle position information obtained through the CAN bus, and vh represents the vehicle speed information obtained through the CAN bus;

[0018] S33, when card(G)≥threshold N1 and card(A)=card(B)=0, use cubic spline interpolation to fit the surface, obtain the relationship xx=f(xu,vh) between the corrected position xx and the UWB positioning data xu and the velocity information vh, and then set G=φ;

[0019] S34, when the millimeter-wave radar does not cover the space or the millimeter-wave radar is abnormal, the positioning data xu and discrete points of time t are used to obtain the speed correction value vu instead of vh after least squares fitting, thereby completing the fitting correction of the position perception error;

[0020] S35. When the UWB signal is lost or there is a perception blind spot, within 3 seconds of the loss of the vehicle's millimeter-wave radar perception data, the vehicle speed curve is obtained by fitting the discrete points of the speed information vh and time t, and the position error is corrected according to the fitted vehicle speed curve.

[0021] Furthermore, step S4 specifically includes:

[0022] Set a first-in-first-out detection queue Qh with a length of N2. When the vehicle passes through the i-th millimeter-wave radar sensing area, the data that satisfies xx∈Fh_i, card(B∩Fh_i)≠φ will be combined into a set Ph. When card(Ph)≥threshold N3, calculate the weighted average Otherwise, a fixed constant N3 is used instead of card(Ph) to calculate the weighted average, and the calculation result is filled into the queue Qh; when ΣQh≤0.6×N2, the millimeter-wave radar state is judged to be abnormal; when ΣQh>0.7×N2, the millimeter-wave radar state is judged to be normal, and the set Ph=φ.

[0023] Furthermore, step S5 specifically includes:

[0024] Set a first-in-first-out detection queue Qu with a length of N2 and a speed threshold value v0. When the vehicle speed detected by the millimeter-wave radar meets When ΣQu≤0.4×N2, the UWB positioning is judged to be abnormal; when ΣQu>0.5×N2, the UWB positioning is judged to be normal.

[0025] The present invention also provides a computer device, comprising:

[0026] processor;

[0027] A memory for storing executable instructions;

[0028] Wherein, the processor is used to read the executable instructions from the memory and execute the executable instructions to implement the multimodal fusion perception method of the vehicle position in a confined space.

[0029] The present invention also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the processor implements the multimodal fusion perception method of the vehicle position in a confined space.

[0030] The beneficial effects of the present invention are:

[0031] The present invention uses a millimeter-wave radar with relatively accurate local ranging to dynamically fit and correct the position perception results. Even if the interference factors change after the equipment is installed, the vehicle position perception effect can be improved. The multi-source perception method can improve the diagnosis of the perception technology status and detect abnormalities or faults in time. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] The present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0033] Figure 1 It is a flow chart of the multimodal fusion perception method of the vehicle position in a confined space of the present invention.

[0034] Figure 2 It is a flow chart of the perceptual error fitting adjustment of the present invention.

[0035] Figure 3 is a flow chart of the perception state diagnosis of the present invention.

[0036] Figure 4 It is a schematic diagram of the vehicle of the present invention traveling in a confined space. DETAILED DESCRIPTION

[0037] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, which only illustrate the basic structure of the present invention in a schematic manner, and therefore only show the components related to the present invention.

[0038] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", "axial", "radial", "circumferential" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present invention. In addition, features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, unless otherwise specified, "multiple" means two or more.

[0039] In the description of the present invention, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0040] like Figures 1 to 4 As shown, the multimodal fusion perception method of the vehicle position in a confined space of the present invention includes: S1. Establish a one-dimensional coordinate system OX in the confined space to obtain a set of coverage areas of each millimeter-wave radar. S2. Obtain the positioning data of the UWB card reader on the vehicle through Ethernet, and calculate the position information of k vehicles in the coordinate system OX to form a set A; obtain the vehicle data of the millimeter-wave radar through the CAN bus, and calculate the position information and speed information of n vehicles in the coordinate system OX to form a set B. S3. Fit and adjust the error of the vehicle position. S4. Dynamically diagnose whether the millimeter-wave radar is abnormal. S5. Dynamically diagnose whether the UWB positioning is abnormal.

[0041] It should be noted that the confined space mainly refers to the underground roadway of a mine, whose width and height are usually only a few meters. At the intersection of roadways, usually only the distance of the vehicle from the intersection is concerned to intelligently control the passing sequence of vehicles. Therefore, a one-dimensional coordinate system can be established, which can also simplify the calculation process. A UWB reader is installed on each vehicle, and UWB positioning base stations are arranged in the roadway. When the vehicle is driving in the roadway, signal transmission can be carried out between the UWB reader and the UWB positioning base station to determine the current position of the vehicle. Multiple millimeter-wave radars are installed on the roadway wall. When the vehicle passes by the millimeter-wave radar, the millimeter-wave radar can feedback the position information and speed information of the vehicle. The signal coverage distance of the UWB positioning base station is 300m - 400m, and the signal coverage distance of the millimeter-wave radar is 50m.

[0042] The set of the coverage areas of each millimeter-wave radar is Fh_1 = {x|m_1 < x < n_1},..., Fh_j = {x|m_j < x < n_j}, where j represents the number of millimeter-wave radars, x represents the signal propagation distance of the millimeter-wave radar, m represents the shortest distance covered by the millimeter-wave radar signal, and n represents the longest distance covered by the millimeter-wave radar signal. According to the characteristics of the millimeter-wave radar, a weighted function between the credibility p and the radar signal propagation distance x is constructed. x0 represents the position where the millimeter radar is installed. The height and width of the confined space are limited, and the radar signal has a certain coverage angle. When the distances are different, affected by multipath transmission and signal absorption, the detection accuracy will vary. Therefore, the present invention sets different calculation formulas for the weighted function according to different distances.

[0043] Step S3 specifically includes:

[0044] S31. When card(A) > 1 or card(B) > 1, where card represents the number of elements in the set (i.e., the number of vehicles included). When card(A) > 1 or card(B) > 1, it indicates that there is more than one vehicle in set A or B, and multiple vehicles pass through this area simultaneously. At this time, let set G = φ, where φ represents an empty set.

[0045] S32. Set the vehicle process data set G = {(xu, xh, vh)|card(A) = 1, card(B) = 1, xu ∈ A, xh ∈ B, vh ∈ B}; xu represents the position information of the vehicle obtained through Ethernet, xh represents the position information of the vehicle obtained through the CAN bus, and vh represents the speed information of the vehicle obtained through the CAN bus. When card(A) = 1 and card(B) = 1, it indicates that there is only one vehicle passing through this area currently, and the mutual influence of signals between vehicles can be excluded. When only one vehicle passes through this area, the position information and speed information (xu, xh, vh) of this vehicle can be obtained.

[0046] S33, when card(G)≥threshold N1 and card(A)=card(B)=0, use the cubic spline interpolation method to fit the surface, and obtain the relationship between the corrected position xx and the UWB positioning data xu and speed information vh xx=f(xu,vh), and then let the set G=φ. When card(A)=card(B)=0, it indicates that there is no vehicle passing through the area at this time, but a car is driving towards the area from a distance. At this time, the driving process data of the car can be used as a correction reference. Use the data of the car to fit the surface using the cubic spline difference method, and obtain xx=f(xu,vh), xx=a*xu+b*vh+c, a, b, c are coefficients, a>b, c depends on the coordinate position of the UWB reader. At this time, let the set G be an empty set in order to allow each reference vehicle to participate in the fitting process as an independent individual to reduce the impact of equipment aging.

[0047] S34. When the millimeter-wave radar does not cover the space or the millimeter-wave radar is abnormal, the positioning data xu and the discrete points of time t are used to obtain the speed correction value vu instead of vh after least squares fitting to complete the fitting correction of the position perception error. The positioning data xu refers to the position calculated by UWB. The function of position and time is the speed expression. When the millimeter-wave radar fails, the speed correction value obtained by fitting the position and time measured by UWB is used to replace the speed value measured by the millimeter-wave radar, which can reduce the impact of data time difference and improve the accuracy of the perception result.

[0048] S35. When the UWB signal is lost or there is a blind spot, within 3 seconds of the loss of the vehicle's millimeter-wave radar perception data, the vehicle speed curve is obtained by fitting the discrete points of the speed information vh and time t, and the position error is corrected according to the fitted vehicle speed curve. If the UWB signal fails, the speed data and time of the millimeter-wave radar can be used to fit the corrected speed value. For example, assuming that the last coordinate position before the UWB positioning data is lost is x p The average speed within 3s is vh p (positive or negative, indicating the direction of vehicle travel), then the corrected position is x'=x p +vh p *t.

[0049] Step S4 specifically includes: setting a first-in-first-out detection queue Qh with a length of N2. When a vehicle passes through the i-th millimeter-wave radar sensing area, the data satisfying xx∈Fh_i, card(B∩Fh_i)≠φ are combined into a set Ph. The set Ph represents a set of vehicle position data located in a certain millimeter-wave radar coverage area detected by the UWB card reader and the millimeter-wave radar at the same time. When card(Ph)≥threshold N3, calculate the weighted average value Otherwise, a fixed constant N3 is used instead of card(Ph) to calculate the weighted average, and the calculation result is filled into the queue Qh; when ΣQh≤0.6×N2, the millimeter-wave radar state is judged to be abnormal; when ΣQh>0.7×N2, the millimeter-wave radar state is judged to be normal, and the set Ph=φ.

[0050] Step S5 specifically includes: setting a first-in-first-out detection queue Qu with a length of N2, a speed threshold value v0, and when the vehicle speed detected by the millimeter-wave radar meets When ΣQu≤0.4×N2, the UWB positioning is judged to be abnormal; when ΣQu>0.5×N2, the UWB positioning is judged to be normal.

[0051] The values ​​of N1, N2, and N3 can be set according to actual scenarios.

[0052] A computer device of the present invention includes a processor and a memory, wherein the memory stores at least one instruction or at least one program, and the at least one instruction or the at least one program is loaded and executed by the processor to implement a multimodal fusion perception method for a vehicle position in a confined space as provided in the above-mentioned embodiment.

[0053] A computer-readable storage medium of the present invention can be set in a server to store at least one instruction or at least one program related to a multimodal fusion perception method for a vehicle position in a confined space, and the at least one instruction or the at least one program is loaded and executed by the processor to implement a multimodal fusion perception method for a vehicle position in a confined space. Optionally, the above-mentioned storage medium can be located in at least one network server among multiple network servers of a computer network. Optionally, the above-mentioned storage medium can include but is not limited to: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

[0054] The present invention uses two signals, UWB and millimeter-wave radar, to notify the sensing vehicle. The millimeter-wave radar data can be detected in segments and in a small range to compensate for the wide coverage of UWB signals and susceptibility to interference from non-line-of-sight transmission, multipath reflection, etc. When one signal fails, the data of the other signal can be used to correct errors, complementing each other. In addition, the status of the two sensing technologies is also detected in real time during the sensing process to reduce the risk of errors in the sensing results. Under different usage environments, the position perception of dynamic targets is affected by errors such as non-line-of-sight transmission and multipath reflection. The present invention uses a millimeter-wave radar with more accurate local ranging to dynamically fit and correct the position perception results. Even if the interference factors change after the equipment is installed, the vehicle position perception effect can be improved; the use of multi-source sensing methods can improve the diagnosis of the state of the sensing technology and detect abnormalities or faults in a timely manner.

[0055] Based on the above ideal embodiments of the present invention, the relevant staff can make various changes and modifications without departing from the technical concept of the present invention through the above description. The technical scope of the present invention is not limited to the contents of the specification, but must be determined according to the scope of the claims.

Claims

1. A multimodal fusion perception method for vehicle position in a confined space, characterized in that: Including: S1. Establish a one-dimensional coordinate system OX within a restricted space to obtain a set of coverage areas of each millimeter-wave radar; S2. Obtain the positioning data of the UWB reader on the vehicle through Ethernet, and calculate the position information of k vehicles in the coordinate system OX to form a set A; obtain the vehicle data of the millimeter-wave radar through the CAN bus, and calculate the position information and speed information of n vehicles in the coordinate system OX to form a set B; S3. Fit and adjust the error of the vehicle position; S4. Dynamically diagnose whether the millimeter-wave radar is abnormal; S5. Dynamically diagnose whether the UWB positioning is abnormal.

2. The multimodal fusion perception method for vehicle position in a confined space as claimed in claim 1, characterized in that: The set of coverage areas of each millimeter-wave radar is Fh_1 = {x|m_1 < x < n_1},..., Fh_j = {x|m_j < x < n_j}, where j represents the number of millimeter-wave radars, x represents the signal propagation distance of the millimeter-wave radar, m represents the shortest distance covered by the millimeter-wave radar signal, and n represents the longest distance covered by the millimeter-wave radar signal.

3. The multi-modal fusion perception method for vehicle position in a confined space as claimed in claim 2, characterized in that: According to the characteristics of millimeter wave radar, a weighted function between the credibility p and the radar signal propagation distance x is constructed. x0 represents the location where the millimeter radar is installed.

4. The multi-modal fusion perception method for vehicle position in a confined space as claimed in claim 3, characterized in that: Step S3 specifically includes: S31. When card(A) > 1 or card(B) > 1, where card represents the number of elements in the set, when the number of elements in set A or B is greater than 1, it indicates that multiple vehicles pass through the same area simultaneously; let set G = φ, where φ represents an empty set; S32. Set the vehicle process data set G = {(xu, xh, vh)|card(A) = 1, card(B) = 1, xu ∈ A, xh ∈ B, vh ∈ B}; xu represents the position information of the vehicle obtained through Ethernet, xh represents the position information of the vehicle obtained through the CAN bus, and vh represents the speed information of the vehicle obtained through the CAN bus; S33. When card(G) ≥ threshold N1 and card(A) = card(B) = 0, use the cubic spline interpolation method to fit the surface, and obtain the relationship xx = f(xu, vh) between the corrected position xx, the UWB positioning data xu, and the speed information vh, and then let set G = φ; S34. When the millimeter-wave radar does not cover the space or the millimeter-wave radar is abnormal, use the discrete points of the positioning data xu and time t, and after fitting by the least squares method, obtain the speed correction value vu to replace vh, and complete the fitting correction of the position perception error; S35. When the UWB signal is lost or there is a perception blind area, within 3 seconds of the loss of the millimeter-wave radar perception data of the vehicle, use the discrete points of the speed information vh and time t to fit to obtain the vehicle speed curve, and correct the position error according to the fitted vehicle speed curve.

5. The multi-modal fusion perception method for vehicle position in a confined space as claimed in claim 4, characterized in that: Step S4 specifically includes: Set a first-in-first-out detection queue Qh with a length of N2. When the vehicle passes through the i-th millimeter-wave radar sensing area, the data that satisfies xx∈Fh_i, card(B∩Fh_i)≠φ will be combined into a set Ph. When card(Ph)≥threshold N3, calculate the weighted average Otherwise, a fixed constant N3 is used instead of card(Ph) to calculate the weighted average, and the calculation result is filled into the queue Qh; when ΣQh≤0.6×N2, the millimeter-wave radar state is judged to be abnormal; when ΣQh>0.7×N2, the millimeter-wave radar state is judged to be normal, and the set Ph=φ.

6. The multi-modal fusion perception method for vehicle position in a confined space as claimed in claim 4, characterized in that: Step S5 specifically includes: Set a first-in-first-out detection queue Qu with a length of N2 and a speed threshold value v0. When the vehicle speed detected by the millimeter-wave radar meets When ΣQu≤0.4×N2, the UWB positioning is judged to be abnormal; when ΣQu>0.5×N2, the UWB positioning is judged to be normal.

7. A computer device, characterized in that: Including: A processor; A memory for storing executable instructions; Wherein, the processor is used to read the executable instructions from the memory and execute the executable instructions to implement the multi-modal fusion perception method for vehicle position under restricted space as described in any one of claims 1 to 6.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by the processor, the processor is enabled to implement the multi-modal fusion perception method for vehicle position under restricted space as described in any one of claims 1 to 6.

Images