Autonomous following control method and device based on multi-source perception, equipment, medium and product

Abstract

The invention discloses an autonomous following control method, device and equipment based on multi-source perception, a medium and a product, and relates to the technical field of coal mine intelligent mining. The method comprises the following steps: acquiring multi-source data; preprocessing the multi-source data to obtain preprocessed multi-source data; on the basis of the preprocessed multi-source data, CFAR algorithm-based detection, visual processing, magnetic field detection and position calculation are carried out to obtain processed data; performing three-machine cooperative control according to the processed data so as to complete one operation cycle of the complete equipment of the coal mine driving working face; wherein the operation cycle comprises autonomous following of the heading machine and the shuttle car, autonomous butt joint and unloading of the shuttle car and the crusher, and autonomous returning of the shuttle car after unloading. The autonomous following control of the complete equipment of the coal mine driving face can be realized based on the multi-source sensing fusion technology.

Description

Technical Field

[0001] This application relates to the field of intelligent coal mining technology, and in particular to an autonomous following control method, device, equipment, medium and product based on multi-source sensing. Background Technology

[0002] Currently, single-sensor guidance (such as UWB or laser) is used in coal mining, but its reliability is poor in dusty environments, and the three-machine coordination of coal mining equipment relies on preset tracks or manual intervention, resulting in low flexibility.

[0003] In addition, current mining follow-up control relies on traditional magnetic track navigation, which requires pre-laying magnetic strips, resulting in high maintenance costs and insufficient dynamic obstacle avoidance capabilities, making it easy for operations to be interrupted due to temporary obstacles.

[0004] Therefore, there is an urgent need for a method that enables autonomous following and control to realize coal mining. Summary of the Invention

[0005] The purpose of this application is to provide an autonomous following control method, device, equipment, medium and product based on multi-source sensing, which can realize autonomous following control of complete sets of equipment in coal mine tunneling faces based on multi-source sensing fusion technology.

[0006] To achieve the above objectives, this application provides the following solution: In a first aspect, this application provides an autonomous following control method based on multi-source perception, which is applied to a complete set of equipment for a coal mine tunneling face. The complete set of equipment for a coal mine tunneling face includes: a tunneling machine, a shuttle car, and a transfer crusher. A machine vision device is arranged on the top of the complete set of equipment for a coal mine tunneling face, a 4D millimeter-wave radar is arranged in the forward or backward direction of the complete set of equipment for a coal mine tunneling face, and a magnetic sensor array is installed at the tail or head of the complete set of equipment for a coal mine tunneling face. The autonomous following control method based on multi-source sensing includes: Acquire multi-source data; the multi-source data is collected based on a three-level vision-radar-magnetic sensing system; the three-level vision-radar-magnetic sensing system is constructed based on machine vision devices, 4D millimeter-wave radar, and magnetic sensor arrays; The multi-source data is preprocessed to obtain preprocessed multi-source data; Based on the preprocessed multi-source data, detection, visual processing, magnetic field detection and position calculation based on the CFAR algorithm are performed to obtain processed data; Based on the processed data, the three machines are coordinated and controlled to complete one work cycle of the complete set of equipment at the coal mine tunneling face; wherein, the work cycle includes: autonomous following of the tunneling machine and the shuttle car, autonomous docking and unloading of the shuttle car and the crusher, and autonomous return of the shuttle car after unloading.

[0007] In one embodiment, based on the preprocessed multi-source data, detection, visual processing, magnetic field detection, and position calculation based on the CFAR algorithm are performed to obtain processed data, specifically including: Based on the 4D millimeter-wave radar detection data contained in the preprocessed multi-source data, the CFAR algorithm is used for global perception and positioning processing to obtain azimuth information and perform path planning. In addition, the conversion between polar coordinates and Cartesian coordinates is performed to determine the docking area information. Based on the image data corresponding to the machine vision device contained in the preprocessed multi-source data, visual processing is performed to determine the visual output, and the visual output is fused with 4D millimeter-wave radar detection data to optimize the path information obtained from path planning. The visual output includes: lateral deviation, heading deviation, and longitudinal distance. The lateral deviation is the lateral distance between the center point of the tunnel boring machine's tail and the center line of the shuttle car, or the lateral distance between the center point of the shuttle car's tail and the center line of the crusher. The heading deviation is the angle between the orientation of the tunnel boring machine's tail and the orientation of the shuttle car, or the angle between the orientation of the shuttle car's tail and the orientation of the crusher. The longitudinal distance is the straight-line distance between the tunnel boring machine and the shuttle car, or between the shuttle car and the crusher. Based on the three-dimensional magnetic field strength vector contained in the preprocessed multi-source data, magnetic field detection is performed to obtain magnetic field strength distribution information. Based on the magnetic field strength distribution information, the magnetic field gradient method is used to calculate the position using the magnetic field strength as a weight to obtain the position offset. The orientation information, the path planning information, the docking area information, the visual output, the optimized path information, and the position offset are determined as the processing data.

[0008] In one embodiment, three-machine coordinated control is performed based on the processed data to complete one work cycle of the complete set of equipment at the coal mine tunneling face, specifically including: During the autonomous following phase of the tunneling machine and shuttle car: In the medium-to-long distance stage where the distance between the tunneling machine and the shuttle car is greater than 10 meters, the tunneling machine is controlled to carry out tunneling operations according to the preset path, and the shuttle car is controlled to control its own speed and direction based on the following instructions, according to the orientation information, the information obtained from the path planning, and the distance and speed information contained in the 4D millimeter-wave radar detection data, so that it maintains a set safe distance from the tunneling machine and carries out following operations. During the close-range following phase where the distance between the tunneling machine and the shuttle car is less than 10 meters and not less than 1 meter, based on the docking area information, the visual output, and the optimized path information, the shuttle car is controlled to travel on the center path behind the tunneling machine to carry out subsequent receiving and unloading operations. During the docking phase when the distance between the tunneling machine and the shuttle car is less than 1 meter, the shuttle car is controlled to adjust its posture according to the position offset. The magnetic signal strength detected in real time by the magnetic attraction sensor array is compared with a predetermined threshold. When the detected magnetic signal strength is greater than the predetermined threshold, the shuttle car is controlled to stop moving and lift the receiving hopper, and the tunneling machine is controlled to unload the material into the receiving hopper of the shuttle car. During the autonomous docking and unloading phase between the shuttle car and the crusher: After detecting that the shuttle is fully loaded, the control shuttle will drive to the crusher at a fixed location based on the corresponding orientation information, the information obtained from the path planning, and the docking area information. In the medium-to-long distance stage where the distance between the shuttle car and the crusher is greater than 10 meters, the shuttle car is controlled to move towards the crusher according to the preset path. Based on the corresponding following instructions, the shuttle car is controlled to control its own speed and direction according to the corresponding orientation information, the information obtained from the path planning, and the distance and speed information contained in the 4D millimeter-wave radar detection data, so that it gradually approaches the crusher. During the mid-distance docking phase, where the distance between the shuttle car and the crusher is less than 10 meters and not less than 1 meter, the shuttle car is controlled to move towards the crusher based on the visual output and the optimized path information to prepare for docking. During the precision alignment stage where the distance between the shuttle car and the crusher is less than 1 meter, the shuttle car is controlled to adjust its posture according to the corresponding position offset, so that the shuttle car moves towards the receiving hopper of the crusher. The magnetic signal strength detected in real time by the magnetic attraction sensor array is compared with a predetermined threshold. When the detected magnetic signal strength is greater than the predetermined threshold, the shuttle car is controlled to stop moving and lift the unloading hopper so that the material is unloaded into the receiving hopper of the crusher. During the autonomous return phase of the shuttle car after unloading: Once the shuttle car has finished unloading, the shuttle car is controlled to return to the location of the tunneling machine based on the return control command, the corresponding orientation information and the information obtained from the path planning, until the distance between the shuttle car and the tunneling machine reaches the conditions corresponding to the autonomous following stage of the tunneling machine and the shuttle car, so as to start a new work cycle.

[0009] In one embodiment, visual processing is performed based on the image data corresponding to the machine vision device contained in the preprocessed multi-source data to determine the visual output quantity, specifically including: The image data is subjected to distortion correction, noise reduction, and contrast enhancement to obtain processed image data; the image data includes: a color image and a depth image; For the corresponding color image in the processed image data, the YOLO algorithm, a deep learning-based object detection network, is used to identify feature points and obtain the corresponding pixel coordinates. Based on the pixel coordinates, the depth value is extracted from the corresponding position of the depth image in the processed image data; the depth value is the straight-line distance between the corresponding position point and the RGB-D camera included in the machine vision device. Based on the intrinsic parameter matrix of the RGB-D camera, the dataset composed of pixel coordinates and depth values ​​is converted into three-dimensional coordinates of feature points relative to the camera coordinate system in order to solve the visual pose. Based on the rigid mounting relationship between the RGB-D camera and the shuttle car body, the transformation matrix is ​​determined, and the visual pose is transformed from the camera coordinate system to the shuttle car body coordinate system to obtain the visual output.

[0010] In one embodiment, based on 4D millimeter-wave radar detection data contained in preprocessed multi-source data, the CFAR algorithm is used for global perception and positioning processing to obtain azimuth information and perform path planning. Furthermore, a conversion between polar coordinates and Cartesian coordinates is performed to determine docking area information. Specifically, this includes: A three-dimensional data cube is determined; the three-dimensional data cube is determined based on the 4D millimeter-wave radar detection data contained in the preprocessed multi-source data after performing range FFT, Doppler FFT and angle FFT processing respectively; A sliding window is constructed using the CFAR algorithm and traversed through the three-dimensional data cube to determine the orientation information, which includes: positioning distance, angle, radial velocity, and radar cross section. Based on the location information, path planning and conversion between polar and Cartesian coordinates are performed to obtain docking area information.

[0011] In one embodiment, the multi-source data is preprocessed to obtain preprocessed multi-source data, specifically including: The multi-source data is normalized to obtain normalized multi-source data; The normalized multi-source data is identified as the preprocessed multi-source data.

[0012] Secondly, this application provides an autonomous following control device based on multi-source sensing, comprising: A multi-source data acquisition module is used to acquire multi-source data; the multi-source data is acquired based on a vision-radar-magnetic three-level perception system; the vision-radar-magnetic three-level perception system is constructed based on a machine vision device, a 4D millimeter-wave radar, and a magnetic sensor array; The preprocessing module is used to preprocess the multi-source data to obtain preprocessed multi-source data; The data processing and determination module is used to perform CFAR-based detection, visual processing, magnetic field detection, and position calculation based on preprocessed multi-source data to obtain processed data. The collaborative control module is used to perform three-machine collaborative control based on the processed data to complete one work cycle of the complete set of equipment at the coal mine tunneling face; wherein, the work cycle includes: autonomous following of the tunneling machine and shuttle car, autonomous docking and unloading of the shuttle car and crusher, and autonomous return of the shuttle car after unloading.

[0013] Thirdly, this application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the above-described autonomous following control method based on multi-source perception.

[0014] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the aforementioned autonomous following control method based on multi-source perception.

[0015] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the aforementioned autonomous following control method based on multi-source perception.

[0016] According to the specific embodiments provided in this application, the following technical effects are disclosed: This application provides an autonomous following control method, device, equipment, medium, and product based on multi-source sensing. This application adopts multi-source sensing fusion technology, collects multi-source data based on a three-level sensing system of vision, radar, and magnetic attraction, processes the multi-source data, and performs three-machine coordinated control to complete one work cycle of the complete set of equipment in a coal mine tunneling face. That is, to perform autonomous following of the tunneling machine and shuttle car, autonomous docking and unloading of the shuttle car and crusher, and autonomous return of the shuttle car after unloading, thereby realizing autonomous following control of the complete set of equipment in a coal mine tunneling face. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 The flowchart shows an autonomous following control method based on multi-source sensing. Figure 2 This is a schematic diagram illustrating the operational steps of the autonomous following control method based on multi-source sensing in practical applications. Figure 3 A schematic diagram of the composition and structure of a complete set of equipment for a coal mine tunneling face; Figure 4 This is a structural diagram of an autonomous following control device based on multi-source sensing. Figure 5 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0019] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0020] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0021] In one exemplary embodiment, this application provides an autonomous following control method based on multi-source sensing, which is applied to a complete set of equipment in a coal mine tunneling face; such as Figure 3 As shown, the complete set of equipment for the coal mine tunneling face includes: a tunneling machine, a shuttle car, and a transfer crusher. A machine vision device (such as a camera) is arranged on the top of the complete set of equipment for the coal mine tunneling face; 4D millimeter-wave radars are arranged in front of or behind the complete set of equipment for the coal mine tunneling face; and a magnetic sensor array is installed at the tail or head of the complete set of equipment for the coal mine tunneling face.

[0022] like Figure 1 As shown, the autonomous following control method based on multi-source sensing includes: Step 100: Acquire multi-source data. The multi-source data is acquired based on a three-level vision-radar-magnetic sensing system; the three-level vision-radar-magnetic sensing system is constructed based on machine vision devices, 4D millimeter-wave radar, and magnetic sensor arrays.

[0023] Step 200: Preprocess the multi-source data to obtain preprocessed multi-source data.

[0024] This includes preprocessing the multi-source data to obtain preprocessed multi-source data, specifically including: The multi-source data is normalized to obtain normalized multi-source data; this normalized multi-source data is then identified as the preprocessed multi-source data. This process transforms raw sensor data with different physical characteristics into unified pose and environmental information with practical physical meaning, preparing for subsequent fusion.

[0025] Step 300: Based on the preprocessed multi-source data, perform detection, visual processing, magnetic field detection, and position calculation based on the CFAR algorithm to obtain processed data.

[0026] Specifically, based on the preprocessed multi-source data, detection, visual processing, magnetic field detection, and position calculation based on the CFAR algorithm are performed to obtain processed data, which includes: Based on the 4D millimeter-wave radar detection data contained in the preprocessed multi-source data, the CFAR algorithm is used for global perception and positioning processing to obtain azimuth information and perform path planning. In addition, the conversion between polar coordinates and Cartesian coordinates is performed to determine the docking area information.

[0027] Based on the image data corresponding to the machine vision device contained in the preprocessed multi-source data, visual processing is performed to determine the visual output. This visual output is then fused with 4D millimeter-wave radar detection data to optimize the path information obtained from path planning. The visual output includes: lateral deviation, heading deviation, and longitudinal distance. The lateral deviation is the lateral distance between the center point of the tunnel boring machine's tail and the center line of the shuttle car, or the lateral distance between the center point of the shuttle car's tail and the center line of the crusher. The heading deviation is the angle between the orientation of the tunnel boring machine's tail and the orientation of the shuttle car, or the angle between the orientation of the shuttle car's tail and the orientation of the crusher. The longitudinal distance is the straight-line distance between the tunnel boring machine and the shuttle car, or between the shuttle car and the crusher.

[0028] Based on the three-dimensional magnetic field strength vector contained in the preprocessed multi-source data, magnetic field detection is performed to obtain magnetic field strength distribution information. Based on the magnetic field strength distribution information, the magnetic field gradient method is used to calculate the position using the magnetic field strength as a weight, and the position offset is obtained.

[0029] The orientation information, path planning information, docking area information, visual output, optimized path information, and position offset are determined as the processing data.

[0030] Step 400: Perform three-machine coordinated control based on the processed data to complete one work cycle of the complete set of equipment at the coal mine tunneling face. The work cycle includes: autonomous following of the tunneling machine and shuttle car, autonomous docking and unloading of the shuttle car and crusher, and autonomous return of the shuttle car after unloading.

[0031] This includes the coordinated control of three machines based on processed data to complete one work cycle of the complete set of equipment at the coal mine tunneling face, specifically including: During the autonomous following phase of the tunneling machine and shuttle car: In the medium-to-long distance stage where the distance between the tunneling machine and the shuttle car is greater than 10 meters, the tunneling machine is controlled to carry out tunneling operations according to the preset path, and the shuttle car is controlled to control its own speed and direction based on the following instructions, according to the orientation information, the information obtained from the path planning, and the distance and speed information contained in the 4D millimeter-wave radar detection data, so that it maintains a set safe distance from the tunneling machine and follows it.

[0032] During the close-range following phase where the distance between the tunneling machine and the shuttle car is less than 10 meters and not less than 1 meter, the shuttle car is controlled to travel on the center path behind the tunneling machine based on docking area information, visual output, and optimized path information, in order to carry out subsequent material receiving and unloading operations.

[0033] During the docking phase when the distance between the tunneling machine and the shuttle car is less than 1 meter, the control shuttle car adjusts its posture according to the position offset. The magnetic signal strength detected in real time by the magnetic attraction sensor array is compared with a predetermined threshold. When the detected magnetic signal strength is greater than the predetermined threshold, the control shuttle car stops moving and lifts the receiving hopper, and the control tunneling machine unloads the material into the receiving hopper of the shuttle car.

[0034] During the autonomous docking and unloading phase between the shuttle car and the crusher: After detecting that the shuttle is fully loaded, the control shuttle moves to the crusher at a fixed location based on the corresponding orientation information, path planning information, and docking area information.

[0035] In the medium-to-long distance stage where the distance between the shuttle car and the crusher is greater than 10 meters, the shuttle car is controlled to move towards the crusher according to a preset path. Based on the corresponding following instructions, the shuttle car controls its speed and direction according to the corresponding orientation information, the information obtained from path planning, and the distance and speed information contained in the 4D millimeter-wave radar detection data, so that it gradually approaches the crusher.

[0036] During the mid-distance docking phase, where the distance between the shuttle car and the crusher is less than 10 meters and not less than 1 meter, the shuttle car is controlled to move towards the crusher based on the visual output and the optimized path information to prepare for docking.

[0037] During the precision alignment stage where the distance between the shuttle car and the crusher is less than 1 meter, the shuttle car is controlled to adjust its posture according to the corresponding position offset so that the shuttle car moves towards the receiving hopper of the crusher. The magnetic signal strength detected in real time by the magnetic attraction sensor array is compared with a predetermined threshold. When the detected magnetic signal strength is greater than the predetermined threshold, the shuttle car is controlled to stop moving and lift the unloading hopper so that the material is unloaded into the receiving hopper of the crusher.

[0038] During the autonomous return phase of the shuttle car after unloading: Once the shuttle car has finished unloading, the shuttle car is controlled to return to the location of the tunneling machine based on the return control command, the corresponding orientation information and the information obtained from the path planning, until the distance between the shuttle car and the tunneling machine reaches the conditions corresponding to the autonomous following stage of the tunneling machine and the shuttle car, so as to start a new work cycle.

[0039] As an optional implementation, visual processing is performed on the image data corresponding to the machine vision device contained in the preprocessed multi-source data to determine the visual output quantity, specifically including: The image data is subjected to distortion correction, noise reduction, and contrast enhancement to obtain the processed image data; the image data includes color images and depth images.

[0040] For the corresponding color image in the processed image data, the YOLO algorithm, a deep learning-based object detection network, is used to identify feature points and obtain the corresponding pixel coordinates.

[0041] Based on pixel coordinates, the depth value is extracted from the corresponding position of the depth image in the processed image data; the depth value is the straight-line distance between the corresponding position point and the RGB-D camera included in the machine vision device.

[0042] Based on the intrinsic parameter matrix of the RGB-D camera, the dataset consisting of pixel coordinates and depth values ​​is converted into three-dimensional coordinates of feature points relative to the camera coordinate system in order to solve the visual pose.

[0043] Based on the rigid mounting relationship between the RGB-D camera and the shuttle car body, the transformation matrix is ​​determined, and the visual pose is transformed from the camera coordinate system to the shuttle car body coordinate system to obtain the visual output.

[0044] In one embodiment, based on 4D millimeter-wave radar detection data contained in preprocessed multi-source data, the CFAR algorithm is used for global perception and positioning processing to obtain azimuth information and perform path planning. Furthermore, a conversion between polar coordinates and Cartesian coordinates is performed to determine docking area information. Specifically, this includes: The three-dimensional data cube is determined based on the 4D millimeter-wave radar detection data contained in the preprocessed multi-source data after performing range FFT, Doppler FFT and angle FFT processing respectively.

[0045] A sliding window is constructed using the CFAR algorithm, and the three-dimensional data cube is traversed to determine the orientation information, which includes: positioning distance, angle, radial velocity, and radar cross section.

[0046] Based on the orientation information, path planning and conversion between polar and Cartesian coordinates are performed to obtain docking area information.

[0047] This application employs multi-source perception fusion technology based on machine vision, 4D millimeter-wave radar, and magnetic sensors to achieve autonomous following control of complete sets of equipment (such as tunneling machines, shuttle cars, and transfer crushers) in coal mine tunneling faces. Figure 2 As shown, the operational steps in practical applications are as follows: Step 1: Data acquisition, constructing a three-level perception system of vision, radar, and magnetic attraction: A machine vision device (RGB-D camera) is installed on the top of the complete set of equipment at the coal mine tunneling face to identify specific optical markings (such as highly reflective QR codes and special pattern labels) installed at the tail of the tunneling machine. It calculates the high-precision lateral deviation and heading angle with the tunneling machine to achieve precise "micro-operation" alignment and is responsible for the position and posture recognition of the shuttle car and the tail of the tunneling machine within 10m.

[0048] 4D millimeter-wave radars are deployed in front of or behind the equipment (complete set of equipment for coal mine tunneling faces) to continuously detect the distance, relative speed, azimuth, and pitch angle of the tunneling machine (leading unit) in front, and to perform point cloud imaging to perceive environmental obstacles ahead, forming a wide-range, medium-to-long-range "macro" field of view. This perception is unaffected by light, dust, or water mist, and can achieve dynamic obstacle detection and multi-target tracking within 30m.

[0049] A magnetic sensor array is installed at the tail or head of the equipment. When the shuttle car approaches the tunneling machine or crusher, it detects the magnetic field strength of the active transmitting magnetic beacon pre-installed below the discharge port of the tunneling machine or the feed port of the crusher, guiding the shuttle car to complete the final fine alignment with the feed port (centimeter-level accuracy) for precise docking position verification within a range of ±50cm.

[0050] Step 2: Host computer processing: multi-source data preprocessing and independent calculation, namely, sensor fusion (multi-source data fusion based on the vision-radar-magnetic three-level perception system), using YOLOv5s for feature point extraction, DBSCAN point cloud clustering and magnetic field gradient calculation, and then performing UKF filtering.

[0051] Execution entity: Vehicle control unit (VCU) on each mobile device.

[0052] Step 3: Three-machine coordinated control process.

[0053] Process 1: Autonomous following between the tunneling machine and the shuttle car ("mining-transportation" stage): The tunneling machine performs tunneling operations according to the set path.

[0054] After receiving the "follow vehicle" command, the shuttle car activates the multi-source sensing system.

[0055] Mid-to-long range (>10 meters): 4D millimeter-wave radar is the primary sensor. The millimeter-wave radar detects the initial position of the shuttle car behind, first using a CFAR (Constant False Alarm Rate) detection algorithm, with a false alarm rate <10%. -6 Then perform a polar coordinate to Cartesian coordinate transformation.

[0056] Among them, the CFAR (Constant False Alarm Rate) detection algorithm: Input data: The input to CFAR processing is a three-dimensional data cube (Range-Doppler-Azimuth Map) processed by range FFT, Doppler FFT, and angle FFT. Each cell of this data cube contains a signal strength value (power or amplitude).

[0057] Sliding Reference Window: The algorithm creates a "sliding window" that iterates through and evaluates each cell in the data cube. The window structure is as follows: Cell Under Test (CUT): The core cell currently being determined to be the target.

[0058] Reference Cells: A ring of cells surrounding the CUT, used to estimate the average level of background noise.

[0059] Guard Cells: A ring of cells adjacent to the CUT to prevent target energy from leaking into the reference cells and causing estimation errors.

[0060] Estimating background noise: Calculate the average signal strength (CA-CFAR) across all reference cells.

[0061] Set the detection threshold: Multiply the calculated average noise power by a threshold factor (T). This factor is calculated mathematically based on the desired false alarm rate P_fa (usually related to a probability density function, such as the Rayleigh distribution). A lower P_fa (e.g., 10) is preferred. -8 The larger the T value, the higher the detection threshold. Threshold = Average noise power × T.

[0062] Target determination: If the signal strength of the CUT is greater than the threshold, the cell is determined to contain a target and output as a "point cloud". Otherwise, it is determined to be noise or clutter and discarded.

[0063] Output: The output of CFAR is a list of all detected target points, each typically represented in polar coordinates, i.e., orientation information, including: location distance (ρ), angle (θ), radial velocity (v), and radar cross section (RCS, reflecting the target size and material).

[0064] The conversion formula is based on trigonometric functions: X-coordinate (vertical distance): This coordinate represents the projected distance of the target in the radar's forward direction (or behind it). x represents the distance of the shuttle car behind the tunneling machine.

[0065] Y-coordinate (horizontal distance): This coordinate represents the lateral offset of the target relative to the radar's central axis. y represents how many meters the shuttle has shifted to the left or right relative to the tunneling machine's centerline.

[0066] The shuttle controller controls its own speed and direction based on the distance and speed information provided by the radar, maintaining a safe distance from the tunneling machine and following it stably.

[0067] Close-range precision following stage (<10 meters): The machine vision system is activated. The vision system identifies the markings at the rear of the tunneling machine, and the controller integrates radar and vision data, focusing on eliminating lateral positional deviations to ensure that the shuttle car travels precisely on the center path behind the tunneling machine, ready to receive the dropped material.

[0068] Vision system processing flow: Image acquisition and preprocessing: The RGB-D camera simultaneously captures color and depth images. Distortion correction, noise reduction, and contrast enhancement are performed on the images to improve recognition success rate.

[0069] Feature Detection and Recognition: On a color image, the YOLO algorithm, a deep learning-based object detection network, is used to quickly locate highly reflective signs. These signs use patterns with specific shapes. The algorithm outputs the pixel coordinates (u, v) of specific feature points (four corner points) on the sign in the image.

[0070] Depth information acquisition: Based on the pixel coordinates (u,v) obtained in step 2, retrieve the depth value d (i.e., the straight-line distance of the point from the camera) from the corresponding position in the depth image.

[0071] Coordinate transformation (key step): Using the camera's intrinsic parameter matrix (obtained through calibration), (u,v,d) is converted into the 3D coordinates of the feature points relative to the camera coordinate system, P_c = (X_c,Y_c,Z_c). Using the four corner points, the complete 6-DOF pose of the marker board can be calculated: that is, (x,y,z,roll,pitch,yaw) relative to the camera.

[0072] Coordinate transformation: Given the rigid mounting relationship between the camera and the shuttle car body (a fixed transformation matrix), transform the pose of the marker plate from the camera coordinate system to the car body coordinate system.

[0073] Final output: The most critical information obtained by the vision system is: Lateral deviation (e_y): The lateral distance between the center point of the tunnel boring machine's tail and the center line of the shuttle car. Heading deviation (e_θ): The angle between the orientation of the tunnel boring machine's tail and the orientation of the shuttle car (Yaw angle). Longitudinal distance (e_x): The straight-line distance between the two vehicles.

[0074] Magnetic components: Magnetic Marker: A permanent magnet installed at a predetermined stopping position at the tail of the tunneling machine. It establishes a stable and predictable static magnetic field spatial distribution.

[0075] Hall sensor array: A matrix sensor board (e.g., 16×16) mounted on the front of the shuttle car. Each Hall sensor can independently measure the magnetic field strength (unit: Tesla, T) at its location.

[0076] Control system: The PID controller (proportional-integral-derivative controller) in the host computer is responsible for processing sensor data and issuing commands.

[0077] Actuator: Hydraulic servo system, capable of receiving commands and driving the shuttle in very precise steps (2mm / time).

[0078] Magnetic attraction process, precise docking within 1m: Step 1: Magnetic Field Detection (Hall Array Detection of Magnetic Field Intensity Distribution) Process: When the shuttle enters the effective range of the magnet (approximately 0.8m), the entire Hall sensor array begins synchronous scanning, measuring the three-dimensional magnetic field intensity vector at each point on its surface. This yields a complete "magnetic field intensity distribution map." Each data point contains: (sensor coordinates (x_i, y_i), magnetic field intensity B_i).

[0079] Step 2: Position Calculation (Calculate relative position: Δx = Σ(B_i∙x_i) / ΣB_i) Process: Two algorithms are used to fuse the calculation to obtain a more accurate offset. Weighted Centroid Method: This algorithm uses magnetic field strength as weight to calculate the weighted average position of the magnet center relative to the origin of the sensor array. The results are stable and have good noise resistance. Magnetic Field Gradient Method: By analyzing the rate of change (gradient) of the magnetic field in the X and Y directions, the accurate direction and translation of the magnet center can be determined more accurately. The deviation (Δx, Δy) between the current position of the shuttle and the target stopping point is calculated.

[0080] Hydraulic fine-tuning (hydraulic system fine-tuning, step size 20mm / time) process: After receiving the deviation (Δx, Δy), the PID controller does not command a movement of 8.75cm all at once. Instead, it treats it as a target and gradually approaches it through closed-loop control.

[0081] Closed-loop verification and completion (effect verification) process: After a fine adjustment of 20mm, the system immediately returns to step 1, restarts the Hall array to scan the magnetic field, and recalculates the new offset.

[0082] Finally, the tunneling machine loads the material into the shuttle car's cargo box, which is the receiving hopper of the shuttle car.

[0083] Process 2: Autonomous docking and unloading of shuttle car and crusher ("transport-crushing" link) (same as process 1).

[0084] Once the shuttle is fully loaded, it receives instructions to move to the crusher at a fixed location.

[0085] The search phase: The shuttle car uses 4D millimeter-wave radar for global perception, and uses the CFAR detection algorithm to initially locate the approximate position of the crusher and plan a path to drive towards the docking area.

[0086] Mid-range docking (10m-1m), visual-radar data fusion, path optimization: visual information (identifying feature points of the crusher feed inlet) and radar (point cloud information) are fused, taking into account the kinematic constraints of the equipment (minimum turning radius 2.5m).

[0087] Precision alignment stage: After entering the magnetic signal's effective range (e.g., <1 meter), the magnetic attraction sensor begins to operate. Based on the magnetic field strength distribution (the direction with the strongest signal is the docking center), the controller guides the shuttle car to adjust its posture and slowly approach the crusher's feed inlet, i.e., the crusher's receiving hopper.

[0088] Final docking and unloading: When the magnetic signal strength reaches the predetermined threshold (indicating precise alignment), the shuttle car stops, the cargo box is lifted (i.e., the unloading hopper is lifted), and the material is unloaded into the crusher feed inlet. The vision system can assist in verifying the docking status.

[0089] Process 3: Autonomous return of the shuttle car after unloading ("return trip" stage).

[0090] After the shuttle car finishes unloading, it receives instructions to return to the location of the tunneling machine.

[0091] The return process repeats the perception and control logic of process one until it reaches the waiting position behind the tunneling machine again, completing one work cycle.

[0092] The above process adopts Path planning is performed, and then motion control is performed based on adaptive model predictive control (MPC).

[0093] In one exemplary embodiment, such as Figure 4 As shown, an autonomous following control device based on multi-source sensing is provided, comprising: The multi-source data acquisition module is used to acquire multi-source data; the multi-source data is collected based on a three-level perception system of vision-radar-magnetic attraction; the three-level perception system of vision-radar-magnetic attraction is constructed based on machine vision devices, 4D millimeter-wave radar and magnetic sensor array.

[0094] The preprocessing module is used to preprocess multi-source data to obtain preprocessed multi-source data.

[0095] The data processing and determination module is used to perform CFAR-based detection, visual processing, magnetic field detection, and position calculation based on preprocessed multi-source data to obtain processed data.

[0096] The collaborative control module is used to perform collaborative control of the three machines based on the processed data to complete one operation cycle of the complete set of equipment in the coal mine tunneling face. The operation cycle includes: autonomous following of the tunneling machine and shuttle car, autonomous docking and unloading of the shuttle car and crusher, and autonomous return of the shuttle car after unloading.

[0097] In one exemplary embodiment, a computer device is provided, which may be a server or a terminal, and its internal structure diagram may be as follows. Figure 5As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The database stores autonomous following control data based on multi-source sensing. The I / O interfaces are used for information exchange between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements the autonomous following control method based on multi-source sensing.

[0098] Those skilled in the art will understand that Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0099] In one exemplary embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0100] In one exemplary embodiment, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0101] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above-described method embodiments.

[0102] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0103] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0104] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0105] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0106] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. An autonomous following control method based on multi-source sensing, characterized in that, The autonomous following control method based on multi-source perception is applied to a complete set of equipment for coal mine tunneling faces. The complete set of equipment for coal mine tunneling faces includes: a tunneling machine, a shuttle car, and a transfer crusher. A machine vision device is arranged on the top of the complete set of equipment for coal mine tunneling faces, a 4D millimeter-wave radar is arranged in front of or behind the complete set of equipment for coal mine tunneling faces, and a magnetic sensor array is installed at the tail or head of the complete set of equipment for coal mine tunneling faces. The autonomous following control method based on multi-source sensing includes: Acquire multi-source data; the multi-source data is collected based on a three-level vision-radar-magnetic sensing system; the three-level vision-radar-magnetic sensing system is constructed based on machine vision devices, 4D millimeter-wave radar, and magnetic sensor arrays; The multi-source data is preprocessed to obtain preprocessed multi-source data; Based on the preprocessed multi-source data, detection, visual processing, magnetic field detection and position calculation based on the CFAR algorithm are performed to obtain processed data; Based on the processed data, the three machines are coordinated and controlled to complete one work cycle of the complete set of equipment at the coal mine tunneling face; wherein, the work cycle includes: autonomous following of the tunneling machine and the shuttle car, autonomous docking and unloading of the shuttle car and the crusher, and autonomous return of the shuttle car after unloading.

2. The autonomous following control method based on multi-source sensing according to claim 1, characterized in that, Based on the preprocessed multi-source data, detection, visual processing, magnetic field detection, and position calculation based on the CFAR algorithm are performed to obtain processed data, specifically including: Based on the 4D millimeter-wave radar detection data contained in the preprocessed multi-source data, the CFAR algorithm is used for global perception and positioning processing to obtain azimuth information and perform path planning. In addition, the conversion between polar coordinates and Cartesian coordinates is performed to determine the docking area information. Based on the image data corresponding to the machine vision device contained in the preprocessed multi-source data, visual processing is performed to determine the visual output, and the visual output is fused with 4D millimeter-wave radar detection data to optimize the path information obtained from path planning. The visual output includes: lateral deviation, heading deviation, and longitudinal distance. The lateral deviation is the lateral distance between the center point of the tunnel boring machine's tail and the center line of the shuttle car, or the lateral distance between the center point of the shuttle car's tail and the center line of the crusher. The heading deviation is the angle between the orientation of the tunnel boring machine's tail and the orientation of the shuttle car, or the angle between the orientation of the shuttle car's tail and the orientation of the crusher. The longitudinal distance is the straight-line distance between the tunnel boring machine and the shuttle car, or between the shuttle car and the crusher. Based on the three-dimensional magnetic field strength vector contained in the preprocessed multi-source data, magnetic field detection is performed to obtain magnetic field strength distribution information. Based on the magnetic field strength distribution information, the magnetic field gradient method is used to calculate the position using the magnetic field strength as a weight to obtain the position offset. The orientation information, the path planning information, the docking area information, the visual output, the optimized path information, and the position offset are determined as the processing data.

3. The autonomous following control method based on multi-source sensing according to claim 1, characterized in that, Based on the processed data, three-machine coordinated control is performed to complete one work cycle of the complete set of equipment at the coal mine tunneling face, specifically including: During the autonomous following phase of the tunneling machine and shuttle car: In the medium-to-long distance stage where the distance between the tunneling machine and the shuttle car is greater than 10 meters, the tunneling machine is controlled to carry out tunneling operations according to the preset path, and the shuttle car is controlled to control its own speed and direction based on the following instructions, according to the orientation information, the information obtained from the path planning, and the distance and speed information contained in the 4D millimeter-wave radar detection data, so that it maintains a set safe distance from the tunneling machine and carries out following operations. During the close-range following phase where the distance between the tunneling machine and the shuttle car is less than 10 meters and not less than 1 meter, based on the docking area information, the visual output, and the optimized path information, the shuttle car is controlled to travel on the center path behind the tunneling machine to carry out subsequent receiving and unloading operations. During the docking phase when the distance between the tunneling machine and the shuttle car is less than 1 meter, the shuttle car is controlled to adjust its posture according to the position offset. The magnetic signal strength detected in real time by the magnetic attraction sensor array is compared with a predetermined threshold. When the detected magnetic signal strength is greater than the predetermined threshold, the shuttle car is controlled to stop moving and lift the receiving hopper, and the tunneling machine is controlled to unload the material into the receiving hopper of the shuttle car. During the autonomous docking and unloading phase between the shuttle car and the crusher: After detecting that the shuttle is fully loaded, the control shuttle will drive to the crusher at a fixed location based on the corresponding orientation information, the information obtained from the path planning, and the docking area information. In the medium-to-long distance stage where the distance between the shuttle car and the crusher is greater than 10 meters, the shuttle car is controlled to move towards the crusher according to the preset path. Based on the corresponding following instructions, the shuttle car is controlled to control its own speed and direction according to the corresponding orientation information, the information obtained from the path planning, and the distance and speed information contained in the 4D millimeter-wave radar detection data, so that it gradually approaches the crusher. During the mid-distance docking phase, where the distance between the shuttle car and the crusher is less than 10 meters and not less than 1 meter, the shuttle car is controlled to move towards the crusher based on the visual output and the optimized path information to prepare for docking. During the precision alignment stage where the distance between the shuttle car and the crusher is less than 1 meter, the shuttle car is controlled to adjust its posture according to the corresponding position offset, so that the shuttle car moves towards the receiving hopper of the crusher. The magnetic signal strength detected in real time by the magnetic attraction sensor array is compared with a predetermined threshold. When the detected magnetic signal strength is greater than the predetermined threshold, the shuttle car is controlled to stop moving and lift the unloading hopper so that the material is unloaded into the receiving hopper of the crusher. During the autonomous return phase of the shuttle car after unloading: Once the shuttle car has finished unloading, the shuttle car is controlled to return to the location of the tunneling machine based on the return control command, the corresponding orientation information and the information obtained from the path planning, until the distance between the shuttle car and the tunneling machine reaches the conditions corresponding to the autonomous following stage of the tunneling machine and the shuttle car, so as to start a new work cycle.

4. The autonomous following control method based on multi-source sensing according to claim 2, characterized in that, Based on the image data corresponding to the machine vision device contained in the preprocessed multi-source data, visual processing is performed to determine the visual output, specifically including: The image data is subjected to distortion correction, noise reduction, and contrast enhancement to obtain processed image data; the image data includes: a color image and a depth image; For the corresponding color image in the processed image data, the YOLO algorithm, a deep learning-based object detection network, is used to identify feature points and obtain the corresponding pixel coordinates. Based on the pixel coordinates, the depth value is extracted from the corresponding position of the depth image in the processed image data; the depth value is the straight-line distance between the corresponding position point and the RGB-D camera included in the machine vision device. Based on the intrinsic parameter matrix of the RGB-D camera, the dataset composed of pixel coordinates and depth values ​​is converted into three-dimensional coordinates of feature points relative to the camera coordinate system in order to solve the visual pose. Based on the rigid mounting relationship between the RGB-D camera and the shuttle car body, the transformation matrix is ​​determined, and the visual pose is transformed from the camera coordinate system to the shuttle car body coordinate system to obtain the visual output.

5. The autonomous following control method based on multi-source sensing according to claim 2, characterized in that, Based on the 4D millimeter-wave radar detection data contained in the preprocessed multi-source data, the CFAR algorithm is used for global perception and positioning processing to obtain azimuth information and perform path planning. Furthermore, a conversion between polar coordinates and Cartesian coordinates is performed to determine the docking area information, specifically including: A three-dimensional data cube is determined; the three-dimensional data cube is determined based on the 4D millimeter-wave radar detection data contained in the preprocessed multi-source data after performing range FFT, Doppler FFT and angle FFT processing respectively; A sliding window is constructed using the CFAR algorithm and traversed through the three-dimensional data cube to determine the orientation information; the orientation information includes: positioning distance, angle, radial velocity, and radar cross section; Based on the location information, path planning and conversion between polar and Cartesian coordinates are performed to obtain docking area information.

6. The autonomous following control method based on multi-source sensing according to claim 1, characterized in that, The multi-source data is preprocessed to obtain preprocessed multi-source data, specifically including: The multi-source data is normalized to obtain normalized multi-source data; The normalized multi-source data is identified as the preprocessed multi-source data.

7. An autonomous following control device based on multi-source sensing, characterized in that, include: Multi-source data acquisition module, used to acquire multi-source data; The multi-source data is obtained based on a three-level perception system of vision, radar, and magnetic attraction. The vision-radar-magnetic three-level perception system is built on machine vision devices, 4D millimeter-wave radar, and magnetic sensor arrays; The preprocessing module is used to preprocess the multi-source data to obtain preprocessed multi-source data; The data processing and determination module is used to perform CFAR-based detection, visual processing, magnetic field detection, and position calculation based on preprocessed multi-source data to obtain processed data. The collaborative control module is used to perform three-machine collaborative control based on the processed data to complete one work cycle of the complete set of equipment at the coal mine tunneling face; wherein, the work cycle includes: autonomous following of the tunneling machine and the shuttle car, autonomous docking and unloading of the shuttle car and the crusher, and autonomous return of the shuttle car after unloading.

8. A computer device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the autonomous following control method based on multi-source perception as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the autonomous following control method based on multi-source perception as described in any one of claims 1-6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the autonomous following control method based on multi-source perception as described in any one of claims 1-6.

Images