Trackless equipment safety early warning method and system based on multi-source information fusion

Abstract

The present invention discloses a safety warning method and system for trackless equipment based on multi-source information fusion. The warning method includes the steps of real-time monitoring of track intersections, environmental safety monitoring, warning level determination, collision determination, and active braking intervention. The warning system includes a central control system running on a server, an on-board host computer installed on each vehicle, an on-board display screen, an on-board positioning device, an on-board alarm device, and the safety warning identification device, as well as a three-dimensional perception system worn by staff. The present invention realizes real-time interactive warning for vehicles and personnel, and can also ensure the accuracy and effectiveness of the alarm, eliminate warning blind spots, improve the timeliness and stability of identification, and provide protection for safe production in mines.

Description

Technical Field

[0001] The present invention belongs to the field of trackless equipment in mines, and in particular relates to an early warning method for the operation of trackless equipment, and also relates to an early warning method. Background Art

[0002] Mechanized mining is a growing trend in my country's underground mining industry. Trackless mining equipment, due to its flexibility and adaptability to complex environments, is widely used in underground metal and non-metal mines, and its use is growing in popularity. However, due to the limited size of underground tunnels and the need to protect against collisions and falling rocks, trackless mining equipment generally suffers from low cab positioning, limited space, and long vehicles. This results in large blind spots, significant safety hazards during operation, and a high risk of accidents. Accidents involving trackless mining equipment have become a significant risk in mines.

[0003] To reduce and eliminate vehicle injuries caused by blind spots in trackless equipment, existing technologies primarily utilize vehicle collision warning systems based on radar or sensors. When identification devices installed around the vehicle detect an obstacle too close, they sound an alarm, alerting the driver and other personnel. However, this type of monitoring and alarm still has drawbacks and potential risks for vehicles traveling in underground tunnels:

[0004] First, due to the height and width of the tunnel, and the influence of people and vehicles on each other, the alarm state will always appear even if there are other obstacles around the vehicle, thus losing its due warning significance.

[0005] Secondly, the existing early warning system is not combined with human behavior recognition, especially it cannot recognize human behavior in blind spots, resulting in early warning blind spots.

[0006] Thirdly, it is impossible to combine human behavior with the status of trackless equipment to make intelligent early warning and control linkage, the early warning effect is poor, and safety cannot be truly improved.

[0007] Fourth, the conditions underground are harsh, the situation is complex, and there are many people. The identification devices on the vehicles cannot sense the location of people in advance, and must always maintain a high-intensity monitoring state. The amount of calculation is large, and the time lag is strong. Dangerous situations cannot be discovered in time and measures cannot be taken immediately. It may even be unable to operate normally due to problems such as high heat generation.

[0008] Fifth, the existing alarm mechanism is a unilateral alarm by the vehicle. The underground environment is difficult to see and noisy. Workers in dangerous situations often cannot receive the alarm signal in time, or cannot immediately find the dangerous vehicle based on the alarm. Summary of the Invention

[0009] The present invention proposes a trackless equipment safety warning method and system based on multi-source information fusion, the purposes of which are: (1) to achieve real-time interactive warning for vehicles and personnel, ensuring that vehicles and personnel can send and receive effective alarm signals; (2) to provide vehicles with blind spot recognition capabilities and eliminate warning blind spots; (3) to reduce the computational complexity of the recognition device and improve the timeliness and stability of recognition; (4) to achieve the linkage between warning and vehicle control and improve safety.

[0010] The technical solutions of the present invention are as follows:

[0011] A trackless equipment safety early warning method based on multi-source information fusion, the steps are as follows:

[0012] Step 1: The vehicle and operator report their positions to the central control system in real time. The central control system determines in real time whether there is an intersection between the vehicle and the operator's trajectory based on their positions.

[0013] Step 2: If there is an intersection between the vehicle and the operator, activate the safety warning identification device installed on the vehicle and the stereoscopic perception system on the operator, and then continue to step 3; if there is no intersection, return to step 1 to continue monitoring;

[0014] Step 3: The environmental safety monitoring system determines whether there is a possibility of collision between the operator and the vehicle and the distance between the two through the safety warning identification device, and obtains a monitoring result;

[0015] Step 4: The central control system determines the warning level based on the results of the environmental safety monitoring system. If the warning level is low, the alarm device on the vehicle and the three-dimensional perception system on the operator will both issue a low-frequency alarm, and then the process will return to step 3. If the warning level is high, the process will continue to step 5.

[0016] Step 5: Predict whether the vehicle will still collide with the operator even if the driver brakes after receiving the alarm. If the prediction result shows that there will be no collision, the alarm device on the vehicle and the stereoscopic perception system on the operator will both issue a high-frequency alarm, and then return to step 3 to re-perform environmental safety monitoring and determine the warning level. If the prediction result shows that there will be a collision, the central control system will send an active braking signal to the vehicle's control system to control the vehicle to actively brake until the danger is eliminated, then unlock the vehicle and return to step 1.

[0017] As a further improvement to the trackless equipment safety warning method based on multi-source information fusion: in step 1, it is determined whether there is a trajectory intersection between the vehicle and the operator based on the electronic fence; the electronic fence refers to a virtual interference range set by the central control system around the real-time position of the vehicle and / or the operator. When judging, if there is an intersection between the electronic fences of the vehicle and the operator, or if there is an intersection between the electronic fences of the vehicle and the operator, or if there is an intersection with the position of the other party, it is considered that the two have a trajectory intersection.

[0018] As a further improvement of the trackless equipment safety warning method based on multi-source information fusion: in step 2, when the central control system finds that the trajectories of the vehicle and the operator intersect, the clean protection device on the safety warning identification device is turned off before starting the safety warning identification device; when the central control system finds that the trajectories of the vehicle and the operator do not intersect, it detects whether the clean protection device on the safety warning identification device has been started, and if not, it starts. After determining that the clean protection device has been started, it returns to step 1.

[0019] As a further improvement of the trackless equipment safety early warning method based on multi-source information fusion, the specific steps of step 3 are:

[0020] Step 3.1: The environmental safety detection system activates the safety warning identification device at a high frame rate;

[0021] Step 3.2: The safety warning identification device evaluates the noise of the acquired environmental information. If the noise is too high, the environmental information noise reduction algorithm is activated before executing step 3.3. Otherwise, step 3.3 is executed directly.

[0022] Step 3.3: The safety warning identification device monitors surrounding obstacles in real time;

[0023] Step 3.4: The safety warning identification device determines whether to switch the frame rate based on the currently acquired data. If switching is required, the process jumps to step 3.3 after the switching. Otherwise, the process executes step 3.5.

[0024] Step 3.5: The safety warning identification device obtains the distance between the monitored obstacle target and the vehicle and provides a monitoring result;

[0025] Step 3.6: Wait for a while. If a shutdown request is received, shut down the environmental safety monitoring system. Otherwise, jump to step 3.3 and continue monitoring.

[0026] As a further improvement of the trackless equipment safety warning method based on multi-source information fusion: in the step 3.5, first, based on the current speed, load and steering wheel position of the vehicle, the space that the vehicle needs to pass through to continue driving in the current state, i.e., the monitoring space, is calculated; then, the camera in the safety warning identification device captures and identifies in real time whether there is an operator entering the monitoring space. If an operator enters, the position and movement speed of the operator relative to the vehicle are obtained through the image captured by the camera, and then the radar scanning result is further obtained, and the distance between the obstacle corresponding to the position in the scanning result and the vehicle is used as the distance between the operator and the vehicle.

[0027] As a further improvement of the trackless equipment safety warning method based on multi-source information fusion: after the image of the operator entering the monitoring space is obtained through the camera, the image is displayed on the vehicle-mounted display screen, and different sound and light alarm states are set according to the distance between the operator and the vehicle. At the same time, warning information of different colors is provided around the operator on the vehicle-mounted display screen.

[0028] As a further improvement of the trackless equipment safety warning method based on multi-source information fusion: in step 4, if the vehicle or operator needs to send out multiple alarm signals of different levels at the same time, the highest level is used for alarm.

[0029] As a further improvement of the trackless equipment safety warning method based on multi-source information fusion: in step 5, if the prediction result indicates that a collision will occur, after the central control system sends an active braking signal to the vehicle's control system, the current braking status of the vehicle is first monitored. If the driver has braked or temporarily canceled the active braking function, the process jumps back to step 3; otherwise, the vehicle is controlled to actively brake.

[0030] A trackless equipment safety warning system based on the above-mentioned warning method comprises the central control system, and also comprises an on-board host, an on-board display screen, an on-board positioning device, an on-board alarm device and the safety warning identification device installed on each vehicle;

[0031] The vehicle-mounted host is communicatively connected to the central control system, and the vehicle-mounted display screen, vehicle-mounted positioning device, vehicle-mounted alarm device and safety warning identification device are respectively communicatively connected to the vehicle-mounted host;

[0032] The early warning system also includes a three-dimensional perception system installed on the operator and connected to the central control system; the three-dimensional perception system includes a human body positioning device;

[0033] The vehicle-mounted alarm device and the stereoscopic perception system both have an audible and visual alarm module and a vibration alarm module.

[0034] As a further improvement of the trackless equipment safety warning system: the safety warning identification device includes four sets of cameras and ultrasonic radars installed at the front left, front right, rear left and rear right of the vehicle;

[0035] The vehicle-mounted alarm device includes an in-vehicle alarm and an out-vehicle alarm.

[0036] Compared with the prior art, the present invention has the following beneficial effects: (1) When there is a possibility of collision, different levels of alarm signals including but not limited to vibration, sound and light are sent to the vehicle and the operator simultaneously, so that the driver and the operator can immediately realize the danger and the degree of danger at the same time, and take corresponding measures quickly, and finally realize the real-time interactive early warning of the vehicle and the personnel, and ensure the effectiveness of the alarm; (2) On the basis of the electronic fence, the safety early warning identification device is further used to judge whether the operator has the possibility of collision with the vehicle and the distance, thereby providing the vehicle with a full range of safety identification and early warning capabilities, eliminating early warning blind spots, and improving the accuracy of early warning; (3) Only in the central control system The safety warning identification device and the three-dimensional perception system are activated only after the electronic fence detects the intersection between the vehicle and the operator, which reduces the computational complexity of the vehicle-mounted safety warning identification device and improves the timeliness and stability of identification; (4) After issuing an advanced collision warning, it can also decide whether to intervene in active braking based on the actual situation, realize the linkage between warning and vehicle control, and improve safety; (5) The safety warning identification device can be automatically cleaned during the time when there is no collision risk, reducing the impact of pollution such as underground dust on the identification device; (6) The environmental safety monitoring system automatically starts the noise reduction algorithm and switches the frame rate according to the surrounding environment and monitoring conditions, improves the accuracy of monitoring, and reduces unnecessary workload. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 is a flow chart of the early warning method of the present invention;

[0038] Figure 2 Flowchart of the environment for environmental safety testing;

[0039] Figure 3 Schematic diagram of the structure of the early warning system of the present invention. DETAILED DESCRIPTION

[0040] The technical solution of the present invention is described in detail below with reference to the accompanying drawings:

[0041] like Figure 3 This embodiment provides a control system, which includes a central control system 1 running on a server, and also includes an on-board host 2, an on-board display screen 3, an on-board positioning device, an on-board alarm device and the safety warning identification device installed on each vehicle.

[0042] The central control system 1 can remotely manage various types of vehicle-mounted terminals, implement remote upgrades, remote troubleshooting and other functions, and can retrieve or view the remote working status of equipment in real time. The platform system supports comprehensive docking and uploading of various types of data.

[0043] The Vehicle Host 2 integrates intelligent four-way blind spot algorithm warning functions, audio and video monitoring, and other features. It can receive and process GPS or high-precision positioning data and CAN bus information, and features audio and video recording, driving information recording, wireless data upload, and on-site printing, demonstrating its strong information processing capabilities. Based on deep learning algorithms, the Vehicle Host 2 accurately detects various alarms and provides timely video and voice alerts, alerting drivers to potential dangers and effectively improving driving safety.

[0044] The vehicle host 2 is wirelessly connected to the central control system 1. The vehicle display 3, vehicle positioning device, vehicle alarm device, and safety warning identification device are each connected to the vehicle host 2. The vehicle positioning device uses GPS, Beidou, or other methods to obtain the vehicle's location and report it in real time.

[0045] The safety warning identification device includes four groups of fisheye cameras 6 (which can be monocular, binocular cameras, or with infrared function) and ultrasonic radars 7 (which can be laser radars or millimeter wave radars) installed on the left front, right front, left rear and right rear of the vehicle, which facilitate the identification of blind spots in different directions and achieve "full coverage".

[0046] The vehicle-mounted alarm device includes an in-vehicle alarm 4 and an out-vehicle alarm 5. The in-vehicle alarm 4 mainly provides an alarm for the driver, and the out-vehicle alarm 5 provides an alarm for the surrounding workers.

[0047] The early warning system also includes a 3D sensing system worn by the operator and connected to the central control system 1. The 3D sensing system can be in the form of a helmet or a safety back clip. It contains a human positioning device that uses GPS, Beidou, or other methods to obtain the operator's location and report it in real time.

[0048] All vehicle alarm devices and stereo perception systems have sound and light alarm modules and vibration alarm modules.

[0049] like Figure 1 ,The working process of the security warning of the above early warning system is as follows:

[0050] The central control system 1 first imports the daily operation tasks, then starts the on-board system of the trackless equipment (including the above-mentioned on-board host 2, on-board display screen 3, on-board positioning device) and the three-dimensional perception system of the operator, and then sends the electronic fence data package and the trackless vehicle operation trajectory data package, and then performs the following steps:

[0051] Step 1: The vehicle and the operator report their positions to the central control system 1 in real time. The central control system 1 determines in real time whether there is any intersection between the vehicle and the operator's trajectory based on their positions.

[0052] Furthermore, in step 1, a determination is made based on the electronic fence whether there is an intersection between the vehicle and the operator's trajectories. The electronic fence refers to a virtual interference range set by the central control system 1 around the real-time location of the vehicle and / or the operator. The size of this range is pre-set based on factors such as the size of the operator and vehicle. If there is an intersection between the electronic fences of the vehicle and the operator, or if there is an intersection between the electronic fences of the vehicle and the operator, then the trajectories are considered to intersect.

[0053] Step 2: If there is an intersection between a vehicle and a worker, activate the safety warning identification device installed on the vehicle and the three-dimensional perception system on the worker to prepare for the warning, and then proceed to Step 3. If there is no intersection, return to Step 1 to continue monitoring.

[0054] Furthermore, if the central control system 1 detects an intersection between the trajectories of a vehicle and another operator, it disables the cleaning device on the safety warning recognition device before activating the safety warning recognition device to prevent the cleaning process from interfering with the warning. If the central control system 1 detects no intersection between the trajectories of a vehicle and another operator, it checks whether the cleaning device on the safety warning recognition device is activated. If not, it activates it. Once it is confirmed that the cleaning device is activated, it returns to step 1. This method allows for optimal cleaning of the recognition device, ensuring that it always operates in optimal condition.

[0055] The cleaning protection device includes a reciprocating brush, a cleaning liquid supply module and other devices. Its structural principle is the same as that of the vehicle glass cleaning device, which will not be described in detail here.

[0056] However, the existence of an intersection only means that the distance is close, and does not mean that a collision is likely to occur. Therefore, the following steps are required for further identification.

[0057] Step 3: The environmental safety monitoring system uses the safety warning identification device to determine whether the operator has the possibility of colliding with the vehicle and the distance between them, and obtains the monitoring result. The environmental safety monitoring system can be run on the vehicle host 2 or in the central control system 1 as needed.

[0058] Specific as Figure 2 As shown:

[0059] Step 3.1: The environmental safety detection system activates the safety warning identification device at a high frame rate.

[0060] Step 3.2: The safety warning identification device evaluates the noise of the acquired environmental information. If the noise is too high, the environmental information noise reduction algorithm is activated before executing step 3.3. Otherwise, step 3.3 is executed directly.

[0061] As an optional noise assessment and noise reduction method:

[0062] First, establish the imaging model of the image in fog noise:

[0063] I λ (x) = J λ (x)t(x)+A λ (1-t(x))

[0064] Where I is the fog image received by the camera, J is the original scene signal, t represents the transmittance of the environment, and A represents the light. Lambda indicates that these terms are related to wavelength and, in the case of images, to the three RGB channels.

[0065] The model is improved, the illumination effect and the transmittance map are integrated, and a unified parameter K is used to represent it. The K value can be used to reconstruct a clear image from the observed fog image:

[0066]

[0067] J λ (x) = K λ (x)I λ (x)-K λ (x)+1.

[0068] The neural network model is used to learn features at different levels, restore the Kmap in the scene, and obtain a clear image of the scene based on the above formula to complete the image denoising process.

[0069] Step 3.3: The safety warning identification device monitors surrounding obstacles in real time by photographing, scanning, etc.

[0070] In step 3.4, the safety warning recognition device determines whether to switch the frame rate based on the currently acquired data. If the captured image is not clear enough to detect human body, it will need to switch to a high frame rate. If the captured image is too clear, it can switch back to a low frame rate. After switching, jump to step 3.3 and retake the image. Otherwise, do not switch and proceed to step 3.5.

[0071] Step 3.5: The safety warning identification device obtains the distance between the monitored obstacle target and the vehicle, and gives the monitoring result through fusion decision-making.

[0072] Specifically, the system first calculates the space the vehicle needs to pass through to continue driving in its current state, based on the vehicle's current speed, load, and steering wheel position. This space, known as the monitoring space, includes both areas directly observable by the driver and blind spots that are difficult to observe. The camera in the safety warning recognition device then captures and identifies in real time whether a worker has entered this monitoring space. If a worker enters, the camera captures the worker's position and speed relative to the vehicle, using the image captured by the camera. The radar scan results are then used to determine the distance between the obstacle corresponding to that position and the vehicle, representing the distance between the worker and the vehicle.

[0073] Furthermore, the safety warning identification device can detect and identify personnel in both static and dynamic conditions, providing monitoring of changes in the relative displacement between all personnel and the trackless equipment within and outside the blind spot. Based on these changes in relative displacement, different alarm priorities can be set, laying the foundation for adopting different alarm strategies, implementing proactive reminders, and implementing classified warnings in subsequent steps. The vehicle-mounted host 2 can access the vehicle body's CAN signal and determine the movement trend by judging the gear position, turn signal, vehicle speed, and steering wheel angle signals, combined with the relative distance between the person and the vehicle, relative movement speed, and movement direction. This lays the foundation for subsequently determining the degree of danger and issuing corresponding warning signals according to predetermined rules.

[0074] Furthermore, after the image of the operator entering the monitoring space is obtained through the camera, the image can be displayed on the vehicle-mounted display screen 3, and the driver can be reminded to pay attention to the safety of relevant personnel through the image, and different sound and light alarm states can be set according to the distance between the operator and the vehicle. At the same time, different colors of warning information are provided around the operator on the vehicle-mounted display screen 3. For example, green data indicates that the person is outside the warning distance, blue data indicates that the person is within the first-level warning distance range, yellow data indicates that the person is within the second-level warning distance range, orange data indicates that the person is within the third-level warning distance range, and red data indicates that the person is within the fourth-level warning distance range. The risk level, category and location are intuitively fed back, and the driver is informed concisely and clearly in a classified and graded manner.

[0075] The distance can also be pre-identified based on images and integrated with radar detection results. For example, for an average person's height, an identification frame of equal size can be set up on the camera 6 screen. By displaying the body parts and speed of people within different warning distances on the camera 6, the person's position and distance from the trackless equipment can be analyzed. The ultrasonic ranging radar then uses the distance data fed back to accurately locate nearby people. Similarly, distance thresholds need to be set in the data processing interface, with different levels of alarms triggered when different thresholds are triggered.

[0076] In this step, as one of the optional decision-making methods, we first define the environmental information weight function:

[0077]

[0078] Where F(k) is the camera information weight function, r(k i ) is the infrared camera information weighting function, G(m) is the radar information weighting function, and J(n) is the positioning information weighting function. i represents the sensor signal transmitted by the i-th device. The environmental information comprehensive weighting function can be used to calculate the collision warning level between the vehicle and the operator, and between vehicles, as well as the coordinate parameters of the safe distance range. These are then transmitted to the central control system 1, which then comprehensively determines and issues safety warnings of different levels.

[0079] It should be noted that the above decision-making method is not the only one. In actual operation, the decision result can also be directly obtained by combining the position and distance of the operator with pre-set rules.

[0080] Step 3.6: Wait for a while. If a shutdown request is received, shut down the environmental safety monitoring system. Otherwise, jump to step 3.3 and continue monitoring.

[0081] Step 4: Figure 1 The central control system 1 determines the warning level based on the results of the environmental safety monitoring system. If the warning level is low, the alarm device on the vehicle and the stereoscopic perception system on the operator will both issue a low-frequency alarm, and then jump back to step 3; if the warning level is high, continue to step 5.

[0082] Step 5: Predict whether the vehicle will still collide with the worker even if the driver brakes after receiving the alarm. If the prediction indicates a collision is unlikely, both the vehicle's alarm device and the worker's stereoscopic perception system will issue a high-frequency alarm, prompting the worker to evade and the driver to brake. The system then returns to Step 3, re-monitoring the environmental safety and determining the warning level. If the prediction indicates a collision, the central control system 1 sends an active braking signal to the vehicle's control system, then monitors the vehicle's current braking status. If the driver has already braked or temporarily disabled the active braking function, the system returns to Step 3. Otherwise, the vehicle will be actively braked until the danger is resolved, at which point the vehicle will be unlocked and the system returns to Step 1.

[0083] Furthermore, if a vehicle or operator needs to issue multiple alarm signals of different levels simultaneously, the highest level will be used. Furthermore, after the operator senses the alarm from the 3D sensing system, they can observe the surrounding environment to see which vehicle's alarm frequency matches their own, and thus determine the vehicle they should avoid.

[0084] It should be noted that, due to the short judgment time in step 5, the audible and visual alarms, the vibration alarm, and the image alarm on the vehicle display screen 3 appear essentially simultaneously. As part of the coordinated alarm, the high-frequency and low-frequency alarms can be further refined during the alarm process. Depending on the distance value, the alarm can be amplified, the speaker volume can be changed, the buzzer prompt can be increased, the speed of the voice prompt can be changed, and other methods can be used to quickly let people around the vehicle know the degree of danger. For example: when a person is between 5 meters and 7 meters away from the front or rear end of the vehicle and the relative distance is decreasing, a first-level warning is issued, and the corresponding image of the front or rear blind spot is popped up on the high-definition screen in the cab; when a person is between 3 meters and 5 meters away from the front or rear end of the vehicle and the relative distance is decreasing, a second-level warning is issued, and the corresponding image of the front or rear blind spot is popped up on the high-definition screen in the cab, and the horn sounds, and the sound and light alarm sounds and flashes; when a person is between 1 meter and 3 meters away from the front or rear end of the vehicle and the relative distance is decreasing, a third-level warning is issued, and the corresponding image of the front or rear blind spot is popped up on the high-definition screen in the cab, and the horn sounds, and the sound and light alarm sounds and flashes; when a person is less than 1 meter away from the front or rear end of the vehicle and the relative distance is decreasing, a fourth-level warning is issued, and the corresponding image of the front or rear blind spot is popped up on the high-definition screen in the cab, and the horn sounds, and the sound and light alarm sounds and flashes, and the engine directly performs emergency braking to avoid vehicle injury. At the front and rear of the vehicle, the first-level alarm area is set to a distance of 7 meters from the vehicle body. When a person is more than 7 meters away from the front or rear end of the vehicle body, the first-level warning is lifted and no warning reminder is given to the driver. When the relative distance is between 5 and 7 meters and the relative distance is increasing, the first-level warning is also lifted and no warning reminder is given to the driver to reduce invalid alarms. The alarm classification of people and vehicles in the left and right blind spots is consistent with the front and rear end division method described above, and the division distance is smaller. The specific settings are as follows: When a person is between 2 and 1 meter away from the left or right side of the vehicle body and the relative distance is decreasing, a first-level warning is issued, and the corresponding image of the left or right blind spot pops up on the high-definition screen in the cab; when a person is between 1 and 0.6 meters away from the left or right side of the vehicle body and the relative distance is decreasing, a second-level warning is issued, and the corresponding image of the left or right blind spot pops up on the high-definition screen in the cab, and the horn sounds, and the sound and light alarm sounds to issue a prompt. When a person is between 0.3m and 0.6m away from the front or rear end of the vehicle and the relative distance is getting smaller, a level 3 warning is issued, and the high-definition screen in the cab pops up the image of the corresponding left or right blind spot, and the horn sounds, and the sound and light alarm emits a prompt sound and flashes; when a person is less than 0.3m away from the front or rear end of the vehicle and the relative distance is getting smaller, a level 4 warning is issued, and the high-definition screen in the cab pops up the image of the corresponding left or right blind spot, and the horn sounds, and the sound and light alarm emits a prompt sound and flashes, and the engine directly takes emergency braking to avoid injury from the vehicle.

[0085] Photos and videos of risk point warnings and related warning information can be stored locally or uploaded to the central control system 1 for storage. After automatic or manual data processing, the blind spot range or warning distance can be increased or decreased. For different scenarios such as reversing, straight driving, turning, etc., the alarm can be further refined to reduce false alarms without danger and improve the effectiveness of the alarm.

Claims

1. A trackless equipment safety early warning method based on multi-source information fusion, characterized in that The steps are: Step 1: The vehicle and the operator report their positions to the central control system (1) in real time; the central control system (1) determines in real time whether there is an intersection between the trajectories of the vehicle and the operator based on their positions; Step 2: If there is an intersection between the vehicle and the operator, activate the safety warning identification device installed on the vehicle and the stereoscopic perception system on the operator, and then continue to step 3; if there is no intersection, return to step 1 to continue monitoring; Step 3: The environmental safety monitoring system determines whether there is a possibility of collision between the operator and the vehicle and the distance between the two through the safety warning identification device, and obtains a monitoring result; Step 4: The central control system (1) determines the warning level based on the results of the environmental safety monitoring system. If the warning level is low, the alarm device on the vehicle and the stereoscopic perception system on the operator both issue a low-frequency alarm, and then jump back to step 3; if the warning level is high, continue to step 5; Step 5: Predict whether the vehicle will still collide with the operator even if the driver brakes after receiving the alarm. If the prediction result shows that there will be no collision, the alarm device on the vehicle and the stereoscopic perception system on the operator will both issue a high-frequency alarm, and then return to step 3 to re-perform environmental safety monitoring and determine the warning level. If the prediction result shows that there will be a collision, the central control system (1) sends an active braking signal to the vehicle control system to control the vehicle to actively brake until the danger is eliminated, then unlock the vehicle and return to step 1.

2. The trackless equipment safety early warning method based on multi-source information fusion according to claim 1, characterized in that: In step 1, it is determined whether there is a trajectory intersection between the vehicle and the operator based on the electronic fence; the electronic fence refers to a virtual interference range set by the central control system (1) around the real-time position of the vehicle and / or the operator. When judging, if there is a trajectory intersection between the electronic fence of the vehicle and the operator, or there is a trajectory intersection between the electronic fence and the position of the other party, then it is considered that there is a trajectory intersection between the two.

3. The trackless equipment safety early warning method based on multi-source information fusion according to claim 1 is characterized in that: In step 2, when the central control system (1) finds that the trajectories of the vehicle and the operator intersect, the clean protection device on the safety warning identification device is turned off before the safety warning identification device is activated; when the central control system (1) finds that the trajectories of the vehicle and the operator do not intersect, it detects whether the clean protection device on the safety warning identification device has been activated, and if not, it activates it. After determining that the clean protection device has been activated, it returns to step 1.

4. The trackless equipment safety early warning method based on multi-source information fusion according to claim 1 is characterized in that The specific steps of step 3 are: Step 3.1: The environmental safety detection system activates the safety warning identification device at a high frame rate; Step 3.2: The safety warning identification device evaluates the noise of the acquired environmental information. If the noise is too high, the environmental information noise reduction algorithm is activated before executing step 3.

3. Otherwise, step 3.3 is executed directly. Step 3.3: The safety warning identification device monitors surrounding obstacles in real time; Step 3.4: The safety warning identification device determines whether to switch the frame rate based on the currently acquired data. If switching is required, the process jumps to step 3.3 after the switching. Otherwise, the process executes step 3.

5. Step 3.5: The safety warning identification device obtains the distance between the monitored obstacle target and the vehicle and provides a monitoring result; Step 3.6: Wait for a while. If a shutdown request is received, shut down the environmental safety monitoring system. Otherwise, jump to step 3.3 and continue monitoring.

5. The trackless equipment safety early warning method based on multi-source information fusion according to claim 4 is characterized in that: In step 3.5, the space that the vehicle needs to pass through to continue driving in the current state, i.e., the monitoring space, is first calculated based on the vehicle's current speed, load, and steering wheel position. Then, the camera in the safety warning identification device captures and identifies in real time whether there are any workers entering the monitoring space. If there are workers entering, the position and movement speed of the workers relative to the vehicle are obtained through the image captured by the camera, and then the radar scanning results are further obtained. The distance between the obstacle corresponding to the position in the scanning results and the vehicle is used as the distance between the worker and the vehicle.

6. The trackless equipment safety early warning method based on multi-source information fusion according to claim 5, characterized in that: After the image of the operator entering the monitoring space is acquired through the camera, the image is displayed on the vehicle-mounted display screen (3), and different sound and light alarm states are set according to the distance between the operator and the vehicle. At the same time, warning information of different colors is provided around the operator on the vehicle-mounted display screen (3).

7. The trackless equipment safety early warning method based on multi-source information fusion according to claim 1, characterized in that: In step 4, if the vehicle or operator needs to send out multiple alarm signals of different levels at the same time, the highest level will be used for alarm.

8. The trackless equipment safety early warning method based on multi-source information fusion according to claim 1, characterized in that: In step 5, if the prediction result indicates that a collision will occur, after the central control system (1) sends an active braking signal to the vehicle's control system, the current braking status of the vehicle is monitored first. If the driver has already braked or temporarily canceled the active braking function, the process jumps back to step 3. Otherwise, the vehicle is controlled to brake actively.

9. A trackless equipment safety warning system based on the warning method according to claim 1, characterized in that: It includes the central control system (1), and also includes an on-board host (2), an on-board display screen (3), an on-board positioning device, an on-board alarm device and the safety warning identification device installed on each vehicle; The vehicle-mounted host (2) is communicatively connected to the central control system (1), and the vehicle-mounted display screen (3), the vehicle-mounted positioning device, the vehicle-mounted alarm device, and the safety warning identification device are respectively communicatively connected to the vehicle-mounted host (2); The early warning system further comprises a three-dimensional perception system installed on the operator and connected to the central control system (1); the three-dimensional perception system comprises a human body positioning device; The vehicle-mounted alarm device and the stereoscopic perception system both have an audible and visual alarm module and a vibration alarm module.

10. The trackless equipment safety warning system according to claim 9, characterized in that: The safety warning identification device comprises four sets of cameras (6) and ultrasonic radars (7) installed at the left front, right front, left rear and right rear of the vehicle; The vehicle-mounted alarm device comprises an in-vehicle alarm (4) and an out-vehicle alarm (5).

Images