Reversing obstacle avoidance method, reversing obstacle avoidance system and medium for remote driving of loading machine

Abstract

The embodiment of the invention relates to a reversing obstacle avoidance method and system for remote driving of a loader and a medium, and belongs to the technical field of intelligent remote driving of loaders, and the obstacle avoidance method comprises the steps that feature extraction is conducted on ultrasonic radar data and millimeter wave radar data, and feature information is obtained; inputting the feature information into a deep neural network to obtain an obstacle type; calculating to obtain the moving path and speed of the obstacle; predicting a first predicted position of the obstacle after the first time interval; on the basis of the current position of the obstacle and the current position of the loader, the transverse distance and the longitudinal distance between the obstacle and the loader in the two-dimensional direction are obtained; the corresponding braking distance is obtained based on the speed of the loader and the speed of the obstacle, and the braking signal is output under the condition that the transverse distance is smaller than the first threshold value and the longitudinal distance is smaller than or equal to the braking distance, so that the control accuracy of the reversing obstacle avoidance process can be improved, sudden braking is avoided, and the use feeling and safety are improved.

Description

Technical Field

[0001] The present application relates to the technical field of intelligent remote driving of loaders, and more specifically, to a reversing obstacle avoidance method, a reversing obstacle avoidance system and a medium for remote driving of loaders. Background Art

[0002] Existing loaders generally use ultrasonic radars. When an obstacle is detected, the vehicle stops and can only continue to operate after the obstacle is removed, which limits the working environment requirements of the vehicle. In addition, the ultrasonic radar has a short detection distance. For fast-moving vehicles, the braking distance is long and it is easy to hit obstacles. The driver feels that the vehicle stops suddenly and is not prepared, which makes the driving experience bad. In addition, the use of ultrasonic radar alone to assist in reversing has problems such as insufficient ranging accuracy and large noise interference, especially poor reliability in complex environments. Summary of the invention

[0003] The present application is provided to solve the above-mentioned defects existing in the prior art. A method, system and medium for reversing obstacle avoidance for remote driving of a loader are needed, which can timely issue reminders by combining ultrasonic radar signals and millimeter wave radar signals, and can also improve the accuracy of reversing obstacle avoidance based on the characteristics of obstacles, and solve the problems of sudden braking and disadvantages for loader operation.

[0004] In a first aspect of the present application, a method for reversing and avoiding obstacles by remote driving of a loader is provided, the method comprising: acquiring ultrasonic radar data and millimeter-wave radar data; performing feature extraction on the ultrasonic radar data and millimeter-wave radar data respectively to obtain feature information; inputting the feature information into a deep neural network to obtain the type of obstacle, and sending it to a remote control end; based on the feature information, calculating the moving path and speed of the obstacle, and sending them to the remote control end; based on the obstacle type, the current position of the obstacle, the moving path and speed of the obstacle, predicting a first predicted position of the obstacle after a first time interval, and sending it to the remote control end; based on the current position of the obstacle and the current position of the loader, obtaining a lateral distance and a longitudinal distance in a two-dimensional direction between the obstacle and the loader; based on the speeds of the loader and the obstacle, obtaining a corresponding braking distance, and outputting a braking signal when the lateral distance is less than a first threshold and the longitudinal distance is less than or equal to the braking distance.

[0005] According to a second aspect of the present application, a reversing obstacle avoidance system for remote driving of a loader is provided, the reversing obstacle avoidance system comprising a vehicle controller, an ultrasonic radar and a millimeter-wave radar arranged on the loader, the vehicle controller being configured to: obtain ultrasonic radar data and millimeter-wave radar data; perform feature extraction on the ultrasonic radar data and the millimeter-wave radar data respectively to obtain feature information; input the feature information into a deep neural network to obtain an obstacle type, and send the feature information to a remote control terminal; based on the feature information, calculate a moving path and speed of the obstacle, and send the result to the remote control terminal; based on the obstacle type, the current position of the obstacle, the moving path and speed of the obstacle, predict a first predicted position of the obstacle after a first time interval, and send the result to the remote control terminal; based on the current position of the obstacle and the current position of the loader, obtain a lateral distance and a longitudinal distance between the obstacle and the loader in a two-dimensional direction; based on the speed of the loader and the obstacle, obtain a corresponding braking distance, and output a braking signal when the lateral distance is less than a first threshold value and the longitudinal distance is less than or equal to the braking distance.

[0006] A third aspect of the present application provides a non-transitory computer-readable medium having instructions stored thereon, which, when executed by a processor, execute the steps of the reversing obstacle avoidance method for remote driving of a loader as described in any embodiment of the present application.

[0007] The embodiments of the present application provide a reversing obstacle avoidance method, a reversing obstacle avoidance system and a medium for remote driving of a loader. The ultrasonic radar signal and the millimeter wave radar signal complement each other, and can detect not only static objects but also objects that are far away and moving quickly. The obstacle type is obtained by using a deep learning neural network, and the current position of the obstacle, the moving path and speed of the obstacle, and the predicted position of the obstacle can be calculated, so that timely and accurate reminders can be provided to the user, and the obstacle distance can be reminded in real time and the brake signal can be output in time. This can not only improve the accuracy of the reversing obstacle avoidance process control, but also solve the problems of sudden braking and unfavorable loader operation, and can also improve the user experience and safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] In the drawings, which are not necessarily drawn to scale, the same reference numerals may describe similar parts in different views. The same reference numerals with letter suffixes or different letter suffixes may represent different instances of similar parts. The accompanying drawings generally illustrate various embodiments by way of example and not limitation, and together with the specification and claims, are used to illustrate the embodiments applied for. When appropriate, the same reference numerals are used throughout the drawings to refer to the same or similar parts. Such embodiments are illustrative and are not intended to be exhaustive or exclusive embodiments of the present apparatus or method.

[0009] Figure 1 A flow chart showing a method for reversing and avoiding obstacles by remote driving of a loader according to an embodiment of the present application; Figure 2 A schematic diagram showing the arrangement of a millimeter wave radar and an ultrasonic radar according to an embodiment of the present application; Figure 3 A structural schematic diagram of a loader remote-controlled reversing obstacle avoidance system according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0010] In order to enable those skilled in the art to better understand the technical solution of the present application, the present application is described in detail below in conjunction with the accompanying drawings and specific implementation methods. The embodiments of the present application are further described in detail below in conjunction with the accompanying drawings and specific implementation methods, but are not intended to limit the present application.

[0011] The words "first", "second" and similar words used in this application do not indicate any order, quantity or importance, but are only used to distinguish. The words "include" or "comprise" and similar words mean that the elements before the word include the elements listed after the word, and do not exclude the possibility of including other elements.

[0012] According to an embodiment of the present application, a method for avoiding obstacles in a loader remotely driven by reversing is provided, and the method for avoiding obstacles comprises the following steps. Figure 1 As shown, in step 101, ultrasonic radar data and millimeter wave radar data are acquired.

[0013] In some embodiments, the ultrasonic data includes distance information, measurement time information, and signal strength information; the millimeter wave data includes distance parameters, speed parameters, and target characteristic parameter information of obstacles.

[0014] Ultrasonic radar encapsulates the signal, and the encapsulated ultrasonic radar signal is a CAN signal. The ultrasonic radar signal includes a frame header, frame length, timestamp, etc. The main function of ultrasonic radar is to measure distance, so distance data is the core part. This part of data usually includes information such as the measured distance value, measurement time, signal strength, etc. Communication protocol: The data format of ultrasonic radar represents the distance in a simple integer or floating point format. Based on the accuracy of the sensor, it may use a 16-bit or other bit length format and be parsed according to the protocol provided by the manufacturer.

[0015] Millimeter wave radar encapsulates the signal, and the encapsulated ultrasonic radar signal is an analog signal, which includes frame header, frame length, timestamp, parameters, etc. This information is used to identify the data frame and ensure the effective transmission and analysis of the data. The distance parameter represents the measured distance between the target and the radar; the speed parameter is measured by the Doppler effect and represents the speed of the target relative to the radar; the target characteristic parameters include the reflection intensity, size, angle, etc. of the target.

[0016] Communication protocol and data format between millimeter-wave radar and vehicle controller: Millimeter-wave radar often uses 16-bit binary complement format to represent data. The data can be stored in interleaved or non-interleaved mode, depending on the type or configuration of the radar chip. The vehicle controller can parse it according to the protocol provided by the manufacturer.

[0017] The combination of ultrasonic radar and millimeter-wave radar avoids the disadvantages of millimeter-wave radar that it cannot detect static objects, and the disadvantages of ultrasonic radar that it has a short detection distance and cannot accurately detect fast-moving objects.

[0018] In step 102, feature extraction is performed on the ultrasonic radar data and the millimeter wave radar data to obtain feature information. The feature extraction process is to extract key features from the ultrasonic data and the millimeter wave data, such as distance, speed, acceleration, object size, and reflectivity under specific environmental conditions. The current position of the obstacle and the distance between the obstacle and the loader can be directly obtained through the feature information.

[0019] In step 103, the feature information is input into the deep learning neural network to obtain the obstacle type and send it to the remote control terminal. The obstacle type may include people, vehicles and fixed obstacles, etc. Fixed obstacles may be walls, material piles, etc. Based on the obstacle type, it can be determined whether the obstacle is a static or dynamic obstacle, and further the moving path and speed of the obstacle can be accurately obtained.

[0020] In step 104, based on the characteristic information, the moving path and speed of the obstacle are calculated and sent to the remote control terminal. For example, the direction information, speed and current position of the obstacle can be used to obtain the moving path by using the distance formula, and the speed can be obtained by calculating the distance change in adjacent time periods. For example, the path of a person walking can be sent to the remote control terminal so that the user can directly follow the moving path and make dynamic adjustments to avoid obstacles.

[0021] In step 105, based on the obstacle type, the current position of the obstacle, the moving path and speed of the obstacle, the first predicted position of the obstacle after the first time interval is predicted and sent to the remote control terminal. The first time interval can be 1 second, 3 seconds, etc. The first predicted position and the second predicted position can be sent to the remote control terminal in real time. In this way, the position of the obstacle and the loader can be fed back in real time. Instead of notifying the user when the dangerous distance is reached, the user is given time to adjust dynamically.

[0022] In step 106, based on the current position of the obstacle and the current position of the loader, the lateral distance and longitudinal distance between the obstacle and the loader in the two-dimensional direction are obtained. For example, taking the middle position of the tail of the loader as the center point, the obstacle is located behind the loader, and there is a lateral distance and a longitudinal distance between the obstacle and the center point. The lateral distance is the distance in the left and right directions from the center point of the loader, and the longitudinal distance is the distance behind the center point of the loader. The longitudinal direction can be understood as the direction in which the loader is traveling straight or reversing.

[0023] In step 107, the corresponding braking distance is obtained based on the speed of the loader and the obstacle, and a braking signal is output when the lateral distance is less than the first threshold and the longitudinal distance is less than or equal to the braking distance. The speed of the loader and the obstacle in the longitudinal direction affects the braking distance, and the speed of the loader and the obstacle has a corresponding relationship with the braking distance, which can be preset. When the lateral distance is less than the first threshold and the longitudinal distance is less than or equal to the braking distance, it is considered to be relatively dangerous and a braking signal will be output.

[0024] The first threshold may be 1 m, etc. In this way, it is possible to determine whether to output a brake signal as the distance decreases, thereby improving safety.

[0025] Therefore, accurate, long-distance and real-time obstacle information can be obtained, which improves reliability; it can also facilitate users to perform real-time obstacle identification and dynamic obstacle avoidance strategy optimization, avoid sudden braking, facilitate continuous operation of the loader, and improve user experience and safety.

[0026] In addition, based on the current position of the loader and the speed of the loader, a second predicted position of the loader after the first time interval can be predicted and sent to the remote control end.

[0027] In some embodiments, the reversing obstacle avoidance method also includes: acquiring environmental information, wherein the environmental information includes lighting information and meteorological information; determining the weights of the ultrasonic data and the millimeter wave data based on the environmental information; performing weighted fusion of the ultrasonic data and the millimeter wave data based on the weights of the ultrasonic data and the millimeter wave data to obtain fused features; and inputting the fused features into a deep learning neural network to obtain the obstacle type.

[0028] By adjusting the weights in complex application scenarios, it is possible to cope with environmental changes, improve the accuracy of feature information in different scenarios, and improve the safety of loader operation in different scenarios. Environmental information can be obtained through environmental sensors and / or cameras on the loader, such as light, humidity and temperature sensors, and weather conditions such as day, night, rain and fog can be identified through image processing technology.

[0029] In some embodiments, the reversing obstacle avoidance method further includes: the weight of the millimeter wave data corresponding to the illumination value greater than the second threshold value indicated by the illumination information is less than the weight of the millimeter wave data corresponding to the illumination value less than the second threshold value, and the weight of the millimeter wave is greater than 0.5 and less than 0.9; If the meteorological information indicates fog, the weight of the millimeter wave data is greater than 0.8; if the meteorological information indicates rainfall, the weight of the millimeter wave data is greater than 0.5 and less than 0.7.

[0030] For example, if the illuminance value is greater than the second threshold, it is considered to be a daytime scene: under such lighting conditions, the sensor data quality is stable, and both millimeter-wave radar and ultrasonic radar data can be reliably collected. At this time, millimeter-wave radar is mainly used to provide long-distance obstacle recognition, and ultrasonic radar is used for close-range fine ranging; Weight adjustment: The weight adjustment scheme is: millimeter wave = 0.7, ultrasonic = 0.3. Because the impact of light is small, the focus is on high-definition long-distance detection.

[0031] If the illumination value is less than the second threshold, it is considered a dark scene: Under such lighting conditions, the light is poor, and the advantage of millimeter-wave radar is apparent. At this time, the use of millimeter-wave radar is strengthened because it is not affected by light, and the reliance on ultrasonic radar for close-range accuracy is increased, especially in urban environments. The weight adjustment scheme is: millimeter wave = 0.8, ultrasonic = 0.2 to enhance the detection of distant and non-light-sensitive obstacles.

[0032] Meteorological information indicates rainfall: Raindrops may cause false alarms for ultrasonic radars and affect millimeter-wave radar signals. At this time, data filtering can be strengthened, Kalman filtering can be used to correct interference, and the millimeter-wave radar can be relied on to process long-distance information, while cross-validating multi-sensor data. The weight adjustment scheme is: millimeter wave = 0.6, ultrasonic = 0.4, increasing data hoarding and fusion to improve system robustness.

[0033] Meteorological information indicates fog: millimeter-wave radar has a strong ability to penetrate fog, but ultrasonic waves are greatly affected by changes in environmental density. At this time, millimeter-wave radar is mainly relied on for detection within all distance ranges, and ultrasonic radar data is used as an auxiliary to reduce errors caused by abnormal environmental density. The weight adjustment plan is: the weight is set to millimeter wave = 0.9, ultrasonic wave = 0.1, giving priority to ensuring the accuracy of millimeter waves for medium and long distances.

[0034] By dynamically adjusting sensor weights and flexible algorithm strategies, the reliability of vehicle detection and obstacle avoidance functions is ensured to adapt to changes in various environmental conditions.

[0035] In some embodiments, the reversing obstacle avoidance method also includes: timestamping the ultrasonic radar signal by the ultrasonic radar; timestamping the millimeter wave radar signal by the millimeter wave radar; and extracting features from the ultrasonic radar signal and the millimeter wave radar signal when the timestamps of the ultrasonic radar signal and the millimeter wave radar signal are consistent.

[0036] Each sensor will be accompanied by an accurate timestamp when collecting data, recording the time when the data was obtained. The data acquisition module ensures that each sensor collects data in the same time interval, and aligns the data from different sensors through timestamp comparison to ensure that the data at the same time is used in subsequent processing, ensuring the accuracy of the distance between the obstacle and the loader.

[0037] In some embodiments, based on the ultrasonic data and millimeter wave data input into a deep learning neural network, the obstacle type is obtained. The deep learning neural network includes a feature extraction network and a data processing network. The feature extraction network is used to obtain the spatial characteristics of the obstacle based on the ultrasonic data and millimeter wave data; the data processing network is used to classify the obstacle based on the spatial characteristics of the obstacle to obtain the obstacle type, and the obstacle type includes fixed obstacles, people and vehicles.

[0038] Feature extraction networks include CNN, etc., and data processing networks include RNN, etc., which are used to process sequence data. The deep learning neural network classifies the input feature vector and can obtain the obstacle type.

[0039] In some embodiments, the reversing obstacle avoidance method further includes: 1) When a brake signal is received and the speed of the travel motor is greater than the third threshold, a torque clearing signal and a brake signal are output; 2) When the speed of the loader is 0 and the speed of the travel motor is less than the third threshold, the brake mode is exited; 3) When the loader speed is 0, the brake mode is exited, and the reverse mode is entered when a reverse signal is received.

[0040] 1) The corresponding situations include: during the reversing process, there is still a certain distance between the obstacle and the loader. At this time, the remote control terminal has issued a brake signal, and then it is determined whether the current travel motor speed is greater than the third threshold. If it is greater than the third threshold, the travel motor clears the torque and the vehicle brakes at the same time. The purpose of outputting the clear torque signal is to ensure that the vehicle does not slip after braking.

[0041] 2) The corresponding situations include: the loader has stopped, but the travel motor speed is less than the third threshold, indicating that it is not moving further, and the braking mode is exited; but if a reversing signal is received after exiting the braking mode, the brakes can be applied again.

[0042] 3) The corresponding situations include: after the vehicle stops and the brake is released, the driver can control whether to brake again when reversing through the control button, so as to reduce the distance the vehicle encounters obstacles to facilitate the vehicle's movement and the vehicle can continue to operate when the obstacle does not affect the vehicle's operation.

[0043] Therefore, during the reversing process, if the vehicle encounters an obstacle and brakes, if the driver (remote control operator) still wants to reverse, he can actively control the vehicle to reverse again so that the vehicle is not affected by the working environment.

[0044] In some embodiments, the deep learning neural network is trained based on multiple ultrasonic data and millimeter wave data and the corresponding actual obstacle types. According to historical data and marked obstacle categories, the stored ultrasonic data and millimeter wave data and the corresponding actual obstacle categories can be used for training. Preferably, the historical data may include ultrasonic data and millimeter wave data under different meteorological conditions and different lighting conditions, as well as the corresponding actual obstacle categories. The deep learning neural network predicts and compares the obstacle type with the actual obstacle category, and then uses the loss function for training to distinguish the obstacle types in different environments.

[0045] like Figure 3As shown, according to an embodiment of the present application, a reversing obstacle avoidance system for remote driving of a loader is also provided, wherein the reversing obstacle avoidance system 300 comprises a vehicle controller 302, an ultrasonic radar 303 and a millimeter-wave radar 304 arranged on the loader, wherein the vehicle controller 302 is configured to: obtain ultrasonic radar data and millimeter-wave radar data; perform feature extraction on the ultrasonic radar data and the millimeter-wave radar data respectively to obtain feature information; input the feature information into a deep neural network to obtain the obstacle type, and send it to the remote control terminal 301; based on the feature information, calculate the moving path and speed of the obstacle, and send it to the remote control terminal 301; based on the The first predicted position of the obstacle after the first time interval is predicted based on the obstacle type, the current position of the obstacle, the moving path and the speed of the obstacle, and the predicted position is sent to the remote control end 301; based on the current position of the loader and the speed of the loader, the second predicted position of the loader is obtained and sent to the remote control end 301; based on the current position of the obstacle and the current position of the loader, the lateral distance and the longitudinal distance between the obstacle and the loader in the two-dimensional direction are obtained; based on the speed of the loader and the obstacle, the corresponding braking distance is obtained, and a braking signal is output when the lateral distance is less than the first threshold value and the longitudinal distance is less than or equal to the braking distance.

[0046] The remote control terminal 301 is in communication connection with the vehicle controller 302 of the loader.

[0047] In some embodiments, the ultrasonic radar and the millimeter-wave radar are arranged at the tail of the loader. Preferably, 4 ultrasonic radars and 1 millimeter-wave radar are arranged, and the millimeter-wave radar is arranged in the middle position, with two ultrasonic radars on each side of the millimeter-wave radar. Ultrasonic detection range: 0-5m, 30° up and down, 90° left and right; millimeter-wave detection range: 0.2-40m, 20° up and down, 120° left and right. Therefore, when the distance between the obstacle and the loader is less than or equal to 5m, the data of the ultrasonic radar and the millimeter-wave radar are received. When the distance between the obstacle and the loader is greater than 5m and less than or equal to 40m, the data of the millimeter-wave radar is received. In order to accurately obtain the relevant information of the obstacle.

[0048] In some embodiments, the vehicle controller 302 is further configured to: obtain environmental information, which includes lighting information and meteorological information; determine the weights of the ultrasonic data and millimeter wave data based on the environmental information; perform weighted fusion of the ultrasonic data and millimeter wave data based on the weights of the ultrasonic data and millimeter wave data to obtain fused features; and input the fused features into a deep learning neural network to obtain the obstacle type.

[0049] In some embodiments, the vehicle controller 302 is also configured to: timestamp the ultrasonic radar signal by the ultrasonic radar; timestamp the millimeter-wave radar signal by the millimeter-wave radar; and perform feature extraction on the ultrasonic radar signal and the millimeter-wave radar signal when the timestamps of the ultrasonic radar signal and the millimeter-wave radar signal are consistent.

[0050] In some embodiments, the vehicle controller 302 is further configured to: 1) When a brake signal is received and the speed of the travel motor is greater than the third threshold, a torque clearing signal and a brake signal are output; 2) When the speed of the loader is 0 and the speed of the travel motor is less than the third threshold, the brake mode is exited; 3) When the loader speed is 0, the brake mode is exited, and the reverse mode is entered when a reverse signal is received.

[0051] In some embodiments, the vehicle controller 302 is further configured to train a deep learning neural network based on multiple ultrasonic data and millimeter wave data and the corresponding actual obstacle types.

[0052] The remote control end can be set up with a remote display screen, which can display the real-time distance of obstacles from the rear of the vehicle on the remote display screen after receiving the signal from the vehicle controller, so that the driver can observe the obstacle avoidance distance in real time. When the obstacle is too close, the remote display screen will pop up a reminder "The obstacle behind is too close" to optimize the driver's driving experience.

[0053] According to an embodiment of the present application, a non-transitory computer-readable medium is also provided, on which instructions are stored. When executed by a processor, the instructions execute the steps of the reversing obstacle avoidance method for remote driving of a loader as described in any embodiment of the present application.

[0054] In addition, although exemplary embodiments have been described herein, the scope includes any and all embodiments based on the present application with equivalent elements, modifications, omissions, combinations (e.g., various embodiments intersecting schemes), adaptations or changes. The elements in the claims are to be interpreted broadly based on the language adopted in the claims, and are not limited to the examples described in this specification or during the practice of this application, and the examples are to be interpreted as non-exclusive. Therefore, this specification and examples are intended to be considered as examples only, and the true scope and spirit are indicated by the following claims and the full scope of their equivalents.

[0055] The above description is intended to be illustrative rather than restrictive. For example, the above examples (or one or more schemes thereof) may be used in combination with each other. For example, a person of ordinary skill in the art may use other embodiments when reading the above description. In addition, in the above-mentioned specific embodiments, various features may be grouped together to simplify the present application. This should not be interpreted as an intention that a feature of an application that does not require protection is necessary for any claim. On the contrary, the subject matter of the present application may be less than all the features of an embodiment of a particular application. Thus, the following claims are incorporated herein in the specific embodiments as examples or embodiments, wherein each claim is independently a separate embodiment, and it is considered that these embodiments may be combined with each other in various combinations or arrangements. The scope of the present invention should be determined with reference to the appended claims and the full scope of equivalent forms granted by these claims.

[0056] The above embodiments are only exemplary embodiments of the present application and are not intended to limit the present invention. The protection scope of the present invention is defined by the claims. Those skilled in the art may make various modifications or equivalent substitutions to the present invention within the essence and protection scope of the present application, and such modifications or equivalent substitutions shall also be deemed to fall within the protection scope of the present invention.

Claims

1. A method for reversing and avoiding obstacles by remote driving of a loader, characterized in that: The obstacle avoidance method comprises: Acquire ultrasonic radar data and millimeter wave radar data; Extract features from the ultrasonic radar data and the millimeter wave radar data to obtain feature information; input the feature information into a deep neural network to obtain the obstacle type, and send it to the remote control end; Based on the characteristic information, the moving path and speed of the obstacle are calculated and sent to the remote control terminal; Based on the obstacle type, the current position of the obstacle, the moving path and speed of the obstacle, predict a first predicted position of the obstacle after a first time interval, and send the predicted position to the remote control terminal; Based on the current position of the obstacle and the current position of the loader, a lateral distance and a longitudinal distance between the obstacle and the loader in a two-dimensional direction are obtained; A corresponding braking distance is obtained based on the speeds of the loader and the obstacle, and a braking signal is output when the lateral distance is less than a first threshold and the longitudinal distance is less than or equal to the braking distance.

2. The method for avoiding obstacles while reversing according to claim 1, characterized in that: The obstacle avoidance method further comprises: Acquiring environmental information, wherein the environmental information includes lighting information and meteorological information; Based on the environmental information, determining weights of the ultrasonic data and the millimeter wave data; Based on the weights of the ultrasonic data and the millimeter wave data, weighted fusion is performed on the ultrasonic data and the millimeter wave data to obtain fused features; The fused features are input into the deep learning neural network to obtain the obstacle type.

3. The method for avoiding obstacles while reversing according to claim 2, characterized in that: The obstacle avoidance method further includes: the weight of the millimeter wave data corresponding to the illumination value greater than the second threshold value indicated by the illumination information is less than the weight of the millimeter wave data corresponding to the illumination value less than the second threshold value, and the weight of the millimeter wave is greater than 0.5 and less than 0.9; If the meteorological information indicates fog, the weight of the millimeter wave data is greater than 0.8; if the meteorological information indicates rainfall, the weight of the millimeter wave data is greater than 0.5 and less than 0.

7.

4. The method for avoiding obstacles while reversing according to claim 1, characterized in that: The obstacle avoidance method further includes: timestamping the ultrasonic radar signal by the ultrasonic radar; timestamping the millimeter wave radar signal by the millimeter wave radar; When the timestamps of the ultrasonic radar signal and the millimeter-wave radar signal are consistent, feature extraction is performed on the ultrasonic radar signal and the millimeter-wave radar signal.

5. The method for avoiding obstacles while reversing according to claim 1, characterized in that: The deep learning neural network includes a feature extraction network and a data processing network, wherein the feature extraction network is used to obtain the spatial features of the obstacle based on the ultrasonic data and the millimeter wave data; The data processing network is used to classify the obstacles based on their spatial features to obtain obstacle types, where the obstacle types include fixed obstacles, people, and vehicles.

6. The method for avoiding obstacles while reversing according to claim 1, characterized in that: The obstacle avoidance method further comprises: When a brake signal is received and the speed of the travel motor is greater than a third threshold, a torque clearing signal and a brake signal are output; When the speed of the loader is 0 and the speed of the travel motor is less than the third threshold, the brake mode is exited; after exiting the brake mode, When the loader speed is 0, the brake mode is exited, and the reverse mode is entered after receiving the reverse signal.

7. The backing obstacle avoidance method according to claim 1, characterized in that: The deep learning neural network is trained based on multiple ultrasonic data and millimeter wave data and the corresponding actual obstacle types.

8. The method for avoiding obstacles while reversing according to claim 1, characterized in that: The ultrasonic data includes distance information, measurement time information and signal strength information; the millimeter wave data includes distance parameters, speed parameters and target characteristic parameter information of obstacles.

9. A loader remote-controlled reversing obstacle avoidance system, characterized in that: The reversing obstacle avoidance system includes a vehicle controller, an ultrasonic radar and a millimeter-wave radar arranged on a loader, and the vehicle controller is configured to: obtain ultrasonic radar data and millimeter-wave radar data; perform feature extraction on the ultrasonic radar data and millimeter-wave radar data respectively to obtain feature information; input the feature information into a deep neural network to obtain the obstacle type, and send it to a remote control end; based on the feature information, calculate the moving path and speed of the obstacle, and send them to the remote control end; based on the obstacle type, the current position of the obstacle, the moving path and speed of the obstacle, predict the first predicted position of the obstacle after a first time interval, and send it to the remote control end; based on the current position of the obstacle and the current position of the loader, obtain the lateral distance and longitudinal distance between the obstacle and the loader in a two-dimensional direction; based on the speed of the loader and the obstacle, obtain the corresponding braking distance, and output a braking signal when the lateral distance is less than a first threshold and the longitudinal distance is less than or equal to the braking distance.

10. A non-transitory computer-readable medium having instructions stored thereon, which, when executed by a processor, execute the steps of the method for reversing and avoiding obstacles by remote driving of a loader according to any one of claims 1 to 8.

Images