Method and device for detecting obstacle of unmanned vehicle in strip mine area

Abstract

The invention provides a strip mine area unmanned vehicle obstacle detection method and detection device. Comprising the steps that pose data detected by inertial navigation systems installed in four directions of the unmanned vehicle in the strip mine area are received, the four directions correspond to four wheels, and at least one inertial navigation system is installed in any direction; under the condition that the deviation of the pose data detected in any two orientations is larger than preset deviation, it is determined that at least one of the wheels corresponding to any two orientations rolls the protrusions or pits, the wheels with the abnormal height are determined according to the height data of the wheels, the deviation of the pose data comprises position deviation and angle deviation, and the deviation of the pose data comprises the position deviation and the angle deviation. The height data is obtained from the pose data, and the wheels with abnormal height are wheels with abnormal height data; and determining position information and size information of the bulge or the pit based on the pose data of the wheel with the abnormal height. According to the scheme, the problem that in the prior art, an obstacle detection method for an unmanned vehicle in a strip mine area is limited in sensing capacity is solved.

Description

Technical Field

[0001] The present application relates to the field of autonomous driving and perception, and more specifically, to an obstacle detection method for an unmanned vehicle in an open-pit mine, a detection device, a computer-readable storage medium, and an unmanned vehicle in an open-pit mine. Background Art

[0002] In the existing technology, the perception ability of unmanned vehicles in open-pit mines is limited by the single environment of the mines, the absence of landmarks and road signs, and the small differentiation between the target objects and the background, resulting in low perception accuracy and recall rate.

[0003] In order to improve the autonomous driving perception capability in mining areas, current solutions often use radar, camera, ultrasonic radar and millimeter wave radar for fusion perception, but the effect is still not good and the price is expensive. There is limited recognition of obstacles such as fallen rocks and potholes on the road in open-pit mines. Summary of the invention

[0004] The main purpose of the present application is to provide an open-pit mine unmanned vehicle obstacle detection method, detection device, computer-readable storage medium and open-pit mine unmanned vehicle, so as to at least solve the problem of limited perception ability of the existing open-pit mine unmanned vehicle obstacle detection method.

[0005] In order to achieve the above-mentioned purpose, according to one aspect of the present application, a method for obstacle detection of an unmanned vehicle in an open-pit mine is provided, comprising: receiving posture data detected by an inertial navigation system installed at four positions of an unmanned vehicle in an open-pit mine, wherein the four positions correspond to four wheels, and at least one of the inertial navigation systems is installed at any of the positions; when the deviation of the posture data detected at any two of the positions is greater than a preset deviation, determining that at least one of the wheels corresponding to any two of the positions has rolled over a protrusion or a pit, and determining a highly abnormal wheel based on the height data of the wheel, wherein the deviation of the posture data includes a position deviation and an angle deviation, the height data is obtained from the posture data, and the highly abnormal wheel is a wheel with abnormal height data; based on the posture data of the highly abnormal wheel, determining the position information and size information of the protrusion or the pit.

[0006] Optionally, determining a wheel with abnormal height according to the height data of the wheel includes: when the difference between the height of the first target wheel from the ground and the heights of multiple non-first target wheels from the ground is greater than a first preset height difference, determining the first target wheel as the wheel with abnormal height, and the first target wheel is one of four wheels; when the difference between the height data of at least two second target wheels among the four wheels and the corresponding reference height data is greater than a second preset height difference, determining at least two second target wheels as the wheels with abnormal height, and the second target wheel is one of the four wheels, and the reference height data is height data obtained by driving on a flat road.

[0007] Optionally, based on the posture data of the wheel with abnormal height, determining the size information of the protrusion or the pit includes: obtaining an average value of the difference between the height of the first target wheel and all the non-first target wheels from the ground to obtain a first average value, and determining the first average value as the height of the protrusion or the depth of the pit; obtaining an average value of the difference between at least two of the second target wheels and the corresponding reference height data to obtain a second average value, and determining the second average value as the height of the protrusion or the depth of the pit.

[0008] Optionally, based on the posture data of the highly abnormal wheel, determining the position information of the protrusion or the pit includes: extracting a first horizontal coordinate and a second horizontal coordinate from the posture data of the highly abnormal wheel; and determining the position information of the protrusion or the pit according to the first horizontal coordinate and the second horizontal coordinate.

[0009] Optionally, multiple inertial navigation systems are installed at any of the positions, and receiving the posture data detected by the inertial navigation systems installed at four positions of the unmanned vehicle in the open-pit mine includes: receiving the posture data detected by multiple inertial navigation systems installed at any of the positions of the unmanned vehicle in the open-pit mine; after receiving the posture data detected by the inertial navigation systems installed at four positions of the unmanned vehicle in the open-pit mine, the method also includes: determining the average value of the posture data detected by multiple inertial navigation systems installed at any of the positions as the final posture data detected at the position.

[0010] Optionally, the method further includes: storing the position information and size information of the protrusion or the pit; and sharing the position information and size information of the protrusion and the pit with other vehicles via vehicle-to-vehicle communication technology V2v or vehicle-to-network communication technology V2N.

[0011] Optionally, the installation position of the inertial navigation system in any of the orientations can be adjusted within a preset position range, and the inertial navigation systems installed in the four orientations are axially symmetrically arranged.

[0012] According to another aspect of the present application, an obstacle detection device for an unmanned vehicle in an open-pit mine is provided, comprising:

[0013] A receiving unit, used to receive position data detected by an inertial navigation system installed at four positions of an unmanned vehicle in an open-pit mine, wherein the four positions correspond to four wheels, and at least one inertial navigation system is installed at any of the positions;

[0014] a first determining unit, configured to determine that at least one of the wheels corresponding to any two of the directions has rolled over a protrusion or a pit, and to determine a wheel with abnormal height according to the height data of the wheel, when the deviation of the posture data detected at any two of the directions is greater than a preset deviation, wherein the deviation of the posture data includes a position deviation and an angle deviation, the height data is obtained from the posture data, and the wheel with abnormal height is a wheel with abnormal height data;

[0015] The second determination unit is used to determine the position information and size information of the protrusion or the pit based on the posture data of the wheel with abnormal height.

[0016] According to another aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium includes a stored program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute any one of the methods for detecting obstacles in an open-pit mine unmanned vehicle.

[0017] According to another aspect of the present application, an open-pit mine unmanned vehicle is provided, comprising a controller, wherein the controller executes any one of the methods for detecting obstacles by the open-pit mine unmanned vehicle.

[0018] Applying the technical solution of the present application, firstly, the posture data detected by the inertial navigation system installed at the four positions of the unmanned vehicle in the open-pit mine is received, wherein the four positions correspond to four wheels, and at least one of the inertial navigation systems is installed at any of the positions; then, when the deviation of the posture data detected at any two of the positions is greater than the preset deviation, it is determined that at least one of the wheels corresponding to any two of the positions has rolled over a protrusion or a pit, and the highly abnormal wheel is determined according to the height data of the wheel, wherein the deviation of the posture data includes position deviation and angle deviation, the height data is obtained from the posture data, and the highly abnormal wheel is the wheel with abnormal height data; finally, based on the posture data of the highly abnormal wheel, the position information and size information of the protrusion or the pit are determined. In this solution, by installing the inertial navigation system at the four positions of the unmanned vehicle in the open-pit mine, the position and height information of the abnormal wheel are determined, and then the position information and size information of the obstacle are determined, thereby solving the problem of limited perception ability of the obstacle detection method of the unmanned vehicle in the open-pit mine in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The drawings constituting part of the present application are used to provide a further understanding of the present application. The exemplary embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0020] Figure 1 A hardware structure block diagram of a mobile terminal of an open-pit mine unmanned vehicle obstacle detection method provided in an embodiment of the present application is shown;

[0021] Figure 2 A schematic diagram of a process for detecting obstacles of an unmanned vehicle in an open-pit mine provided according to an embodiment of the present application is shown;

[0022] Figure 3 A schematic diagram of the installation of an inertial navigation system for an obstacle detection method for an unmanned vehicle in an open-pit mine provided in accordance with an embodiment of the present application is shown;

[0023] Figure 4 The overall flow chart of an obstacle detection method for an unmanned vehicle in an open-pit mine provided according to an embodiment of the present application is shown;

[0024] Figure 5 A schematic diagram of an obstacle detection device for an unmanned vehicle in an open-pit mine provided according to an embodiment of the present application is shown;

[0025] Figure 6 A schematic diagram of an unmanned vehicle in an open-pit mine according to an obstacle detection method for an unmanned vehicle in an open-pit mine provided in an embodiment of the present application is shown.

[0026] The above drawings include the following reference numerals:

[0027] 102, processor; 104, memory; 106, transmission device; 108, input and output devices. DETAILED DESCRIPTION

[0028] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0029] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0030] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0031] As introduced in the background technology, the existing technology often uses radar, camera, ultrasonic radar and millimeter wave radar for fusion perception, but the effect is still not good and the price is expensive. The recognition of obstacles such as fallen rocks and potholes on the road in open-pit mines is limited. In order to solve the problem of limited perception ability of obstacle detection methods for unmanned vehicles in open-pit mines, the embodiments of the present application provide an obstacle detection method for unmanned vehicles in open-pit mines, a detection device, a computer-readable storage medium and an unmanned vehicle in open-pit mines.

[0032] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0033] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking running on a mobile terminal as an example, Figure 11 is a hardware structure block diagram of a mobile terminal of an open-pit mine unmanned vehicle obstacle detection method according to an embodiment of the present invention. Figure 1 As shown, the mobile terminal may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the mobile terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the mobile terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0034] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the obstacle detection method for unmanned vehicles in open-pit mines in the embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories can be connected to the mobile terminal via a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof. The transmission device 106 is used to receive or send data via a network. The above-mentioned specific examples of the network may include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0035] In this embodiment, a method for detecting obstacles in an open-pit mine unmanned vehicle running on a mobile terminal, a computer terminal or a similar computing device is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in an order different from that shown here.

[0036] Figure 2 FIG. 1 is a flow chart of an obstacle detection method for an unmanned vehicle in an open-pit mine according to an embodiment of the present application. Figure 2 As shown, the method comprises the following steps:

[0037] Step S201, receiving the position and posture data detected by the inertial navigation system installed at four positions of the unmanned vehicle in the open-pit mine, wherein the four positions correspond to four wheels, and at least one of the inertial navigation systems is installed at any of the four positions;

[0038] Specifically, at least one inertial navigation system is installed at four positions corresponding to the four wheels of the vehicle. The installation diagram of the inertial navigation system is as follows: Figure 3 As shown, the four directions refer to the front left, front right, rear left and rear right of the vehicle. At least one inertial navigation system is installed in each direction, that is, at least four inertial navigation systems are distributed in the four corners of the vehicle, close to each wheel respectively, and can be installed in the front left, front right, rear left and rear right of the vehicle chassis. Such a layout can more comprehensively monitor the posture changes of the vehicle during driving, especially when the wheels encounter obstacles, such as fallen rocks or potholes on the road. The inertial navigation system can capture the slight posture changes when the wheels contact the ground.

[0039] Specifically, the inertial navigation system is also called an inertial navigation system. The inertial navigation system can provide the vehicle's positioning, speed and attitude information, including three-dimensional position, three-dimensional speed and angular velocity in three directions.

[0040] Step S202, when the deviation of the posture data detected at any two of the above directions is greater than a preset deviation, it is determined that at least one of the wheels corresponding to any two of the above directions has rolled over a protrusion or a pit, and a height-abnormal wheel is determined according to the height data of the above wheels, wherein the deviation of the posture data includes a position deviation and an angle deviation, the above height data is obtained from the posture data, and the height-abnormal wheel is a wheel with abnormal height data;

[0041] Specifically, when the vehicle passes through a certain area, when the deviation between the posture data detected by the inertial navigation system in any two directions exceeds the preset deviation, this indicates that the vehicle may have encountered an uneven road surface, that is, a wheel may have run over a bump or a pit. Once it is detected that the posture data deviation exceeds the preset deviation, further analysis will be performed to determine which wheel or wheels have encountered an obstacle. If the height data of a certain wheel is significantly different from the height data of other wheels, that is, it is considered to be a wheel with abnormal height data, then this wheel is considered to be a highly abnormal wheel, that is, the wheel has run over an obstacle. Through the above steps, the wheel that has run over a bump or a pit, that is, the highly abnormal wheel, can be determined, which is the key to the subsequent positioning of the specific position of the obstacle and determining the size of the obstacle.

[0042] Step S203, determining the position information and size information of the protrusion or the pit based on the posture data of the wheel with abnormal height.

[0043] Specifically, the position data of the wheel with abnormal height and the posture data can be used to determine the position information of the obstacle at the wheel location, that is, the specific position of the protrusion or the pit. At the same time, the height data of the wheel with abnormal height can be used to determine the size information of the obstacle. For a protrusion, the height data of the wheel with abnormal height corresponds to the height of the protrusion. For a pit, the height data of the wheel with abnormal height corresponds to the depth of the pit.

[0044] In the above embodiment, firstly, the posture data detected by the inertial navigation system installed at the four positions of the unmanned vehicle in the open-pit mine is received, wherein the four positions correspond to four wheels, and at least one of the above inertial navigation systems is installed at any of the above positions; then, when the deviation of the above posture data detected at any two of the above positions is greater than the preset deviation, it is determined that at least one of the wheels corresponding to any two of the above positions has rolled over a protrusion or a pit, and the highly abnormal wheel is determined according to the height data of the above wheel, wherein the deviation of the above posture data includes position deviation and angle deviation, the above height data is obtained from the above posture data, and the highly abnormal wheel is the wheel with abnormal height data; finally, based on the posture data of the above highly abnormal wheel, the position information and size information of the above protrusion or the above pit are determined. In this scheme, by installing the inertial navigation system at the four positions of the unmanned vehicle in the open-pit mine, the position and height information of the abnormal wheel are determined, and then the position information and size information of the obstacle are determined, thereby solving the problem of limited perception ability of the obstacle detection method of the unmanned vehicle in the open-pit mine in the prior art.

[0045] In one embodiment of the present application, determining a wheel with abnormal height based on the height data of the above-mentioned wheels includes: when the difference between the height of the first target wheel from the ground and the heights of multiple non-first target wheels from the ground are all greater than a first preset height difference, determining the above-mentioned first target wheel as the above-mentioned wheel with abnormal height, and the above-mentioned first target wheel is one of the four wheels; when the difference between the above-mentioned height data of at least two second target wheels among the four wheels and the corresponding reference height data is greater than a second preset height difference, determining at least two of the above-mentioned second target wheels as the above-mentioned wheels with abnormal height, and the above-mentioned second target wheel is one of the four wheels, and the above-mentioned reference height data is height data obtained by driving on a flat road.

[0046] Specifically, in an autonomous driving vehicle, there are usually four wheels, and an inertial navigation system is installed near each wheel to monitor the relative height between the wheel and the ground. When the vehicle is driving, if the height of a wheel center from the ground suddenly changes, and this change exceeds a preset threshold, that is, the above-mentioned first preset height difference, and at the same time, if the height change of other wheels centers from the ground is not so large, then it is considered that the wheel with a larger change has encountered a bump or a pit. This is because under normal driving conditions, the height difference between all wheels and the ground should be relatively stable and similar. If the height difference of one of the wheels suddenly increases or decreases, and the height difference of other wheels does not undergo similar changes, it indicates that the wheel has run over a bump or driven into a pit. For example, when the left front wheel runs over a fallen rock, the height of the left front wheel will suddenly rise, resulting in a significant difference in height with the other three wheels. At this time, if the difference in height between the left front wheel and the ground and the other three wheels exceeds a first preset height difference, the left front wheel will be identified as an abnormally high wheel, that is, the left front wheel has run over an obstacle. The first preset height difference can be set to a few centimeters or tens of centimeters, and the specific value can be determined based on the size of the obstacle and the safety needs of the vehicle.

[0047] Specifically, the above-mentioned reference height data refers to the relative height between the wheels and the ground when the vehicle does not encounter any obstacles, that is, when it is driving on a flat road. When the vehicle encounters a protrusion or a pit, the distance between the wheel that rolls over the protrusion or enters the pit and the ground will suddenly change. If the difference between two or more wheels and the reference height exceeds the second preset height difference, it is considered that these wheels have rolled over an obstacle. For example, when the vehicle drives into a larger pit or rolls over multiple obstacles at the same time, the height data of more than one wheel will change significantly. When the two front wheels of the vehicle roll over fallen rocks at the same time, the height of these two wheels will be much higher than the rear wheels and the reference height data. In this case, if the difference between the height data of the two front wheels and the corresponding reference height data is greater than the second preset height difference, it is considered that the two front wheels are wheels with abnormal heights. For another example, when the left front wheel and two rear wheels of a vehicle enter a pothole at the same time, the heights of these three wheels will be much lower than the right front wheel and the reference height data. In this case, if the difference between the height data of the left front wheel and the two rear wheels and the corresponding reference height data is greater than the second preset height difference, the left front wheel and the two rear wheels are considered to be wheels with abnormal heights. The second preset height difference can be determined according to the size of the obstacle and the safety requirements of the vehicle.

[0048] Specifically, the benchmark height data refers to the standard distance between the wheels and the ground when the vehicle does not encounter any obstacles. It is determined by the historical wheel height data of the vehicle when it is driving on a flat road. The benchmark height data provides a reference point, enabling the vehicle to compare the real-time height change data with this benchmark value to determine whether the vehicle has encountered an obstacle on the road.

[0049] In a specific embodiment, based on the posture data of the wheel with abnormal height, the size information of the protrusion or the pit is determined, including: obtaining the average value of the difference between the height of the first target wheel and all the non-first target wheels from the ground to obtain a first average value, and determining the first average value as the height of the protrusion or the depth of the pit; obtaining the average value of the difference between at least two of the second target wheels and the corresponding reference height data to obtain a second average value, and determining the second average value as the height of the protrusion or the depth of the pit.

[0050] Specifically, the height difference between the abnormal wheel and all normal wheels is obtained, the average value of these differences is calculated, and the calculated average value is used as the height of the protrusion or the depth of the pit. This is because when the vehicle runs over an obstacle, the height of the abnormal wheel will be significantly different from that of other normal wheels, and the difference reflects the height of the protrusion or the depth of the pit. For example, assuming that the right front wheel runs over a fallen rock, the height of the right front wheel relative to the ground is recorded as H1, and the left front wheel, left rear wheel and right rear wheel do not encounter obstacles. The heights of the left front wheel, left rear wheel and right rear wheel relative to the ground are recorded as H2, H3 and H4 respectively, and then the height difference between the right front wheel and the left front wheel ΔH1=H1-H2, the height difference between the right front wheel and the left rear wheel ΔH2=H1-H3, and the height difference between the right front wheel and the right rear wheel ΔH3=H1-H4 are calculated. Then, these differences are averaged to obtain the above-mentioned first average value, that is, the above-mentioned first average value is (ΔH1+ΔH2+ΔH3) / 3. The first average value reflects the height of the fallen rock encountered by the right front wheel. Therefore, this average value can be regarded as the height of the fallen rock.

[0051] Specifically, the height of the wheel encountering an obstacle is compared with the reference height data to obtain a difference. At the same time, if other wheels also encounter an obstacle, the difference between their heights and the reference height data is also obtained. Then, these differences are averaged to obtain the second average value mentioned above. According to the obtained second average value, the size of the obstacle is determined. If the second average value is positive, it indicates that the wheel has encountered a protrusion, and the size of the second average value is the height of the protrusion. On the contrary, if the second average value is negative, it indicates that the wheel has encountered a pit, and the absolute value of the second average value is the depth of the pit. For example, when the vehicle is driving normally, the reference height data is h, and the height data of all four wheels is h at this time. When the height of the left front wheel becomes H5 and the height of the right front wheel becomes H6, if H5 < h and H6 < h, it indicates that the heights of the left front wheel and the right front wheel have both decreased, which means that the left front wheel and the right front wheel have entered a pit. Calculate the difference between the height of the left front wheel and the reference height data ΔH4 = h - H5, and the difference between the height of the right front wheel and the reference height data ΔH5 = h - H6. Then calculate the average value of the difference ΔH4 and the difference ΔH5 as (ΔH4 + ΔH5) / 2, and this average value (ΔH4 + ΔH5) / 2 is the depth of the depression of the pit mentioned above.

[0052] In an embodiment of the present application, based on the pose data of the above-mentioned height-abnormal wheel, the position information of the above-mentioned protrusion or the above-mentioned pit is determined, including: extracting a first horizontal coordinate and a second horizontal coordinate from the pose data of the above-mentioned height-abnormal wheel; determining the position information of the above-mentioned protrusion or the above-mentioned pit according to the first horizontal coordinate and the second horizontal coordinate.

[0053] Specifically, first, a corresponding first horizontal coordinate and a second horizontal coordinate are extracted from the pose data provided by the inertial navigation system associated with the wheel detected with abnormal height. The first horizontal coordinate usually refers to the X-axis coordinate, and the second horizontal coordinate usually refers to the Y-axis coordinate. These coordinate information reflect the position of the wheel at the moment of contacting the obstacle. Then, using the extracted first horizontal coordinate and second horizontal coordinate, the exact position of the protrusion or the pit on the ground can be determined. For example, if the driving trajectory of the vehicle is a straight line, then the X-axis coordinate of the left front wheel corresponds to the longitudinal position of the protrusion or the pit on the road, and the Y-axis coordinate of the left front wheel corresponds to the lateral position of the protrusion or the pit relative to the vehicle's center line. In this way, the location of the protrusion or the pit can be accurately located, avoiding the vehicle running over the same obstacle again later.

[0054] In one embodiment of the present application, a plurality of the above-mentioned inertial navigation systems are installed at any of the above-mentioned positions, and receiving the posture data detected by the inertial navigation systems installed at four positions of the unmanned vehicle in the open-pit mine includes: receiving the posture data detected by the plurality of the above-mentioned inertial navigation systems installed at any of the above-mentioned positions of the unmanned vehicle in the open-pit mine; after receiving the posture data detected by the inertial navigation systems installed at four positions of the unmanned vehicle in the open-pit mine, the method further includes: determining the average value of the posture data detected by the plurality of the above-mentioned inertial navigation systems installed at any of the above-mentioned positions as the final posture data detected at the above-mentioned positions.

[0055] Specifically, in order to improve the accuracy and reliability of perception, multiple inertial navigation systems can be installed at the position corresponding to each wheel. When the vehicle is driving, all inertial navigation systems are continuously detecting the vehicle's posture data. For the inertial navigation systems at each position, the data detected will be collected and averaged to obtain more accurate and stable posture data. Finally, the average posture data of multiple inertial navigation systems at each position will be determined as the final posture data of the position for subsequent obstacle perception calculations. For example, for the four positions of the vehicle, three inertial navigation systems can be installed at each position, and the three inertial navigation systems at the left front position are marked as A1, A2 and A3. When the vehicle is driving, A1, A2 and A3 will independently detect and record the posture data, calculate the average of the posture data measured by A1, A2 and A3, and use the average as the final posture data of the left front position. The calculation method of the final posture data of the other three positions is the same, which will not be repeated here.

[0056] In one embodiment of the present application, the above method also includes: storing the position information and size information of the above protrusions or the above pits; sharing the position information and size information of the above protrusions and the above pits to other vehicles through vehicle-to-vehicle communication technology V2V or vehicle-to-network communication technology V2N.

[0057] Specifically, the position information and size information of the detected protrusions or pits are stored. This step establishes a real-time database of obstacles. This database is used to record the position information and size information of obstacles, so that the position information and size information of obstacles can be shared with other vehicles later, so that other vehicles can avoid encountering obstacles before reaching the location through the shared information.

[0058] Specifically, V2V communication technology allows vehicles to exchange data directly without going through a central server, which greatly improves the efficiency and real-time nature of information transmission. V2N communication is the communication between vehicles and the central network or cloud platform of the mine area, which transmits information to a wider range of vehicles and systems through the network. This information sharing mechanism ensures that all vehicles can promptly understand the information of possible obstacles in the mine area, so as to plan routes in advance, avoid collisions and damage, and improve the safety and efficiency of the overall operation.

[0059] More specifically, vehicle-to-vehicle communication (V2V) communication usually uses technical standards such as dedicated short range communication (DSRC) or cellular vehicle-to-everything (C-V2X). DSRC is a wireless communication technology based on IEEE 802.11p, operating in the 5.9GHz frequency band, capable of providing high-speed, low-latency communication services, suitable for real-time traffic information exchange. C-V2X is a communication standard based on cellular network technology (such as 4G LTE or 5G NR). It uses the existing cellular network infrastructure to provide a wider coverage and better communication quality, while also supporting low-latency and highly reliable communication. Through V2V communication, vehicles can share information such as obstacles and road conditions they perceive, so that each vehicle can obtain more comprehensive environmental information, thereby improving the overall performance and safety of the autonomous driving system. Vehicle-to-everything via network (V2N) refers to wireless communication between vehicles and network infrastructure. V2N communication technology mainly relies on cellular networks, such as 4G and 5G, to achieve long-distance, high-speed data transmission. With the development of 5G technology, V2N communication latency will be further reduced and data transmission speed will be faster, which will greatly enhance the intelligent decision-making ability and safety of vehicles in complex traffic environments. In the field of autonomous driving, V2N communication technology enables vehicles to exchange information with road infrastructure, cloud servers, etc., which plays an important role in building a more intelligent, safe and efficient road traffic system.

[0060] In one embodiment of the present application, the installation position of the inertial navigation system in any of the above-mentioned directions can be adjusted within a preset position range, and the inertial navigation systems installed in the four above-mentioned directions are arranged axially symmetrically.

[0061] Specifically, the installation position of the inertial navigation system allows adjustment within a preset position range, that is, the inertial navigation system is not limited to a fixed position, but can be moved within a certain range according to specific needs or vehicle design, which optimizes the layout of the perception system, and the preset position range can be adjusted according to the specific design of the vehicle and the obstacle detection requirements. In addition, the inertial navigation systems installed in the four directions are arranged in an axisymmetric manner, such as Figure 3 As shown in the figure, the inertial navigation systems on both sides of the vehicle are symmetrical in position about the vehicle's central axis. For example, if an inertial navigation system is installed on the left front wheel of the vehicle, then a symmetrical inertial navigation system should also be installed on the right front wheel of the vehicle to maintain symmetry. The same principle applies to the installation positions of the inertial navigation systems on the left and right rear wheels to ensure that the system's perception ability can be balanced in all directions of the vehicle, improving overall accuracy and reliability.

[0062] In order to enable those skilled in the art to more clearly understand the technical solution of the present application, the implementation process of the obstacle detection method for unmanned vehicles in open-pit mines of the present application will be described in detail below in combination with specific embodiments.

[0063] This embodiment relates to a specific method for detecting obstacles in an open-pit mine by an unmanned vehicle. The overall flow chart of the method is as follows: Figure 4 As shown, the following steps are included:

[0064] S1: Obtain the position and posture data detected by the inertial navigation system in four directions.

[0065] S2: Calculate the deviation of the posture data detected at any two orientations.

[0066] S3: Determine whether the deviation of the posture data detected in any two directions is greater than a preset deviation.

[0067] S4: If the deviation of the posture data detected in any two directions is less than or equal to the preset deviation, jump to step S1.

[0068] S5: If the deviation of the posture data detected at any two directions is greater than a preset deviation, it is determined that at least one of the wheels corresponding to the any two directions has rolled over a protrusion or a pit.

[0069] S6: Determine the wheel with abnormal height according to the wheel height data.

[0070] S7: Based on the posture data of the highly abnormal wheel, determine the position information and size information of the protrusion or the pit.

[0071] Specifically, at least one inertial navigation system is installed in four directions corresponding to the four wheels of the vehicle. The four directions refer to the front left, front right, rear left and rear right of the vehicle. At least one inertial navigation system is installed in each direction, that is, at least four inertial navigation systems are distributed in the four corners of the vehicle, close to each wheel respectively, and can be installed in the front left, front right, rear left and rear right of the vehicle chassis. Such a layout can more comprehensively monitor the posture changes of the vehicle during driving, especially when the wheels encounter obstacles, such as fallen rocks or potholes on the road. The inertial navigation system can capture the tiny posture changes when the wheels contact the ground. The inertial navigation system is also called an inertial navigation system. The inertial navigation system can provide the vehicle's positioning, speed and posture information, including three-dimensional position, three-dimensional speed and angular velocity in three directions.

[0072] When the vehicle passes through a certain area, when the deviation between the posture data detected by the inertial navigation system in any two directions exceeds the preset deviation, this indicates that the vehicle may have encountered an uneven road surface, that is, a wheel may have run over a bump or a pit. Once it is detected that the posture data deviation exceeds the preset deviation, further analysis will be performed to determine which wheel or wheels have encountered the obstacle. If the height data of a certain wheel is significantly different from the height data of other wheels, that is, it is considered to be a wheel with abnormal height data, then this wheel is considered to be a highly abnormal wheel, that is, the wheel has run over an obstacle. Through the above steps, the wheel that has run over a bump or a pit, that is, the wheel with abnormal height, can be determined, which is the key to subsequently locating the specific position of the obstacle and determining the size of the obstacle.

[0073] Through the position and posture data of the wheel with abnormal height, the position information of the obstacle at the position of the wheel can be determined, that is, the specific position of the protrusion or the pit can be determined. At the same time, the height data of the wheel with abnormal height can be determined. The size information of the obstacle can be determined. For a protrusion, the height data of the wheel with abnormal height corresponds to the height of the protrusion. For a pit, the height data of the wheel with abnormal height corresponds to the depth of the pit.

[0074] In an autonomous vehicle, there are usually four wheels, and an inertial navigation system is installed near each wheel to monitor the relative height between the wheel and the ground. When the vehicle is driving, if the height of a wheel center from the ground suddenly changes, and this change exceeds a preset threshold, that is, the first preset height difference mentioned above, and at the same time, if the height change of other wheels from the ground center is not so large, then it is considered that the wheel with a larger change has encountered a bump or a pit. This is because under normal driving conditions, the height difference between all wheels and the ground should be relatively stable and similar. If the height difference of one of the wheels suddenly increases or decreases, and the height difference of other wheels does not undergo similar changes, it indicates that the wheel has run over a bump or driven into a pit. For example, when the left front wheel runs over a fallen rock, the height of the left front wheel will suddenly rise, resulting in a significant difference in height with the other three wheels. At this time, if the difference in height between the left front wheel and the ground and the other three wheels exceeds a first preset height difference, the left front wheel will be identified as an abnormally high wheel, that is, the left front wheel has run over an obstacle. The first preset height difference can be set to a few centimeters or tens of centimeters, and the specific value can be determined based on the size of the obstacle and the safety needs of the vehicle.

[0075] The above-mentioned reference height data refers to the relative height between the wheels and the ground when the vehicle does not encounter any obstacles, that is, when it is driving on a flat road. When the vehicle encounters a bump or a pothole, the distance between the wheel that runs over the bump or enters the pothole and the ground will suddenly change. If the difference between two or more wheels and the reference height exceeds the second preset height difference, it is considered that these wheels have run over an obstacle. For example, when the vehicle runs into a larger pothole or runs over multiple obstacles at the same time, the height data of more than one wheel will change significantly. When the two front wheels of the vehicle run over fallen rocks at the same time, the height of these two wheels will be much higher than the rear wheels and the reference height data. In this case, if the difference between the height data of the two front wheels and the corresponding reference height data is greater than the second preset height difference, the two front wheels are considered to be wheels with abnormal heights. For another example, when the left front wheel and two rear wheels of a vehicle enter a pothole at the same time, the height of these three wheels will be much lower than the right front wheel and the reference height data. In this case, if the difference between the height data of the left front wheel and the two rear wheels and the corresponding reference height data is greater than the second preset height difference, the left front wheel and the two rear wheels are considered to be wheels with abnormal heights. Among them, the second preset height difference can be determined according to the size of the obstacle and the safety needs of the vehicle. The reference height data refers to the standard distance between the wheel and the ground when the vehicle does not encounter any obstacles. It is determined by the historical wheel height data when the vehicle is driving on a flat road. The reference height data provides a reference point, which enables the vehicle to compare with this reference value based on the real-time height change data, so as to determine whether the vehicle has encountered an obstacle on the road.

[0076] Obtain the height differences between the abnormal wheel and all normal wheels, calculate the average value of these differences, and take the calculated average value as the height of the protrusion or the depth of the depression. This is because when the vehicle runs over an obstacle, the height of the abnormal wheel will be significantly different from that of other normal wheels, and the difference reflects the height of the protrusion or the depth of the depression. For example, assume that the right front wheel runs over a falling rock, record the height of the right front wheel relative to the ground as H1, and the left front wheel, left rear wheel, and right rear wheel do not encounter obstacles. Record the heights of the left front wheel, left rear wheel, and right rear wheel relative to the ground as H2, H3, and H4 respectively. Then calculate the height difference between the right front wheel and the left front wheel ΔH1 = H1 - H2, the height difference between the right front wheel and the left rear wheel ΔH2 = H1 - H3, and the height difference between the right front wheel and the right rear wheel ΔH3 = H1 - H4. Then, average these differences to obtain the above-mentioned first average value, that is, the above-mentioned first average value is (ΔH1 + ΔH2 + ΔH3) / 3. The above-mentioned first average value reflects the height of the falling rock encountered by the right front wheel. Therefore, this average value can be regarded as the height of the falling rock.

[0077] Compare the height of the wheel that encounters the obstacle with the reference height data to obtain a difference. At the same time, if other wheels also encounter obstacles, also obtain their differences from the reference height data. Then, average these differences to obtain the above-mentioned second average value. Determine the size of the obstacle according to the obtained second average value. If the second average value is positive, it indicates that the wheel has encountered a protrusion, and the magnitude of the second average value is the height of the protrusion. On the contrary, if the second average value is negative, it indicates that the wheel has encountered a depression, and the absolute value of the second average value is the depth of the depression. For example, when the vehicle is running normally, the reference height data is h, and at this time, the height data of all four wheels is h. When the height of the left front wheel becomes H5 and the height of the right front wheel becomes H6, if H5 < h and H6 < h, it indicates that the heights of the left front wheel and the right front wheel have both become lower, indicating that the left front wheel and the right front wheel have entered a depression. Calculate the difference between the height of the left front wheel and the reference height data ΔH4 = h - H5, and the difference between the height of the right front wheel and the reference height data ΔH5 = h - H6. Then calculate the average value of the difference ΔH4 and the difference ΔH5 as (ΔH4 + ΔH5) / 2, and this average value (ΔH4 + ΔH5) / 2 is the depth of the above-mentioned depression.

[0078] First, extract the corresponding first horizontal coordinate and second horizontal coordinate from the posture data provided by the inertial navigation system associated with the wheel with detected height abnormality. The first horizontal coordinate usually refers to the X-axis coordinate, and the second horizontal coordinate usually refers to the Y-axis coordinate. These coordinate information reflects the position of the wheel at the moment of contact with the obstacle. Then, using the extracted first horizontal coordinate and second horizontal coordinate, the exact position of the protrusion or pit on the ground can be determined. For example, if the vehicle's driving trajectory is a straight line, the X-axis coordinate of the left front wheel will correspond to the longitudinal position of the protrusion or pit on the road, and the Y-axis coordinate of the left front wheel will correspond to the lateral position of the protrusion or pit relative to the center line of the vehicle. In this way, the location of the protrusion or pit can be accurately located to prevent subsequent vehicles from running over the same obstacle again.

[0079] In order to improve the accuracy and reliability of perception, multiple inertial navigation systems can be installed at the position corresponding to each wheel. When the vehicle is driving, all inertial navigation systems are continuously detecting the vehicle's posture data. For the inertial navigation systems at each position, the data detected will be collected and the average value will be calculated to obtain more accurate and stable posture data. Finally, the average posture data of multiple inertial navigation systems at each position will be determined as the final posture data of the position for subsequent obstacle perception calculations. For example, for the four positions of the vehicle, three inertial navigation systems can be installed at each position, and the three inertial navigation systems at the left front position are marked as A1, A2 and A3. When the vehicle is driving, A1, A2 and A3 will independently detect and record the posture data, calculate the average value of the posture data measured by A1, A2 and A3, and use the average value as the final posture data of the left front position. The calculation method of the final posture data of the other three positions is the same, which will not be repeated here.

[0080] The position and size information of the detected protrusions or pits are stored. This step establishes a real-time database of obstacles. This database is used to record the position and size information of obstacles so that the position and size information of obstacles can be shared with other vehicles later, so that other vehicles can avoid encountering obstacles before reaching the location through the shared information.

[0081] V2V communication technology allows vehicles to exchange data directly without going through a central server, which greatly improves the efficiency and real-time nature of information transmission. V2N communication is the communication between vehicles and the central network or cloud platform of the mine area, which transmits information to a wider range of vehicles and systems through the network. This information sharing mechanism ensures that all vehicles can timely understand the information of possible obstacles in the mine area, so as to plan routes in advance, avoid collisions and damage, and improve the safety and efficiency of the overall operation.

[0082] Vehicle-to-Vehicle Communication (V2V) communication usually uses technical standards such as Dedicated Short Range Communications (DSRC) or Cellular V2X (C-V2X). DSRC is a wireless communication technology based on IEEE 802.11p, operating in the 5.9GHz frequency band, which can provide high-speed, low-latency communication services and is suitable for real-time traffic information exchange. C-V2X is a communication standard based on cellular network technology (such as 4G LTE or 5G NR). It uses the existing cellular network infrastructure to provide a wider coverage and better communication quality, while also supporting low-latency and highly reliable communication. Through V2V communication, vehicles can share information such as obstacles and road conditions they perceive, so that each vehicle can obtain more comprehensive environmental information, thereby improving the overall performance and safety of the autonomous driving system. Vehicle-to-Everything via Network (V2N) refers to wireless communication between vehicles and network infrastructure. V2N communication technology mainly relies on cellular networks, such as 4G and 5G, to achieve long-distance, high-speed data transmission. With the development of 5G technology, V2N communication latency will be further reduced and data transmission speed will be faster, which will greatly enhance the intelligent decision-making ability and safety of vehicles in complex traffic environments. In the field of autonomous driving, V2N communication technology enables vehicles to exchange information with road infrastructure, cloud servers, etc., which plays an important role in building a more intelligent, safe and efficient road traffic system.

[0083] The installation position of the inertial navigation system allows adjustment within a preset position range, that is, the inertial navigation system is not limited to a fixed position, but can be moved within a certain range according to specific needs or vehicle design, which optimizes the layout of the perception system. The preset position range can be adjusted according to the specific design of the vehicle and the obstacle detection needs. In addition, the inertial navigation systems installed in the four directions are arranged axially symmetrically, that is, the inertial navigation systems on both sides of the vehicle are symmetrical in position. For example, if an inertial navigation system is installed in the left front wheel position of the vehicle, then a symmetrical inertial navigation system should also be installed in the right front wheel position of the vehicle to maintain symmetry. The same principle applies to the installation positions of the inertial navigation systems in the left and right rear wheel positions to ensure that the system's perception capabilities can be balanced in all directions of the vehicle and improve overall accuracy and reliability.

[0084] The embodiment of the present application also provides an unmanned vehicle obstacle detection device for an open-pit mine. It should be noted that the unmanned vehicle obstacle detection device for an open-pit mine in the embodiment of the present application can be used to execute the unmanned vehicle obstacle detection method for an open-pit mine provided in the embodiment of the present application. The device is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions thereof will not be repeated here. As used below, the term "module" may be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation in hardware, or a combination of software and hardware, is also possible and conceivable.

[0085] The following is an introduction to the obstacle detection device for unmanned vehicles in open-pit mines provided in an embodiment of the present application.

[0086] Figure 5 Schematic diagram of an obstacle detection device for unmanned vehicles in open-pit mines according to an embodiment of the present application. Figure 5 As shown, the device comprises:

[0087] The receiving unit 10 is used to receive the position and posture data detected by the inertial navigation system installed at four positions of the unmanned vehicle in the open-pit mine, wherein the four positions correspond to four wheels, and at least one of the inertial navigation systems is installed at any of the four positions;

[0088] The above-mentioned open-pit mine unmanned vehicles are wide-body vehicles, such as Figure 6 As shown, it includes wheels, box and front end.

[0089] Specifically, at least one inertial navigation system is installed in four positions corresponding to the four wheels of the vehicle. The four positions refer to the front left, front right, rear left and rear right of the vehicle. At least one inertial navigation system is installed in each position, that is, at least four inertial navigation systems are distributed in the four corners of the vehicle, close to each wheel respectively, and can be installed in the front left, front right, rear left and rear right of the vehicle chassis. Such a layout can more comprehensively monitor the posture changes of the vehicle during driving, especially when the wheels encounter obstacles, such as fallen rocks or potholes on the road. The inertial navigation system can capture the subtle posture changes when the wheels contact the ground.

[0090] Specifically, the inertial navigation system is also called an inertial navigation system. The inertial navigation system can provide the vehicle's positioning, speed and attitude information, including three-dimensional position, three-dimensional speed and angular velocity in three directions.

[0091] The first determination unit 20 is used to determine that at least one of the wheels corresponding to any two of the above-mentioned directions has rolled over a protrusion or a pit when the deviation of the above-mentioned posture data detected at any two of the above-mentioned directions is greater than a preset deviation, and determine the height-abnormal wheel according to the height data of the above-mentioned wheels, wherein the deviation of the above-mentioned posture data includes a position deviation and an angle deviation, the above-mentioned height data is obtained from the above-mentioned posture data, and the above-mentioned height-abnormal wheel is a wheel with abnormal height data;

[0092] Specifically, when the vehicle passes through a certain area, when the deviation between the posture data detected by the inertial navigation system in any two directions exceeds the preset deviation, this indicates that the vehicle may have encountered an uneven road surface, that is, a wheel may have run over a bump or a pit. Once it is detected that the posture data deviation exceeds the preset deviation, further analysis will be performed to determine which wheel or wheels have encountered an obstacle. If the height data of a certain wheel is significantly different from the height data of other wheels, that is, it is considered to be a wheel with abnormal height data, then this wheel is considered to be a highly abnormal wheel, that is, the wheel has run over an obstacle. Through the above steps, the wheel that has run over a bump or a pit, that is, the highly abnormal wheel, can be determined, which is the key to the subsequent positioning of the specific position of the obstacle and determining the size of the obstacle.

[0093] The second determination unit 30 is used to determine the position information and size information of the protrusion or the pit based on the posture data of the wheel with abnormal height.

[0094] Specifically, the position data of the wheel with abnormal height and the posture data can be used to determine the position information of the obstacle at the wheel location, that is, the specific position of the protrusion or the pit. At the same time, the height data of the wheel with abnormal height can be used to determine the size information of the obstacle. For a protrusion, the height data of the wheel with abnormal height corresponds to the height of the protrusion. For a pit, the height data of the wheel with abnormal height corresponds to the depth of the pit.

[0095] In the above embodiment, the posture data detected by the inertial navigation system installed at the four positions of the unmanned vehicle in the open-pit mine is received, wherein the four positions correspond to four wheels, and at least one of the above inertial navigation systems is installed at any of the above positions; when the deviation of the above posture data detected at any two of the above positions is greater than the preset deviation, it is determined that at least one of the wheels corresponding to any two of the above positions has rolled over a protrusion or a pit, and the height-abnormal wheel is determined according to the height data of the above wheel, wherein the deviation of the above posture data includes position deviation and angle deviation, the above height data is obtained from the above posture data, and the above-mentioned height-abnormal wheel is a wheel with abnormal height data; based on the posture data of the above-mentioned height-abnormal wheel, the position information and size information of the above-mentioned protrusion or the above-mentioned pit are determined. In this scheme, by installing the inertial navigation system at the four positions of the unmanned vehicle in the open-pit mine, the position and height information of the abnormal wheel are determined, and then the position information and size information of the obstacle are determined, thereby solving the problem of limited perception ability of the obstacle detection method of the unmanned vehicle in the open-pit mine in the prior art.

[0096] In one embodiment of the present application, the first determining unit includes:

[0097] A first determination module is used to determine that the first target wheel is the wheel with abnormal height when the difference between the height of the first target wheel from the ground and the height of multiple non-first target wheels from the ground is greater than a first preset height difference, and the first target wheel is one of the four wheels;

[0098] The second determination module is used to determine that at least two of the above-mentioned second target wheels are the above-mentioned height-abnormal wheels when the difference between the above-mentioned height data of at least two second target wheels among the above-mentioned four wheels and the corresponding reference height data is greater than a second preset height difference, the above-mentioned second target wheel is one of the four wheels, and the above-mentioned reference height data is the height data obtained by driving on a flat road.

[0099] Specifically, in an autonomous driving vehicle, there are usually four wheels, and an inertial navigation system is installed near each wheel to monitor the relative height between the wheel and the ground. When the vehicle is driving, if the height of a wheel center from the ground suddenly changes, and this change exceeds a preset threshold, that is, the above-mentioned first preset height difference, and at the same time, if the height change of other wheels centers from the ground is not so large, then it is considered that the wheel with a larger change has encountered a bump or a pit. This is because under normal driving conditions, the height difference between all wheels and the ground should be relatively stable and similar. If the height difference of one of the wheels suddenly increases or decreases, and the height difference of other wheels does not undergo similar changes, it indicates that the wheel has run over a bump or driven into a pit. For example, when the left front wheel runs over a fallen rock, the height of the left front wheel will suddenly rise, resulting in a significant difference in height with the other three wheels. At this time, if the difference in height between the left front wheel and the ground and the other three wheels exceeds a first preset height difference, the left front wheel will be identified as an abnormally high wheel, that is, the left front wheel has run over an obstacle. The first preset height difference can be set to a few centimeters or tens of centimeters, and the specific value can be determined based on the size of the obstacle and the safety needs of the vehicle.

[0100] Specifically, the above-mentioned reference height data refers to the relative height between the wheels and the ground when the vehicle does not encounter any obstacles, that is, when it is driving on a flat road. When the vehicle encounters a protrusion or a pit, the distance between the wheel that rolls over the protrusion or enters the pit and the ground will suddenly change. If the difference between two or more wheels and the reference height exceeds the second preset height difference, it is considered that these wheels have rolled over an obstacle. For example, when the vehicle drives into a larger pit or rolls over multiple obstacles at the same time, the height data of more than one wheel will change significantly. When the two front wheels of the vehicle roll over fallen rocks at the same time, the height of these two wheels will be much higher than the rear wheels and the reference height data. In this case, if the difference between the height data of the two front wheels and the corresponding reference height data is greater than the second preset height difference, it is considered that the two front wheels are wheels with abnormal heights. For another example, when the left front wheel and two rear wheels of a vehicle enter a pothole at the same time, the heights of these three wheels will be much lower than the right front wheel and the reference height data. In this case, if the difference between the height data of the left front wheel and the two rear wheels and the corresponding reference height data is greater than the second preset height difference, the left front wheel and the two rear wheels are considered to be wheels with abnormal heights. The second preset height difference can be determined according to the size of the obstacle and the safety requirements of the vehicle.

[0101] Specifically, the benchmark height data refers to the standard distance between the wheels and the ground when the vehicle does not encounter any obstacles. It is determined by the historical wheel height data of the vehicle when it is driving on a flat road. The benchmark height data provides a reference point, enabling the vehicle to compare the real-time height change data with this benchmark value to determine whether the vehicle has encountered an obstacle on the road.

[0102] In a specific embodiment, the second determining unit includes:

[0103] A third determination module is used to obtain an average value of the difference between the height of the first target wheel and all the non-first target wheels from the ground to obtain a first average value, and determine the first average value as the height of the protrusion or the depth of the pit;

[0104] The fourth determination module is used to obtain the average value of the difference between at least two of the second target wheels and the corresponding reference height data to obtain a second average value, and determine the second average value as the height of the protrusion or the depth of the pit.

[0105] Specifically, the height difference between the abnormal wheel and all normal wheels is obtained, the average value of these differences is calculated, and the calculated average value is used as the height of the protrusion or the depth of the pit. This is because when the vehicle runs over an obstacle, the height of the abnormal wheel will be significantly different from that of other normal wheels, and the difference reflects the height of the protrusion or the depth of the pit. For example, assuming that the right front wheel runs over a fallen rock, the height of the right front wheel relative to the ground is recorded as H1, and the left front wheel, left rear wheel and right rear wheel do not encounter obstacles. The heights of the left front wheel, left rear wheel and right rear wheel relative to the ground are recorded as H2, H3 and H4 respectively, and then the height difference between the right front wheel and the left front wheel ΔH1=H1-H2, the height difference between the right front wheel and the left rear wheel ΔH2=H1-H3, and the height difference between the right front wheel and the right rear wheel ΔH3=H1-H4 are calculated. Then, these differences are averaged to obtain the above-mentioned first average value, that is, the above-mentioned first average value is (ΔH1+ΔH2+ΔH3) / 3. The first average value reflects the height of the fallen rock encountered by the right front wheel. Therefore, this average value can be regarded as the height of the fallen rock.

[0106] Specifically, the height of the wheel encountering an obstacle is compared with the reference height data to obtain a difference. At the same time, if other wheels also encounter an obstacle, the difference between their heights and the reference height data is also obtained. Then, these differences are averaged to obtain the above-mentioned second average value. Based on the obtained second average value, the size of the obstacle is determined. If the second average value is positive, it indicates that the wheel has encountered a protrusion, and the magnitude of the second average value is the height of the protrusion. On the contrary, if the second average value is negative, it indicates that the wheel has encountered a pit, and the absolute value of the second average value is the depth of the pit. For example, when the vehicle is driving normally, the reference height data is h, and the height data of all four wheels is h at this time. When the height of the left front wheel becomes H5 and the height of the right front wheel becomes H6, if H5 < h and H6 < h, it indicates that the heights of the left front wheel and the right front wheel have both decreased, which means that the left front wheel and the right front wheel have entered a pit. Calculate the difference between the height of the left front wheel and the reference height data ΔH4 = h - H5, and the difference between the height of the right front wheel and the reference height data ΔH5 = h - H6. Then calculate the average value of the difference ΔH4 and the difference ΔH5 as (ΔH4 + ΔH5) / 2, and this average value (ΔH4 + ΔH5) / 2 is the depth of the depression of the above-mentioned pit.

[0107] In an embodiment of the present application, the above-mentioned second determination unit includes:

[0108] An extraction module, configured to extract a first horizontal coordinate and a second horizontal coordinate from the pose data of the above-mentioned wheel with abnormal height;

[0109] A fifth determination module, configured to determine the position information of the above-mentioned protrusion or the above-mentioned pit according to the above-mentioned first horizontal coordinate and the above-mentioned second horizontal coordinate.

[0110] Specifically, first, extract the corresponding first horizontal coordinate and second horizontal coordinate from the pose data provided by the inertial navigation system associated with the wheel detected with abnormal height. The first horizontal coordinate usually refers to the X-axis coordinate, and the second horizontal coordinate usually refers to the Y-axis coordinate. These coordinate information reflect the position of the wheel at the moment of contacting the obstacle. Then, using the extracted first horizontal coordinate and second horizontal coordinate, the exact position of the protrusion or pit on the ground can be determined. For example, if the driving trajectory of the vehicle is a straight line, then the X-axis coordinate of the left front wheel corresponds to the longitudinal position of the protrusion or pit on the road, and the Y-axis coordinate of the left front wheel corresponds to the transverse position of the protrusion or pit relative to the vehicle's center line. In this way, the location of the protrusion or pit can be accurately located to avoid the vehicle running over the same obstacle again later.

[0111] In one embodiment of the present application, a plurality of the above-mentioned inertial navigation systems are installed at any of the above-mentioned positions, and the posture data detected by the inertial navigation systems installed at four positions of the unmanned vehicle in the open-pit mine are received, including: receiving the posture data detected by the plurality of the above-mentioned inertial navigation systems installed at any of the above-mentioned positions of the unmanned vehicle in the open-pit mine; after receiving the posture data detected by the inertial navigation systems installed at four positions of the unmanned vehicle in the open-pit mine, the above-mentioned device also includes: a third determination unit, which is used to determine the average value of the posture data detected by the plurality of the above-mentioned inertial navigation systems installed at any of the above-mentioned positions as the final posture data detected at the above-mentioned positions.

[0112] Specifically, in order to improve the accuracy and reliability of perception, multiple inertial navigation systems can be installed at the position corresponding to each wheel. When the vehicle is driving, all inertial navigation systems are continuously detecting the vehicle's posture data. For the inertial navigation systems at each position, the data detected will be collected and averaged to obtain more accurate and stable posture data. Finally, the average posture data of multiple inertial navigation systems at each position will be determined as the final posture data of the position for subsequent obstacle perception calculations. For example, for the four positions of the vehicle, three inertial navigation systems can be installed at each position, and the three inertial navigation systems at the left front position are marked as A1, A2 and A3. When the vehicle is driving, A1, A2 and A3 will independently detect and record the posture data, calculate the average of the posture data measured by A1, A2 and A3, and use the average as the final posture data of the left front position. The calculation method of the final posture data of the other three positions is the same, which will not be repeated here.

[0113] In one embodiment of the present application, the above-mentioned device further includes:

[0114] A storage unit, used to store the position information and size information of the protrusion or the pit;

[0115] The sharing unit includes sharing the position information and size information of the protrusions and the pits with other vehicles through vehicle-to-vehicle communication technology V2V or vehicle-to-network communication technology V2N.

[0116] Specifically, the position information and size information of the detected protrusions or pits are stored. This step establishes a real-time database of obstacles. This database is used to record the position information and size information of obstacles, so that the position information and size information of obstacles can be shared with other vehicles later, so that other vehicles can avoid encountering obstacles before reaching the location through the shared information.

[0117] Specifically, V2V communication technology allows vehicles to exchange data directly without going through a central server, which greatly improves the efficiency and real-time nature of information transmission. V2N communication is the communication between vehicles and the central network or cloud platform of the mine area, which transmits information to a wider range of vehicles and systems through the network. This information sharing mechanism ensures that all vehicles can promptly understand the information of possible obstacles in the mine area, so as to plan routes in advance, avoid collisions and damage, and improve the safety and efficiency of the overall operation.

[0118] More specifically, vehicle-to-vehicle communication (V2V) communication usually uses technical standards such as dedicated short range communication (DSRC) or cellular vehicle-to-everything (C-V2X). DSRC is a wireless communication technology based on IEEE 802.11p, operating in the 5.9GHz frequency band, capable of providing high-speed, low-latency communication services, suitable for real-time traffic information exchange. C-V2X is a communication standard based on cellular network technology (such as 4G LTE or 5G NR). It uses the existing cellular network infrastructure to provide a wider coverage and better communication quality, while also supporting low-latency and highly reliable communication. Through V2V communication, vehicles can share information such as obstacles and road conditions they perceive, so that each vehicle can obtain more comprehensive environmental information, thereby improving the overall performance and safety of the autonomous driving system. Vehicle-to-everything via network (V2N) refers to wireless communication between vehicles and network infrastructure. V2N communication technology mainly relies on cellular networks, such as 4G and 5G, to achieve long-distance, high-speed data transmission. With the development of 5G technology, V2N communication latency will be further reduced and data transmission speed will be faster, which will greatly enhance the intelligent decision-making ability and safety of vehicles in complex traffic environments. In the field of autonomous driving, V2N communication technology enables vehicles to exchange information with road infrastructure, cloud servers, etc., which plays an important role in building a more intelligent, safe and efficient road traffic system.

[0119] In one embodiment of the present application, the installation position of the inertial navigation system in any of the above-mentioned directions can be adjusted within a preset position range, and the inertial navigation systems installed in the four above-mentioned directions are arranged axially symmetrically.

[0120] Specifically, the installation position of the inertial navigation system allows adjustment within a preset position range, that is, the inertial navigation system is not limited to a fixed position, but can be moved within a certain range according to specific needs or vehicle design, which optimizes the layout of the perception system. The preset position range can be adjusted according to the specific design of the vehicle and the obstacle detection needs. In addition, the inertial navigation systems installed in the four directions are arranged axially symmetrically, that is, the inertial navigation systems on both sides of the vehicle are symmetrical in position. For example, if an inertial navigation system is installed in the left front wheel position of the vehicle, then a symmetrically positioned inertial navigation system should also be installed in the right front wheel position of the vehicle to maintain symmetry. The same principle applies to the installation positions of the inertial navigation systems in the left rear wheel and right rear wheel positions to ensure that the system's perception capabilities can be balanced in all directions of the vehicle and improve overall accuracy and reliability.

[0121] The above-mentioned obstacle detection device for unmanned vehicles in open-pit mines includes a processor and a memory. The above-mentioned receiving unit, the first determination unit, the second determination unit, etc. are all stored in the memory as program units, and the processor executes the above-mentioned program units stored in the memory to realize corresponding functions. The above-mentioned modules are all located in the same processor; or, the above-mentioned modules are located in different processors in the form of any combination.

[0122] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0123] An embodiment of the present invention provides a computer-readable storage medium, which includes a stored program. When the program is running, the device where the computer-readable storage medium is located is controlled to execute the open-pit mine unmanned vehicle obstacle detection method.

[0124] An embodiment of the present invention provides an open-pit mine unmanned vehicle, including a controller, and the controller executes the steps of the open-pit mine unmanned vehicle obstacle detection method.

[0125] An embodiment of the present invention provides a device, which includes a processor, a memory, and a program stored in the memory and executable on the processor. When the processor executes the program, the steps of the above-mentioned open-pit mine unmanned vehicle obstacle detection method are implemented.

[0126] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0127] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0128] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0129] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0130] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0131] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0132] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0133] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0134] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0135] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for detecting obstacles in open-pit mines by unmanned vehicles, characterized in that: include: Receiving position data detected by an inertial navigation system installed at four positions of an unmanned vehicle in an open-pit mine, wherein the four positions correspond to four wheels, and at least one of the inertial navigation systems is installed at any of the positions; When the deviation of the posture data detected at any two of the orientations is greater than a preset deviation, it is determined that at least one of the wheels corresponding to any two of the orientations has rolled over a protrusion or a pit, and a wheel with abnormal height is determined according to the height data of the wheel, wherein the deviation of the posture data includes a position deviation and an angle deviation, the height data is obtained from the posture data, and the wheel with abnormal height is a wheel with abnormal height data; Based on the position data of the wheel with abnormal height, the position information and size information of the protrusion or the pit are determined.

2. The method according to claim 1, characterized in that Determining a wheel with abnormal height according to the height data of the wheel includes: In the case where the difference between the height of the first target wheel from the ground and the height of multiple non-first target wheels from the ground is greater than a first preset height difference, determining that the first target wheel is the wheel with abnormal height, and the first target wheel is one of the four wheels; When the difference between the height data of at least two second target wheels among the four wheels and the corresponding reference height data is greater than the second preset height difference, at least two of the second target wheels are determined to be the height-abnormal wheels, the second target wheel is one of the four wheels, and the reference height data is the height data obtained by driving on a flat road.

3. The method according to claim 2, characterized in that Determining the size information of the protrusion or the pit based on the posture data of the wheel with abnormal height includes: Obtaining an average value of the difference between the heights of the first target wheel and all the non-first target wheels from the ground to obtain a first average value, and determining the first average value as the height of the protrusion or the depth of the pit; The average value of the difference between at least two of the second target wheels and the corresponding reference height data is obtained to obtain a second average value, and the second average value is determined as the height of the protrusion or the depth of the pit.

4. The method according to claim 1, characterized in that: Determining the position information of the protrusion or the pit based on the posture data of the wheel with abnormal height includes: Extracting a first horizontal coordinate and a second horizontal coordinate from the position data of the wheel with abnormal height; The position information of the protrusion or the pit is determined according to the first horizontal coordinate and the second horizontal coordinate.

5. The method according to claim 1, characterized in that A plurality of said inertial navigation systems are installed in any said position, Receive the position data detected by the inertial navigation system installed on the four positions of the unmanned vehicle in the open-pit mine, including: Receiving position and posture data detected by a plurality of inertial navigation systems installed at any of the positions of the unmanned vehicle in the open-pit mine; After receiving the posture data detected by the inertial navigation systems installed at four positions of the unmanned vehicle in the open-pit mine, the method also includes: determining the average value of the posture data detected by multiple inertial navigation systems installed at any of the positions as the final posture data detected at the position.

6. The method according to claim 1, characterized in that The method further comprises: storing the position information and size information of the protrusion or the pit; The position information and size information of the protrusion and the pit are shared with other vehicles through vehicle-to-vehicle communication technology V2V or vehicle-to-network communication technology V2N.

7. The method according to any one of claims 1 to 6, characterized in that The installation position of the inertial navigation system in any of the directions can be adjusted within a preset position range, and the inertial navigation systems installed in the four directions are arranged in an axisymmetric manner.

8. An obstacle detection device for unmanned vehicles in open-pit mines, characterized in that: include: A receiving unit, used to receive position data detected by an inertial navigation system installed at four positions of an unmanned vehicle in an open-pit mine, wherein the four positions correspond to four wheels, and at least one inertial navigation system is installed at any of the positions; a first determining unit, configured to determine that at least one of the wheels corresponding to any two of the directions has rolled over a protrusion or a pit, and to determine a wheel with abnormal height according to the height data of the wheel, when the deviation of the posture data detected at any two of the directions is greater than a preset deviation, wherein the deviation of the posture data includes a position deviation and an angle deviation, the height data is obtained from the posture data, and the wheel with abnormal height is a wheel with abnormal height data; The second determination unit is used to determine the position information and size information of the protrusion or the pit based on the posture data of the wheel with abnormal height.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the obstacle detection method for unmanned vehicles in open-pit mines as described in any one of claims 1 to 7.

10. An unmanned vehicle for an open-pit mine, characterized in that: It includes a controller, which executes the obstacle detection method for unmanned vehicles in open-pit mines as described in any one of claims 1 to 7.

Images