Method and device for identifying obstacle of mine car, computer equipment and storage medium

Abstract

The invention relates to the technical field of automatic control, and discloses an identification method and device for an obstacle of a mine car, computer equipment and a storage medium, the method is applied to a carrier, and the method comprises the following steps: judging whether an abnormal area exists in a collection area of a thermal imaging camera or not through thermal imaging data of the thermal imaging camera; if the abnormal area is located in the scanning area of the laser radar, a target area is divided in the scanning area, and the target area comprises the abnormal area; comparing the point cloud data of the laser radar on the abnormal area with the thermal imaging data of the thermal imaging camera on the abnormal area, and judging whether dust fog interference exists between the mine card and the abnormal area; and if the dust fog interference exists, identifying the abnormal region based on the thermal imaging data. The mine car can accurately identify other mine cars and people in a mine field, so that the safety of the mine car in the mine field is improved.

Description

Technical Field

[0001] The present invention relates to the field of automatic control, and in particular to a method, a device, a computer device and a storage medium for identifying obstacles of a mine car. Background Art

[0002] Mining dump trucks are the main equipment for automobile transportation in large open-pit mines at home and abroad, responsible for important tasks such as transporting ore, rock and soil. It consists of a front, frame, carriage, wheels, and accompanying power, brake, shock-absorbing and buffering devices, and has the characteristics of heavy weight, heavy load and large size. However, in open-pit mines, collision accidents are prone to occur due to factors such as the large workload of automobile transportation, complex road distribution, and boring driving work. In addition, due to the large size of mining dump trucks, the cab only occupies a small part of the front of the vehicle, forming a large area of ​​visual blind spot, further increasing the risk of accidents.

[0003] In the mining area, due to the uneven level of infrastructure construction, there are many factors that affect driving, such as light and dust, which makes it very difficult for drivers to drive safely. In some cases, it is difficult to see whether there are obstacles, pedestrians or small vehicles on the road ahead. At the same time, trucks, trucks, trailers, dump trucks and various large engineering vehicles have large blind spots. Even experienced drivers cannot guarantee that they will drive safely. Summary of the invention

[0004] In a first aspect, the present application provides a method for identifying obstacles in a mine car, which is applied to a mine car, wherein the mine car is equipped with a thermal imaging camera and a laser radar, and the method comprises: Determining whether there is an abnormal area in the acquisition area of ​​the thermal imaging camera through the thermal imaging data of the thermal imaging camera; If the abnormal area is within the scanning area of ​​the laser radar, a target area is divided within the scanning area, and the target area includes the abnormal area; Compare the point cloud data of the laser radar on the abnormal area with the thermal imaging data of the thermal imaging camera on the abnormal area to determine whether there is dust and fog interference between the mining truck and the abnormal area; If there is dust and fog interference, the abnormal area is identified based on thermal imaging data.

[0005] Furthermore, the “point cloud data of the laser radar on the abnormal area” is obtained by the following steps: Based on whether there is a change in the thermal imaging data of the abnormal area within a unit time, determining whether there is a target object in the abnormal area; If there is a target object in the abnormal area, the point cloud data of the abnormal area is obtained based on the point cloud data on the target area and the thermal imaging data on the target area.

[0006] Furthermore, the “comparing the point cloud data of the laser radar on the abnormal area with the thermal imaging data of the thermal imaging camera on the abnormal area” includes: According to the activity traces of the thermal imaging of the abnormal area within a preset time period, the thermal imaging data changes of the abnormal area are obtained; According to the activity traces of the point cloud data of the abnormal area in the same corresponding time period, the point cloud data changes of the abnormal area are obtained; Compare the changes in thermal imaging data in the abnormal area with the changes in point cloud data in the abnormal area.

[0007] Furthermore, the “determining whether there is dust and fog interference between the mining truck and the abnormal area” includes: If the difference between the change in thermal imaging data in the abnormal area and the change in point cloud data in the abnormal area is less than the preset value, it is determined that there is no dust and fog interference between the mining truck and the abnormal area; If the difference between the change in thermal imaging data in the abnormal area and the change in point cloud data in the abnormal area is not less than the preset value, it is determined that there is dust and fog interference between the mining truck and the abnormal area.

[0008] Furthermore, a reference object is provided on the mining car, and the reference object is provided within the acquisition area of ​​the thermal imaging camera; the “if the abnormal area is within the scanning area of ​​the laser radar” includes: According to the preset distance between the reference object and the thermal imaging camera, in combination with the thermal imaging image data, the actual distance between the abnormal area and the thermal imaging camera is obtained; According to the actual distance between the abnormal area and the thermal imaging camera, it is repeatedly determined whether the abnormal area is within the scanning area of ​​the laser radar.

[0009] Furthermore, the “obtaining the actual distance between the abnormal area and the thermal imaging camera based on the preset distance between the reference object and the thermal imaging camera and combining the thermal imaging image data” includes: According to a preset distance between the reference object and the thermal imaging camera, an actual distance represented by a unit pixel in the current frame of thermal imaging data is obtained, wherein the actual distance represented by a unit pixel in the current frame of thermal imaging data is defined as a first reference distance; Based on the first reference distance and the number of pixels of the abnormal area and the reference object in the current frame of thermal imaging data, the actual distance between the abnormal area and the thermal imaging camera is obtained.

[0010] Furthermore, the “repeatedly determining whether the abnormal area is within the scanning area of ​​the laser radar according to the actual distance between the abnormal area and the thermal imaging camera” includes: According to the actual distance between the abnormal area and the thermal imaging camera and the scanning distance of the laser radar, confirm whether the abnormal area is within the scanning area of ​​the laser radar; If the abnormal area is not within the scanning area of ​​the laser radar, the actual distance between the abnormal area and the visible light camera in each subsequent frame of thermal imaging data is continuously calculated until the abnormal area is within the scanning area of ​​the laser radar.

[0011] Furthermore, the “if there is dust and fog interference, identifying the abnormal area based on thermal imaging data” includes: Dividing the target area into at least two identification sub-areas, wherein the identification sub-areas are distributed along the height of the thermal imaging image frame composed of the thermal imaging data; Overlap the abnormal area with the identification sub-area to obtain the distribution information of the abnormal area in the identification sub-area; Obstacle recognition results are obtained based on the distribution information of the abnormal area in the recognition sub-area.

[0012] Furthermore, the reference object is the laser radar.

[0013] In a second aspect, the present application further provides a device for identifying obstacles in a mine car, which is applied to a mine car, wherein the mine car is equipped with a thermal imaging camera and a laser radar, and the device comprises: A thermal imaging module, used to determine whether there is an abnormal area in the acquisition area of ​​the thermal imaging camera through the thermal imaging data of the thermal imaging camera; A division module, for dividing a target area in the scanning area if the abnormal area is within the scanning area of ​​the laser radar, wherein the target area includes the abnormal area; A judgment module, used to compare the point cloud data of the laser radar on the abnormal area with the thermal imaging data of the thermal imaging camera on the abnormal area, and judge whether there is dust and fog interference between the mining truck and the abnormal area; The recognition module is used to identify abnormal areas based on thermal imaging data if there is dust and fog interference.

[0014] In a third aspect, the present application further provides a terminal device, including a processor and a memory, wherein the memory stores a computer program, and when the computer program runs on the processor, the method for identifying obstacles in a mine car is executed.

[0015] In a fourth aspect, the present application further provides a readable storage medium storing a computer program, wherein the computer program executes the method for identifying obstacles of a mine car when running on a processor.

[0016] The present invention discloses a method, device, computer equipment and storage medium for identifying obstacles of a mine car, which are applied to a vehicle, and the method comprises: judging whether there is an abnormal area in the acquisition area of ​​the thermal imaging camera through the thermal imaging data of the thermal imaging camera; if the abnormal area is in the scanning area of ​​the laser radar, dividing the target area in the scanning area, and the target area includes the abnormal area; comparing the point cloud data of the laser radar on the abnormal area with the thermal imaging data of the thermal imaging camera on the abnormal area, judging whether there is dust and fog interference between the mine truck and the abnormal area; if there is dust and fog interference, identifying the abnormal area based on the thermal imaging data. The mine car can accurately identify other mine cars and people in the mine field, so as to increase the safety of the mine car in the mine field. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solution of the present invention, the following is a brief introduction to the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and should not be regarded as limiting the scope of protection of the present invention. In each of the drawings, similar components are numbered similarly.

[0018] Figure 1 A schematic flow chart of a method for identifying obstacles in a mine car according to an embodiment of the present application is shown; Figure 2 A schematic diagram of a carrier according to an embodiment of the present application is shown; Figure 3 A schematic diagram of identifying sub-regions according to an embodiment of the present application is shown; Figure 4 A schematic structural diagram of an obstacle recognition device for a mine car according to an embodiment of the present application is shown. Implementation

[0019] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0020] The components of the embodiments of the present invention generally described and shown in the drawings herein may be arranged and designed in a variety of different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0021] Hereinafter, the terms "including", "having" and their cognates, which may be used in various embodiments of the present invention, are intended only to indicate specific features, numbers, steps, operations, elements, components or combinations of the foregoing items, and should not be understood as first excluding the existence of one or more other features, numbers, steps, operations, elements, components or combinations of the foregoing items or adding the possibility of one or more features, numbers, steps, operations, elements, components or combinations of the foregoing items.

[0022] Furthermore, the terms “first”, “second”, “third”, etc. are merely used for distinguishing descriptions and are not to be understood as indicating or implying relative importance.

[0023] Unless otherwise defined, all terms (including technical terms and scientific terms) used herein have the same meanings as those generally understood by those skilled in the art to which the various embodiments of the present invention belong. The terms (such as those defined in generally used dictionaries) will be interpreted as having the same meanings as the contextual meanings in the relevant technical field and will not be interpreted as having idealized meanings or overly formal meanings unless clearly defined in the various embodiments of the present invention.

[0024] The technical solution of the present application is applied to vehicles in various venues, such as mining trucks in mines, engineering vehicles or cars in construction sites, etc. Specifically, it utilizes the collaboration between infrared thermal imaging cameras and lidars. When the dust and fog are severe, it can also help mining trucks identify other moving obstacles in the mine environment, so as to perform corresponding obstacle avoidance operations to ensure that mining trucks can be driven safely in the mine environment, thereby improving the driving safety of mining trucks.

[0025] Next, the technical solution of the present application is described with specific embodiments. Example

[0026] The technical solution of this embodiment is applied to the obstacle avoidance and identification process of mine cars in mines. No matter what type of mine, it is difficult to avoid the problem of dust and fog. For both mine cars and people, visibility is low in such dusty and foggy environments, and traffic accidents are likely to occur. In mines, the obstacles that mine cars need to avoid are basically movable obstacles, such as people or another mine car. However, in the case of dust and fog, drivers often cannot detect these obstacles in advance, and it is also difficult to detect people or vehicles behind the dust and fog using laser radar.

[0027] like Figure 1 As shown, the method for identifying obstacles in a mine car of this embodiment is applied to a mine car, wherein the mine car is equipped with a thermal imaging camera and a laser radar, and the method comprises: Step S100: determining whether there is an abnormal area in the acquisition area of ​​the thermal imaging camera through the thermal imaging data of the thermal imaging camera.

[0028] The vehicle may be a mining truck in a mine, an engineering vehicle or a car at a construction site, etc. For the sake of convenience of explanation, this embodiment takes a mining truck as an example, wherein the mining truck is equipped with multiple laser radars, and the laser radars scan the space around the mining truck in real time to obtain point cloud data of the space around the mining truck.

[0029] Specific as Figure 2 As shown, it is a simple top view of a mining truck 100. A laser radar 110 can be respectively arranged at the front, rear and both sides of the vehicle body of the mining truck 100, and a thermal imaging camera 120 is arranged near the laser radar 110. The laser radar and the thermal imaging camera are arranged in a matching manner. These four laser radars 110 can obtain spatial information around the mining truck 100 by real-time scanning. In order to ensure that the scanning area covers the surroundings of the mining truck 100 as much as possible, the laser radar 110 can be arranged in the middle position of the front, rear and both sides of the vehicle body, and the thermal imaging camera can sense the heat source within the sensing range of the laser radar.

[0030] The four laser radars 110 are used to obtain point cloud data around the mining truck, which can represent obstacles around the mining truck. The point cloud data can then be analyzed to obtain corresponding obstacle orientation, type, distance, and other data. The laser radar located at the front of the vehicle can be a CH128X1 radar with a smaller scanning angle, and the laser radars located on both sides of the vehicle body can be a CB64S1 laser radar with a larger scanning angle.

[0031] When scanning, the thermal imaging camera can be used to determine whether there are abnormal areas within the current collection range. The abnormal area indicates the area where there may be heat sources. The thermal imaging camera mainly uses infrared rays to sense the heat sources within the collection area. In a working environment such as a mine, there are usually only mine carts and miners, and the main moving obstacles are also these two. The mine cart is equipped with a laser radar, which is also a heat source, and people are also heat sources, so these two objects can be detected by the thermal imaging camera. These two objects are targets that need to be avoided, so when a heat source is detected, the area where the heat source is detected will be marked as an abnormal area.

[0032] Step S200: If the abnormal area is within the scanning area of ​​the laser radar, a target area is divided in the scanning area, and the target area includes the abnormal area.

[0033] In this embodiment, the actual distance between the abnormal area and the thermal imaging camera is used to determine whether an area is within the scanning area of ​​the laser radar. It can be understood that the farthest distance of the laser radar's scanning area is closer than that of the thermal imaging camera. Therefore, when the thermal imaging camera can identify the abnormal area, the abnormal area is not within the scanning range of the laser radar, so a judgment must be made first.

[0034] A reference object is also provided on the mine car of the present embodiment. The reference object is set in the recognition area of ​​the thermal imaging camera and maintains a fixed preset distance from the thermal imaging camera. Therefore, the distance between the mine car and the abnormal area can be calculated by calculating the distance between the reference object and the abnormal area in the thermal imaging data.

[0035] The reference object can be an additional device that can be set to emit heat, or a laser radar can be used directly as a reference object to calculate the distance between the mine car and the abnormal area. Therefore, as long as the reference object is set in the collection area of ​​the thermal imaging camera and the preset distance between the thermal imaging camera and the reference object is recorded, the actual distance between the abnormal area and the thermal imaging camera can be obtained. Based on the actual distance, it can be determined whether the abnormal area has entered the scanning area of ​​the laser radar.

[0036] Specifically, the actual distance represented by a unit pixel in the current frame of thermal imaging data may be obtained according to a preset distance between the reference object and the thermal imaging camera, and the actual distance represented by a unit pixel in the current frame of thermal imaging data is defined as a first reference distance.

[0037] Then, the actual distance between the abnormal area and the thermal imaging camera can be obtained based on the first reference distance and the number of pixels of the abnormal area and the reference object in the current frame of thermal imaging data.

[0038] For example, the preset distance between the thermal imaging camera and the reference object is 10 centimeters. In the thermal imaging image, the thermal imaging camera and the reference object are 30 pixels apart. Therefore, it can be known that 30 pixels represent 10 centimeters. The first reference distance is the actual distance represented by a single pixel, which is one-third of a centimeter. If the number of pixels between the abnormal area and the reference object obtained after measurement is 3000, it means that the abnormal area and the mining truck are 10 meters apart. Then compare the 10 meters with the scanning distance of the laser radar to know whether it is within the scanning range of the laser radar. If it is found that the abnormal area is not within the scanning area of ​​the laser radar, the actual distance between the abnormal area and the visible light camera in each frame of thermal imaging data is continuously calculated until the abnormal area is within the scanning area of ​​the laser radar.

[0039] Among them, when the abnormal area is within the scanning area of ​​the laser radar, a target area can be divided for the abnormal area according to the scanning result of the laser radar, so that the target area includes the abnormal area.

[0040] It is understandable that the thermal imaging camera only captures the heat source, but the specific heat source cannot be known based on the content captured by the thermal imaging camera, because in addition to the people and mining vehicles that need to be avoided, there are also some other heat sources, such as street lamps, mining machines and other heat-generating equipment, so it is necessary to scan the laser radar to determine what the object in the abnormal area is. Prior to this, the target area is circled to circle the image area that needs to be analyzed, so as to facilitate subsequent analysis operations.

[0041] Step S300, comparing the point cloud data of the laser radar on the abnormal area with the thermal imaging data of the thermal imaging camera on the abnormal area to determine whether there is dust and fog interference between the mining truck and the abnormal area.

[0042] In this embodiment, the main purpose is to let the mining truck avoid movable obstacles, such as other mining trucks and people, or even some small animals. Both of these targets that need to be avoided will move autonomously, so the thermal imaging data changes of the abnormal area can be obtained based on the activity traces of the thermal imaging of the abnormal area within a preset time period, so as to determine whether the obstacle in the abnormal area is a movable obstacle or a static obstacle.

[0043] Specifically, it is possible to determine whether there is a target object in the abnormal area based on whether there is a change in the thermal imaging data of the abnormal area within a unit time, and the target object is a movable obstacle. It is understandable that whether it is a person or a car, as long as it is moving, its image data in the thermal imaging image will move or deform, so it is possible to compare the thermal imaging data within a unit time and determine the change in the thermal imaging data to determine whether the obstacle is a movable obstacle.

[0044] If there is a target object, the point cloud data of the abnormal area can be obtained based on the point cloud data on the target area and the thermal imaging data on the target area. If there is no target object, it means that there is no object to be avoided, so there is no need to obtain point cloud data for analysis.

[0045] When there is a target object, it is necessary to analyze and identify these point cloud data to determine whether the target object is a person or a car. It is understandable that in a mine, there is usually a lot of dust to form dust fog. When there is no dust fog interference in the site, clear and complete data can be obtained. However, when there is dust fog interference in the site, it is necessary to consider the impact of dust fog interference for identification and analysis, so it is also necessary to determine whether there is dust fog interference.

[0046] Among them, lidar acquires point cloud data by emitting lasers, but lasers have weak penetration into dust and fog. When the dust and fog are thick, the point cloud data behind the dust and fog cannot be identified. The wavelength of infrared light in the electromagnetic spectrum is between red light and microwaves. It can pass through many objects such as clouds, fog, smoke, dust, glass, etc. Therefore, even in a dusty and foggy environment, infrared cameras can capture the heat source behind the morning fog.

[0047] Whether there is a difference between the change in thermal imaging data of the abnormal area and the change in point cloud data of the abnormal area that can be identified, if the difference in change is less than a preset value, it is determined that there is no dust and fog interference between the mining card and the abnormal area; if the difference in the change in thermal imaging data of the abnormal area and the change in point cloud data of the abnormal area is not less than a preset value, it is determined that there is dust and fog interference between the mining card and the abnormal area.

[0048] The difference between the change in thermal imaging data and the change in point cloud data in the abnormal area refers to the difference between thermal imaging data and point cloud data in the established area. For example, if there is a person in the identification area, the acquisition of thermal imaging data is not affected by dust and fog, so a human figure can be identified, while point cloud data identification is affected by dust and fog. Therefore, if the recognized shape is very different from the thermal imaging data, it means that there is dust and fog interference, causing the laser radar to identify unnecessary things. If the two are similar, both are human figures, it means that there is no dust and fog interference. The above-mentioned preset value can be similarity.

[0049] Step S400: If there is dust and fog interference, the abnormal area is identified based on the thermal imaging data.

[0050] It can be understood that when there is no dust and fog interference, the point cloud data obtained by the laser radar can directly obtain the distance between the mining truck and the target object in the abnormal area. Identify the object and then perform the corresponding obstacle avoidance operation. Therefore, this embodiment mainly solves the identification and obstacle avoidance under the condition of dust and fog interference.

[0051] First, the target area may be divided into at least two identification sub-areas, and the identification sub-areas are distributed along the height of the thermal imaging image frame composed of the thermal imaging data.

[0052] like Figure 3 The following is an example of the distribution of three recognition sub-areas in the target area. Figure 3 In addition to the upper, middle and lower identification sub-areas, more identification sub-areas can also be set.

[0053] The abnormal region is overlapped with the identification sub-region to obtain the distribution information of the abnormal region in the identification sub-region.

[0054] By overlapping the abnormal area and the identification sub-area, it is equivalent to dividing the abnormal area. In this application, the identification sub-area is divided mainly to divide the height of the abnormal area. By dividing the height, the distribution of people or mining trucks in the abnormal area is determined. Therefore, the sub-area can be set up at each preset height, for example, a sub-area is set up every 1.5 meters. Figure 3 The height of the middle area is 4.5 meters. If the area is higher, 6 meters, then 4 sub-areas can be set.

[0055] Then, the obstacle recognition result can be obtained based on the distribution information of the abnormal area in the recognition sub-area.

[0056] Specifically, the height of a mining truck is generally above 4 to 5 meters, while the height of a person is generally between 1.6 and 1.8 meters. It can be seen that there is a significant difference in the height of a vehicle and a person. Therefore, in the case of dust and fog interference, the height of the identified heat source can be used to determine whether the obstacle in an area is a person or another mining truck. For example, a person will occupy the third identification sub-area and the second identification sub-area, while a mining truck will occupy the first identification sub-area, the second identification sub-area, and the third identification sub-area. In addition, there may be some animals such as dogs raised in the mine, which will only occupy the third identification sub-area. The above sub-areas are designed according to height, so it is possible to automatically determine whether the target object is a person or a car based on the sub-area where the target object is located, and then perform corresponding obstacle avoidance operations based on the difference between the person and the car.

[0057] It is understandable that the transportation environment of a mine car in a mine is different from driving on general traffic roads. The environment in a mine is simple and there are fewer movable obstacles. Therefore, once an movable obstacle is identified, it is necessary to pay attention and perform corresponding operations. Although the environment in the mine makes it impossible to accurately identify the outline of the obstacle with a lidar, because the types of movable obstacles that appear are limited, there is no need to accurately identify the obstacles. Only a rough classification is required, and the next obstacle avoidance method can be determined based on the classification results.

[0058] The embodiment of the present application provides a method for identifying obstacles of a mine truck, which mainly determines whether there is an abnormal area in the acquisition area of ​​the thermal imaging camera through the thermal imaging data of the thermal imaging camera; if the abnormal area is in the scanning area of ​​the laser radar, a target area is divided in the scanning area, and the target area includes the abnormal area; the point cloud data of the laser radar on the abnormal area and the thermal imaging data of the thermal imaging camera on the abnormal area are compared to determine whether there is dust and fog interference between the mine truck and the abnormal area; if there is dust and fog interference, the abnormal area is identified based on the thermal imaging data. When the mining bus is driving in a mine pit or a mine field, the thermal imaging camera and the laser radar can cooperate to smoothly identify obstacles even in the face of dust and fog interference, and perform corresponding classification and identification for people and vehicles, thereby providing powerful intelligence for the automatic obstacle avoidance method, enhancing the obstacle recognition ability of the mining truck in the mine field environment, thereby increasing the obstacle avoidance ability, and improving the safety of the mining truck and miners. Example

[0059] like Figure 4 As shown, this embodiment provides a device 200 for identifying obstacles in a mine car, which is applied to a mine car. The mine car is equipped with a thermal imaging camera and a laser radar. The device includes: The thermal imaging module 210 is used to determine whether there is an abnormal area in the acquisition area of ​​the thermal imaging camera through the thermal imaging data of the thermal imaging camera; A division module 220, configured to divide a target area in the scanning area if the abnormal area is within the scanning area of ​​the laser radar, wherein the target area includes the abnormal area; A judgment module 230 is used to compare the point cloud data of the laser radar on the abnormal area with the thermal imaging data of the thermal imaging camera on the abnormal area to determine whether there is dust and fog interference between the mining truck and the abnormal area; The identification module 240 is used to identify the abnormal area based on the thermal imaging data if there is dust and fog interference.

[0060] The present application also provides a terminal device, including a processor and a memory, wherein the memory stores a computer program, and when the computer program is run on the processor, the method for identifying obstacles of a mine car is executed.

[0061] The terminal device can be a computer, a tablet, or a vehicle-mounted computer, etc., which can be installed in the vehicle to perform corresponding automatic driving control.

[0062] The present application also provides a readable storage medium storing a computer program, wherein the computer program executes the method for identifying obstacles of a mine car when running on a processor.

[0063] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and structure diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present invention. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in an alternative implementation, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the structure diagram and / or the flow chart, and the combination of boxes in the structure diagram and / or the flow chart, can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0064] In addition, the functional modules or units in the various embodiments of the present invention may be integrated together to form an independent part, or each module may exist independently, or two or more modules may be integrated to form an independent part.

[0065] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a smart phone, a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0066] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A method for identifying obstacles in a mine car, characterized in that: Applied to a mine car, the mine car is equipped with a thermal imaging camera and a laser radar, the method comprises: Determining whether there is an abnormal area in the acquisition area of ​​the thermal imaging camera through the thermal imaging data of the thermal imaging camera; If the abnormal area is within the scanning area of ​​the laser radar, a target area is divided in the scanning area, and the target area includes the abnormal area; Compare the point cloud data of the laser radar on the abnormal area with the thermal imaging data of the thermal imaging camera on the abnormal area to determine whether there is dust and fog interference between the mining truck and the abnormal area; If there is dust and fog interference, the abnormal area is identified based on thermal imaging data.

2. The identification method according to claim 1, characterized in that: The point cloud data of the laser radar on the abnormal area is obtained by the following steps: Based on whether there is a change in the thermal imaging data of the abnormal area within a unit time, determining whether there is a target object in the abnormal area; If there is a target object in the abnormal area, the point cloud data of the abnormal area is obtained based on the point cloud data on the target area and the thermal imaging data on the target area.

3. The identification method according to claim 1 or 2, characterized in that: The comparing the point cloud data of the laser radar on the abnormal area with the thermal imaging data of the thermal imaging camera on the abnormal area comprises: According to the activity traces of the thermal imaging of the abnormal area within a preset time period, the thermal imaging data changes of the abnormal area are obtained; According to the activity traces of the point cloud data of the abnormal area in the same corresponding time period, the point cloud data changes of the abnormal area are obtained; Compare the changes in thermal imaging data in the abnormal area with the changes in point cloud data in the abnormal area.

4. The identification method according to claim 3, characterized in that: The determining whether there is dust and fog interference between the mining truck and the abnormal area includes: If the difference between the change in thermal imaging data in the abnormal area and the change in point cloud data in the abnormal area is less than the preset value, it is determined that there is no dust and fog interference between the mining truck and the abnormal area; If the difference between the change in thermal imaging data in the abnormal area and the change in point cloud data in the abnormal area is not less than the preset value, it is determined that there is dust and fog interference between the mining truck and the abnormal area.

5. The identification method according to claim 1, 2 or 4, characterized in that: The mine car is also provided with a reference object, and the reference object is provided in the collection area of ​​the thermal imaging camera; if the abnormal area is in the scanning area of ​​the laser radar, it includes: According to the preset distance between the reference object and the thermal imaging camera, combined with the image data captured by the thermal imaging camera, the actual distance between the abnormal area and the thermal imaging camera is obtained; According to the actual distance between the abnormal area and the thermal imaging camera, it is repeatedly determined whether the abnormal area is within the scanning area of ​​the laser radar.

6. The identification method according to claim 5, characterized in that: The reference object is the laser radar.

7. The identification method according to claim 1, 2, 4 or 6, characterized in that: If there is dust and fog interference, the abnormal area is identified based on the thermal imaging data, including: Dividing the target area into at least two identification sub-areas, wherein the identification sub-areas are distributed along the height of the thermal imaging image frame composed of the thermal imaging data; Overlap the abnormal area with the identification sub-area to obtain the distribution information of the abnormal area in the identification sub-area; Obstacle recognition results are obtained based on the distribution information of the abnormal area in the recognition sub-area.

8. A device for identifying obstacles in a mine car, characterized in that: Applied to a mining car, the mining car is equipped with a thermal imaging camera and a laser radar, and the device includes: A thermal imaging module, used to determine whether there is an abnormal area in the acquisition area of ​​the thermal imaging camera through the thermal imaging data of the thermal imaging camera; A division module, for dividing a target area in the scanning area if the abnormal area is within the scanning area of ​​the laser radar, wherein the target area includes the abnormal area; A judgment module, used to compare the point cloud data of the laser radar on the abnormal area with the thermal imaging data of the thermal imaging camera on the abnormal area, and judge whether there is dust and fog interference between the mining truck and the abnormal area; The recognition module is used to identify abnormal areas based on thermal imaging data if there is dust and fog interference.

9. A terminal device, characterized in that: The invention comprises a processor and a memory, wherein the memory stores a computer program, and when the computer program is run on the processor, the method for identifying obstacles of a mining vehicle according to any one of claims 1 to 7 is executed.

10. A readable storage medium, characterized in that: The device stores a computer program, which executes the method for identifying obstacles of a mining vehicle according to any one of claims 1 to 7 when the computer program is run on a processor.

Images