A mine area obstacle identification method and device for edge device deployment

Abstract

The application discloses a mine area obstacle identification method and device for edge device deployment, a storage medium and a computer device. The method comprises the following steps: determining a target point cloud frame based on a point cloud frame collected by a preset laser radar, processing the target point cloud frame by using a preset processing tool, and obtaining a target feature; inputting the target feature into a preset identification model with multiple detection heads, obtaining first information corresponding to the target point cloud frame, wherein the first information comprises obstacle attribute information, obstacle position information, obstacle size information and obstacle heading information corresponding to different obstacles; determining second information corresponding to the target point cloud frame based on the first information and historical first information, wherein the second information comprises obstacle motion speed information; and outputting the first information and the second information. The application can make the determination of information corresponding to different obstacles more accurate, and is beneficial to improving the safety of mine equipment.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a method and apparatus for identifying obstacles in mining areas for edge device deployment, a storage medium, and a computer device. Background Technology

[0002] With the continuous development of smart mines, many mining areas are now using unmanned driving technology for their equipment. This application not only reduces safety risks for mine operators but also significantly improves production efficiency in mining areas.

[0003] The application of autonomous driving technology is inseparable from deep learning models. However, in the current application of many autonomous mining equipment, the construction mode of deep learning models is too simple, and the accuracy of extracting relevant information about various obstacles in the mining environment is limited. Summary of the Invention

[0004] In view of this, this application provides a method and apparatus for identifying obstacles in mining areas, a storage medium, and a computer device for deployment on edge devices, which can make the determination of the first information more accurate, and thus make the determination of the second information more accurate, which is conducive to improving the safety of mining equipment.

[0005] According to one aspect of this application, a method for identifying obstacles in mining areas for edge device deployment is provided, comprising:

[0006] The target point cloud frame is determined based on the point cloud frame collected by the preset lidar, and the target point cloud frame is processed using the preset processing tool to obtain the target features;

[0007] The target features are input into a preset recognition model with multiple detection heads to obtain first information corresponding to the target point cloud frame. The first information includes obstacle attribute information, obstacle position information, obstacle size information and obstacle heading information corresponding to different obstacles.

[0008] Based on the first information and the historical first information, the second information corresponding to the target point cloud frame is determined, and the second information includes obstacle movement speed information;

[0009] Output the first information and the second information.

[0010] According to another aspect of this application, a mine obstacle recognition device for edge device deployment is provided, comprising:

[0011] The point cloud frame determination module is used to determine the target point cloud frame based on the point cloud frame collected by the preset lidar, and to process the target point cloud frame using a preset processing tool to obtain target features;

[0012] The image input module is used to input the target features into a preset recognition model with multiple detection heads to obtain first information corresponding to the target point cloud frame. The first information includes obstacle attribute information, obstacle position information, obstacle size information and obstacle heading information corresponding to different obstacles.

[0013] An information determination module is used to determine second information corresponding to the target point cloud frame based on the first information and historical first information, wherein the second information includes obstacle movement speed information;

[0014] The information output module is used to output the first information and the second information.

[0015] According to another aspect of this application, an autonomous mining truck is provided, the autonomous mining truck including the above-described obstacle recognition device for edge device deployment in the mining area.

[0016] According to another aspect of this application, a storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the above-described method for identifying obstacles in mining areas for edge device deployment.

[0017] According to another aspect of this application, a computer device is provided, including a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein the processor executes the program to implement the above-described method for identifying obstacles in mining areas for edge device deployment.

[0018] Using the above technical solution, this application provides a method and apparatus for obstacle recognition in mining areas deployed on edge devices, a storage medium, and a computer device. First, a target point cloud frame can be determined using point cloud frames acquired by a preset lidar. Then, the target point cloud frame can be processed using a preset processing tool to obtain corresponding target features. After obtaining the target features, they can be input into a preset recognition model with multiple detection heads. The preset recognition model can output first information of the target point cloud frame. Here, the first information may include obstacle attribute information, obstacle location information, obstacle size information, obstacle heading information, etc., corresponding to different obstacles. After the preset recognition model outputs the first information, it can further determine second information corresponding to the target point cloud frame based on the first information and historical first information. The second information may include obstacle movement speed information in the target point cloud frame, etc. After obtaining the first and second information, they can be output to achieve automatic driving of mining equipment. This application embodiment sets the preset recognition model to have multiple detection heads, and selects different detection heads for obstacles of different sizes to determine the first information, which can make the determination of the first information more accurate, and thus make the determination of the second information more accurate, which is conducive to improving the safety of mining equipment.

[0019] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0020] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments of this application and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0021] Figure 1 A flowchart illustrating a method for identifying obstacles in mining areas for edge device deployment, provided in an embodiment of this application, is shown.

[0022] Figure 2 A flowchart illustrating another obstacle identification method for edge device deployment in mining areas provided by an embodiment of this application is shown.

[0023] Figure 3 This illustration shows a schematic diagram of the installation location of a preset lidar according to an embodiment of this application;

[0024] Figure 4 This illustration shows a structural schematic diagram of a mine obstacle recognition device for edge device deployment, provided in an embodiment of this application. Detailed Implementation

[0025] The present application will be described in detail below with reference to the accompanying drawings and embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in the embodiments of the present application can be combined with each other.

[0026] This embodiment provides a method for obstacle recognition in mining areas for deployment on edge devices, such as... Figure 1 As shown, the method includes:

[0027] Step 101: Determine the target point cloud frame based on the point cloud frame collected by the preset lidar, and process the target point cloud frame using a preset processing tool to obtain the target features;

[0028] The obstacle recognition method for mining areas, deployed on edge devices, provided in this application embodiment can be deployed on mining equipment and can significantly improve the speed of obstacle recognition. First, a target point cloud frame can be determined using point cloud frames acquired by a preset LiDAR. Then, a preset processing tool can be used to process the target point cloud frame to obtain the corresponding target features. The preset LiDAR can be one or multiple. When there is only one preset LiDAR, the point cloud frame acquired by that LiDAR can be used as the target point cloud frame. When there are multiple preset LiDARs, the target point cloud frame can be a superimposed point cloud frame of these multiple point cloud frames. Here, the preset processing tool can be a tool based on the PointPillars concept. This tool can perform pillar projection on the target point cloud frame and then use PointNet to extract pillar features, i.e., target features.

[0029] Step 102: Input the target features into a preset recognition model with multiple detection heads to obtain first information corresponding to the target point cloud frame. The first information includes obstacle attribute information, obstacle position information, obstacle size information and obstacle heading information corresponding to different obstacles.

[0030] In this embodiment, after obtaining the target features, the target features can be input into a preset recognition model with multiple detection heads. The preset recognition model can output the first information of the target point cloud frame. Here, the preset recognition model can be a trained deep learning model. Each detection head in the preset recognition model corresponds to a scale range and can be used to determine the first information of obstacles belonging to that scale range. When the scale of an obstacle matches one of the scale ranges, the first information of the obstacle is determined using the detection head corresponding to that scale range. The scale of the obstacle can be determined based on the obstacle's own size. The first information can include obstacle attribute information, obstacle position information, obstacle size information, obstacle heading information, etc. Among them, obstacle attribute information can be obstacle type information, such as van, pickup truck, etc.; obstacle position information can be the coordinate position information of the obstacle in the target point cloud frame; obstacle size information can be the length, width, and height dimensions of the obstacle's bounding box; and obstacle heading information can be the obstacle's direction of movement.

[0031] Step 103: Based on the first information and the historical first information, determine the second information corresponding to the target point cloud frame, the second information including obstacle movement speed information;

[0032] In this embodiment, after the preset recognition model outputs first information, second information corresponding to the target point cloud frame can be determined based on the first information and historical first information. The second information may include obstacle movement speed information, etc. The historical first information can be the first information corresponding to the previous frame or the previous N frames of the target point cloud frame; in other words, any first information obtained before the first information corresponding to the current target point cloud frame can be called historical first information. The second information can be obtained based on first information from multiple frames.

[0033] Step 104: Output the first information and the second information.

[0034] In this embodiment, after obtaining the first information and the second information, the first information and the second information can be output to realize the automatic driving of the mining equipment.

[0035] Optionally, before step 103, the method further includes: determining invalid detection boxes and overlapping detection boxes based on the detection box information in the first information, and removing the detection box information corresponding to the invalid detection boxes and the overlapping detection boxes from the first information.

[0036] In this embodiment, the first information further includes detection box information. The first information obtained according to the preset recognition model may also include detection box information. Before determining the second information, it can be identified whether the detection box information included in each piece of the first information corresponds to an invalid detection box or an overlapping detection box. If it is an invalid or overlapping detection box, the corresponding detection box information is removed from the first information, ensuring that the remaining detection box information in the first information corresponds to valid detection boxes. By removing invalid or overlapping detection boxes, this embodiment effectively avoids the problems of inaccurate determination and low precision when determining the second information, while improving the efficiency of determining the second information.

[0037] By applying the technical solution of this embodiment, firstly, the target point cloud frame can be determined using the point cloud frame acquired by the preset lidar. Then, the target point cloud frame can be processed using a preset processing tool to obtain the corresponding target features. After obtaining the target features, the target features can be input into a preset recognition model with multiple detection heads. The preset recognition model can output the first information of the target point cloud frame. Here, the first information may include obstacle attribute information, obstacle position information, obstacle size information, obstacle heading information, etc., corresponding to different obstacles. After the preset recognition model outputs the first information, it can further determine the second information corresponding to the target point cloud frame based on the first information and historical first information. The second information may include obstacle movement speed information in the target point cloud frame, etc. After obtaining the first and second information, the first and second information can be output to achieve automatic driving of the mining equipment. This embodiment of the application sets the preset recognition model to have multiple detection heads, selecting different detection heads for obstacles of different sizes to determine the first information, making the determination of the first information more accurate, and consequently, the determination of the second information more accurate, which is beneficial to improving the safety of the mining equipment.

[0038] Furthermore, as a refinement and extension of the specific implementation of the above embodiments, and to fully illustrate the specific implementation process of this embodiment, another method for obstacle identification in mining areas deployed for edge devices is provided, such as... Figure 2 As shown, the method includes:

[0039] Step 201: Receive the first target point cloud frame sent by the first radar set at the first target location of the mining equipment, and the second target point cloud frame sent by the second radar set at the second target location.

[0040] In this embodiment, the preset lidar includes a first lidar and a second lidar. Due to the large size of mining equipment, high-beam lidars are typically installed at the front of the vehicle, making 360° circumferential AI perception impossible. Currently, the 3D environmental AI perception function of mining equipment mainly relies on a single high-beam lidar placed at the front of the vehicle, which has a small horizontal field of view and limited perception range. This application installs two lidars on the mining equipment, positioned on either side of the front of the vehicle, with both lidars installed at the same height and tilted perpendicular to the bottom of the vehicle. Figure 3 As shown, the laser radar can be positioned at the midpoint of the left and right edges of the approximate surface of the mining equipment on the plane of the windshield. This location allows for a superimposed scanning area covering a horizontal field of view of nearly 340°, significantly improving the coverage of the scanning area and reducing the false negative rate. The laser radar can be a multi-line laser radar, and the approximate surface of the mining equipment can be the area projected onto the plane of the windshield by the front of the mining equipment. The midpoints of the left and right edges of the plane of the windshield can be designated as the first target location and the second target location, respectively. The laser radar positioned at the first target location can be called the first radar, and the laser radar positioned at the second target location can be called the second radar. First, it can receive the first target point cloud frame acquired by the first radar and the second target point cloud frame acquired by the second radar.

[0041] Step 202: Overlay the first target point cloud frame and the second target point cloud frame to obtain the target point cloud frame;

[0042] In this embodiment, after receiving the first target point cloud frame and the second target point cloud frame, the first target point cloud frame and the second target point cloud frame can be superimposed to obtain the corresponding target point cloud frame. Superposition processing involves merging two target point cloud frames into one target point cloud frame. The acquisition time corresponding to the two target point cloud frames in the superposition process is the same.

[0043] Step 203: Use the preset processing tool to perform grid division on the target point cloud frame;

[0044] In this embodiment, after obtaining the target point cloud frame, a preset processing tool can be used to divide the target point cloud frame into multiple grids, each of which can include multiple point cloud points. The point cloud can be projected onto a plane of its top view to create a pseudo-2D image, and then the pseudo-2D image is divided into grids according to preset dimensions.

[0045] Step 204: After the partitioning is completed, multiple idle threads are called through the target kernel function to extract the original features of the corresponding point cloud points in each grid.

[0046] In this embodiment, after the mesh generation is complete, the target kernel function can be used to call multiple idle threads, which can then jointly extract the original features of the corresponding point cloud points in each mesh. Specifically, point cloud data falling within the same mesh are grouped into a pillar, and for a point cloud point in each pillar, the original features (x, y, z, r, x) are extracted. c y c , z c x p y p Since point cloud point operations need to be performed individually, CUDA acceleration technology is used to improve real-time performance. This involves calling multiple idle threads simultaneously through the target kernel function, which accelerates the processing and significantly increases processing speed. The target kernel function can be a CUDA kernel function.

[0047] Step 205: Based on the original features of the corresponding point cloud points in all grids, obtain the target features;

[0048] In this embodiment, after obtaining the original features of each point cloud point within each grid, the target features can be obtained using these original features of the point cloud points.

[0049] Step 206: Input the target features into the preset recognition model, determine the scale of any obstacle in the target features, determine the target detection head corresponding to any obstacle from the plurality of detection heads according to the scale, and obtain the first sub-information corresponding to any obstacle based on the target detection head;

[0050] In this embodiment, since the preset recognition model corresponds to multiple detection heads, the scale corresponding to each obstacle in the target feature can be determined first. Then, the target detection head used to determine the first sub-information of each obstacle can be found according to the scale corresponding to different obstacles. For obstacles of different scales, appropriate anchor sizes can be designed in different detection heads. Anchors refer to reference boxes of different sizes and aspect ratios preset on the image. For example, after the preset recognition model determines that the target feature includes three obstacles, the scales of these three obstacles are A, B, and C, respectively. The preset recognition model corresponds to three detection heads: the scale range of detection head 1 is a to b, the scale range of detection head 2 is b to c, and the scale range of detection head 3 is c to d. A is in the range of a to b, B is in the range of b to c, and C is in the range of c to d. Therefore, the first sub-information of the obstacle with scale A can be determined using detection head 1, the first sub-information of the obstacle with scale B can be determined using detection head 2, and the first sub-information of the obstacle with scale C can be determined using detection head 3. The inference acceleration of the preset recognition model can be achieved using the TensorRT inference platform. To ensure inference performance and speed, the parameters can be quantized using FP16. This application embodiment is geared towards the development of an embedded computing platform for vehicles, possessing high real-time inference performance and enabling real-time 3D perception of the environment during the operation of autonomous mining equipment in mining areas.

[0051] Before using the preset recognition model, an initial recognition model can be obtained and trained using training sample frames. When the model loss value of the trained initial recognition model is less than a preset loss threshold, the preset recognition model is obtained. Here, the initial recognition model can be a deep learning model based on the deep learning model currently used in autonomous driving technology, corresponding to multiple detection heads; specifically, it can be a PointPillars network model. To improve the richness of the scenarios corresponding to the training sample frames, in addition to collecting environmental data under normal working conditions in the mining area, scenarios are supplemented by defining specific collection scenarios. Considering the diversity of transportation roads in the mining area, the collection scenarios are designated as straight roads, slopes, and curves, further divided into single lanes, one-way two-lane roads, and two-way lanes based on lane width; the working face is designated as loading and unloading areas. A varying number of obstacles are set in each of these scenarios, and various behaviors of the obstacles are specified, such as overtaking, acceleration / deceleration, and turning. Point cloud data collected in the above scenarios is cleaned and labeled to construct the final training sample frames.

[0052] Step 207: Based on the first sub-information corresponding to the different obstacles, obtain the first information, which includes obstacle attribute information, obstacle position information, obstacle size information, and obstacle heading information corresponding to the different obstacles;

[0053] In this embodiment, after determining the first sub-information corresponding to each obstacle, the first information corresponding to the target point cloud frame can be obtained based on this first sub-information. The first sub-information corresponding to each obstacle includes obstacle attribute information, obstacle location information, obstacle size information, and obstacle heading information. The first sub-information corresponding to different obstacles is then aggregated to obtain the first information.

[0054] Step 208: Based on the first information, the historical first information, and the Hungarian algorithm, the different obstacles are associated to obtain association information; based on the association information, the first information, the historical first information, and the Kalman filter method, the second information is obtained, the second information including obstacle movement speed information;

[0055] In this embodiment, after determining the first information, the first information, historical first information, and the Hungarian algorithm can be used to associate different obstacles to obtain associated information. Then, the associated information, the first information, historical first information, and the Kalman filter method can be used to obtain the second information. The second information may include obstacle movement speed information corresponding to each obstacle in the target point cloud frame.

[0056] Step 209: Output the first information and the second information.

[0057] In this embodiment, after obtaining the first information and the second information, the first information and the second information can be output to realize the automatic driving of the mining equipment.

[0058] Optionally, before step 201, the method further includes: receiving a first sample point cloud frame sent by a first radar set at a first target location of the mining equipment, and a second sample point cloud frame sent by a second radar set at a second target location, wherein the first sample point cloud frame and the second sample point cloud frame include point cloud sets corresponding to the same marker; extracting the first point cloud set of the marker in the first sample point cloud frame and the second point cloud set of the marker in the second sample point cloud frame; and performing registration processing on the first point cloud set and the second point cloud set to obtain a coordinate transformation matrix.

[0059] In this embodiment, before applying the first and second radars to achieve unmanned operation of the mining equipment, the first and second radars can be calibrated. First, a marker can be placed in a preset environment. The first radar collects a first sample point cloud frame, and the second radar collects a second sample point cloud frame. Both the first and second sample point cloud frames can include the point cloud set corresponding to the marker. Next, the first point cloud set corresponding to the marker can be extracted from the first sample point cloud frame, and the second point cloud set corresponding to the marker can be extracted from the second sample point cloud frame. Since the first and second point cloud sets correspond to the same marker, registration processing can be performed based on the first and second point cloud sets, resulting in a coordinate transformation matrix.

[0060] Optionally, in this embodiment of the application, step 202 includes: using the coordinate transformation matrix to perform coordinate transformation processing on the first target point cloud frame and the second target point cloud frame, and superimposing them after the coordinate transformation processing to obtain the target point cloud frame.

[0061] In this embodiment, after obtaining the coordinate transformation matrix, when superimposing the first target point cloud frame and the second target point cloud frame, the coordinate transformation matrix can be used to perform coordinate transformation on the first target point cloud frame and the second target point cloud frame respectively. After the transformation process, they are superimposed together to obtain the target point cloud frame.

[0062] Furthermore, as Figure 1 In terms of specific implementation, this application provides a mining area obstacle recognition device for deployment on edge devices, such as... Figure 4 As shown, the device includes:

[0063] The point cloud frame determination module is used to determine the target point cloud frame based on the point cloud frame collected by the preset lidar, and to process the target point cloud frame using a preset processing tool to obtain target features;

[0064] The image input module is used to input the target features into a preset recognition model with multiple detection heads to obtain first information corresponding to the target point cloud frame. The first information includes obstacle attribute information, obstacle position information, obstacle size information and obstacle heading information corresponding to different obstacles.

[0065] An information determination module is used to determine second information corresponding to the target point cloud frame based on the first information and historical first information, wherein the second information includes obstacle movement speed information;

[0066] The information output module is used to output the first information and the second information.

[0067] Optionally, the image input module is used for:

[0068] The target features are input into the preset recognition model to determine the scale of any obstacle in the target features. Based on the scale, a target detection head corresponding to any obstacle is determined from the plurality of detection heads, and first sub-information corresponding to any obstacle is obtained based on the target detection head. Based on the first sub-information corresponding to different obstacles, the first information is obtained.

[0069] Optionally, the point cloud frame determination module is used for:

[0070] The target point cloud frame is divided into grids using the preset processing tool; after the division is completed, multiple idle threads are called through the target kernel function to extract the original features of the corresponding point cloud points in each grid; based on the original features of the corresponding point cloud points in all grids, the target features are obtained.

[0071] Optionally, the preset lidar includes a first radar and a second radar; the point cloud frame determination module includes:

[0072] The receiving unit is used to receive a first target point cloud frame transmitted by a first radar set at a first target location in the mining equipment, and a second target point cloud frame transmitted by a second radar set at a second target location.

[0073] The overlay unit is used to overlay the first target point cloud frame and the second target point cloud frame to obtain the target point cloud frame.

[0074] Optionally, the device further includes:

[0075] The point cloud frame receiving module is used to receive, before receiving the first target point cloud frame sent by the first radar set at the first target location of the mining equipment, a first sample point cloud frame sent by the first radar set at the first target location of the mining equipment, and a second sample point cloud frame sent by the second radar set at the second target location, wherein the first sample point cloud frame and the second sample point cloud frame include a point cloud set corresponding to the same marker.

[0076] The extraction module is used to extract the first point cloud set of the marker in the first sample point cloud frame and the second point cloud set of the marker in the second sample point cloud frame, respectively.

[0077] The registration module is used to register the first point cloud set and the second point cloud set to obtain a coordinate transformation matrix;

[0078] Accordingly, the superposition unit is used for:

[0079] Using the coordinate transformation matrix, the first target point cloud frame and the second target point cloud frame are transformed by coordinate transformation, and then superimposed after coordinate transformation to obtain the target point cloud frame.

[0080] Optionally, the information determination module is used to:

[0081] Based on the first information, the historical first information, and the Hungarian algorithm, the different obstacles are associated to obtain association information; based on the association information, the first information, the historical first information, and the Kalman filter method, the second information is obtained.

[0082] Optionally, the first information further includes detection box information; the device further includes:

[0083] The elimination module is used to determine invalid detection boxes and overlapping detection boxes based on the detection box information in the first information before determining the second information corresponding to the target point cloud frame based on the first information and the historical first information, and to remove the detection box information corresponding to the invalid detection boxes and the overlapping detection boxes from the first information.

[0084] According to another aspect of this application, an autonomous mining truck is provided, the autonomous mining truck including the above-described obstacle recognition device for edge device deployment in the mining area.

[0085] It should be noted that other corresponding descriptions of the functional units involved in the obstacle recognition device for edge device deployment provided in this application embodiment can be found in the following references. Figures 1 to 3 The corresponding descriptions in the method will not be repeated here.

[0086] Based on the above, Figures 1 to 3 Accordingly, this application also provides a storage medium storing a computer program, which, when executed by a processor, implements the above-described method. Figures 1 to 3 The method for obstacle recognition in mining areas is shown for deployment on edge devices.

[0087] Based on this understanding, the technical solution of this application can be embodied in the form of a software product. This software product can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, or portable hard drive), and includes several instructions to cause a computer device (such as a personal computer, server, or network device) to execute the methods described in the various implementation scenarios of this application.

[0088] Based on the above, Figures 1 to 3 The method shown, and Figure 4To achieve the above objectives, the present application also provides a computer device, specifically a personal computer, server, network device, etc., as shown in the virtual device embodiment. This computer device includes a storage medium and a processor; the storage medium stores a computer program; the processor executes the computer program to achieve the above-described objectives. Figures 1 to 3 The method for obstacle recognition in mining areas is shown for deployment on edge devices.

[0089] Optionally, the computer device may also include a user interface, a network interface, a camera, radio frequency (RF) circuitry, sensors, audio circuitry, a Wi-Fi module, etc. The user interface may include a display screen, input units such as a keyboard, etc., and optional user interfaces may also include USB interfaces, card reader interfaces, etc. The network interface may optionally include standard wired interfaces, wireless interfaces (such as Bluetooth interfaces, Wi-Fi interfaces), etc.

[0090] Those skilled in the art will understand that the computer device structure provided in this embodiment does not constitute a limitation on the computer device, and may include more or fewer components, or combine certain components, or have different component arrangements.

[0091] The storage medium may also include an operating system and a network communication module. The operating system is a program that manages and stores the hardware and software resources of a computer device, supporting the operation of information processing programs and other software and / or programs. The network communication module is used to enable communication between the various components within the storage medium, as well as communication with other hardware and software within the physical device.

[0092] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented using software plus necessary general-purpose hardware platforms, or it can be implemented using hardware. First, the target point cloud frame can be determined using point cloud frames acquired by a preset LiDAR. Then, the target point cloud frame can be processed by a preset processing tool to obtain the corresponding target features. After obtaining the target features, the target features can be input into a preset recognition model with multiple detection heads. The preset recognition model can output the first information of the target point cloud frame. Here, the first information may include obstacle attribute information, obstacle position information, obstacle size information, obstacle heading information, etc., corresponding to different obstacles. After the preset recognition model outputs the first information, it can further determine the second information corresponding to the target point cloud frame based on the first information and historical first information. The second information may include obstacle movement speed information in the target point cloud frame, etc. After obtaining the first and second information, the first and second information can be output to realize the automatic driving of mining equipment. This application embodiment sets the preset recognition model to have multiple detection heads, and selects different detection heads for obstacles of different sizes to determine the first information, which can make the determination of the first information more accurate, and thus make the determination of the second information more accurate, which is conducive to improving the safety of mining equipment.

[0093] Those skilled in the art will understand that the accompanying drawings are merely schematic diagrams of a preferred embodiment, and the modules or processes shown in the drawings are not necessarily essential for implementing this application. Those skilled in the art will understand that the modules in the apparatus of the embodiment can be distributed within the apparatus of the embodiment as described, or can be modified to be located in one or more apparatuses different from this embodiment. The modules of the above-described embodiment can be combined into one module, or further divided into multiple sub-modules.

[0094] The serial numbers in this application are for descriptive purposes only and do not represent the superiority or inferiority of any particular implementation scenario. The above disclosures are merely a few specific implementation scenarios of this application; however, this application is not limited thereto, and any variations conceived by those skilled in the art should fall within the protection scope of this application.

Claims

1. A method for obstacle identification in mining areas for edge device deployment, characterized in that, include: The target point cloud frame is determined based on the point cloud frame collected by the preset lidar, and the target point cloud frame is processed using the preset processing tool to obtain the target features; The target features are input into a preset recognition model with multiple detection heads to obtain first information corresponding to the target point cloud frame. The first information includes obstacle attribute information, obstacle position information, obstacle size information and obstacle heading information corresponding to different obstacles. Based on the first information and the historical first information, the second information corresponding to the target point cloud frame is determined, and the second information includes obstacle movement speed information; Output the first information and the second information; The step of inputting the target features into a preset recognition model with multiple detection heads to obtain first information corresponding to the target point cloud frame includes: The target features are input into the preset recognition model to determine the scale of any obstacle in the target features. Based on the scale, the target detection head corresponding to any obstacle is determined from the plurality of detection heads, and the first sub-information corresponding to any obstacle is obtained based on the target detection head. The first information is obtained based on the first sub-information corresponding to the different obstacles; The process of processing the target point cloud frame using a preset processing tool to obtain target features includes: The target point cloud frame is divided into grids using the preset processing tool; After the partitioning is completed, the target kernel function calls multiple idle threads to extract the original features of the corresponding point cloud points in each grid. The target features are obtained based on the original features of the corresponding point cloud points within all grids.

2. The method according to claim 1, characterized in that, The preset lidar includes a first lidar and a second lidar; the determination of the target point cloud frame based on the point cloud frame acquired by the preset lidar includes: Receives a first target point cloud frame transmitted by a first radar set at a first target location in the mining equipment, and a second target point cloud frame transmitted by a second radar set at a second target location; The first target point cloud frame and the second target point cloud frame are superimposed to obtain the target point cloud frame.

3. The method according to claim 2, characterized in that, Before receiving the first target point cloud frame transmitted by the first radar set at the first target location of the mining equipment, the method further includes: The system receives a first sample point cloud frame transmitted by a first radar set at a first target location in the mining equipment, and a second sample point cloud frame transmitted by a second radar set at a second target location. The first sample point cloud frame and the second sample point cloud frame include a point cloud set corresponding to the same marker. Extract the first point cloud set of the marker in the first sample point cloud frame and the second point cloud set of the marker in the second sample point cloud frame, respectively; The first point cloud set and the second point cloud set are registered to obtain a coordinate transformation matrix; Accordingly, the step of overlaying the first target point cloud frame and the second target point cloud frame to obtain the target point cloud frame includes: Using the coordinate transformation matrix, the first target point cloud frame and the second target point cloud frame are transformed by coordinate transformation, and then superimposed after coordinate transformation to obtain the target point cloud frame.

4. The method according to claim 1, characterized in that, The step of determining the second information corresponding to the target point cloud frame based on the first information and historical first information includes: Based on the first information, the historical first information, and the Hungarian algorithm, the different obstacles are associated to obtain association information; The second information is obtained based on the associated information, the first information, the historical first information, and the Kalman filter method.

5. The method according to claim 1, characterized in that, The first information also includes detection box information; before determining the second information corresponding to the target point cloud frame based on the first information and historical first information, the method further includes: Based on the detection box information in the first information, invalid detection boxes and overlapping detection boxes are identified, and the detection box information corresponding to the invalid detection boxes and overlapping detection boxes is removed from the first information.

6. A mine obstacle recognition device for edge device deployment, characterized in that, include: The point cloud frame determination module is used to determine the target point cloud frame based on the point cloud frame collected by the preset lidar, and to process the target point cloud frame using a preset processing tool to obtain target features; The image input module is used to input the target features into a preset recognition model with multiple detection heads to obtain first information corresponding to the target point cloud frame. The first information includes obstacle attribute information, obstacle position information, obstacle size information and obstacle heading information corresponding to different obstacles. An information determination module is used to determine second information corresponding to the target point cloud frame based on the first information and historical first information, wherein the second information includes obstacle movement speed information; The information output module is used to output the first information and the second information; The image input module is used for: The target features are input into the preset recognition model to determine the scale of any obstacle in the target features. Based on the scale, a target detection head corresponding to any obstacle is determined from the plurality of detection heads, and first sub-information corresponding to any obstacle is obtained based on the target detection head. Based on the first sub-information corresponding to different obstacles, the first information is obtained. The point cloud frame determination module is used for: The target point cloud frame is divided into grids using the preset processing tool; after the division is completed, multiple idle threads are called through the target kernel function to extract the original features of the corresponding point cloud points in each grid; The target features are obtained based on the original features of the corresponding point cloud points within all grids.

7. An autonomous mining truck, characterized in that, include: The obstacle recognition device for mining areas deployed for edge devices as described in claim 6.

8. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 5.

9. A computer device, comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1 to 5.

Images