Mine road boundary detection method, device, medium and product

Abstract

The application provides a mine road boundary detection method, device, medium and product, and relates to the field of edge detection. The method comprises the following steps: collecting first point cloud data of a mine road, and obtaining a feature image based on height fusion features according to the first point cloud data; wherein the feature values of the feature image fuse height information of the point cloud data; inputting the feature image into an image segmentation model to obtain a mine boundary image; and processing the mine boundary image according to a maximum inscribed circle algorithm to obtain a mine road boundary. The application can improve the accuracy of boundary detection of the mine road.

Description

Technical Field

[0001] This application relates to the field of edge detection, and in particular to a method, equipment, medium and product for detecting the boundary of a road in a mining area. Background Technology

[0002] Multiple operational activities, such as mining, transportation, and maintenance, occur within the mining area, and the overlap between these activities can pose a threat to road safety. Vehicles traveling on such roads face significant safety hazards; therefore, accurately identifying the boundaries of roads in open-pit mines is crucial.

[0003] Unlike structured roads, open-pit mines are typical unstructured roads, characterized by complex terrain, harsh environment, and irregular road boundaries. Due to the complexity of the roads, the accuracy of existing boundary detection methods is not high.

[0004] To address the aforementioned shortcomings, there is an urgent need for a method, equipment, medium, and product for detecting the boundaries of roads in mining areas, which can improve the accuracy of boundary detection for roads in mining areas. Summary of the Invention

[0005] This application provides a method, equipment, medium, and product for detecting the boundary of roads in mining areas, which can improve the accuracy of boundary detection for roads in mining areas.

[0006] Firstly, this application provides a method for detecting the boundary of a road in a mining area, comprising:

[0007] First point cloud data of the mining area road is collected, and a feature image based on high-level fusion features is obtained based on the first point cloud data; wherein, the feature value of the feature image is fused with the height information of the point cloud data;

[0008] The feature image is input into the image segmentation model to obtain the boundary image of the mining area;

[0009] The boundary image of the mining area is processed using the maximum inscribed circle algorithm to obtain the road boundary of the mining area.

[0010] In one possible design, the step of processing the mining area boundary image using the maximum inscribed circle algorithm to obtain the mining area road boundary includes:

[0011] Using the boundary image of the mining area as the current segmentation image, the maximum inscribed circle of the current segmentation image is obtained according to a preset starting point;

[0012] The current segmented image is segmented according to the maximum inscribed circle segmentation to obtain a new current segmented image; a new maximum inscribed circle is obtained according to the new current segmented image; the new maximum inscribed circle is tangent to the maximum inscribed circle.

[0013] Repeat the steps of segmenting the current segmented image based on the maximum inscribed circle until the diameter of the new maximum inscribed circle is less than a preset value;

[0014] The boundaries of the mining area road are obtained by connecting the centers of the largest inscribed circles in the order of their acquisition time.

[0015] In one possible design, obtaining a feature image based on highly fused features from the first point cloud data includes:

[0016] Transform the first point cloud data into the second point cloud data in the world coordinate system;

[0017] The planar coordinate information in the second point cloud data is processed into a grid matrix.

[0018] Each grid in the grid matrix is ​​treated as a pixel; the height information in the grid is fused to obtain a height fusion feature value, and the height fusion feature value is used as the pixel value of the pixel.

[0019] In one possible design, the process of fusing the height information in the grille to obtain a height fusion feature value includes:

[0020] For each of the grids, the height information of the second point cloud data in the grid is obtained, and the maximum height, maximum height difference, and average height are obtained from the height information;

[0021] The height fusion feature value is obtained by weighting the maximum height value, the maximum height difference value, and the average height value.

[0022] In one possible design, the first point cloud data is point cloud data collected by a lidar sensor on a mining truck, and the conversion of the first point cloud data into second point cloud data in a world coordinate system includes:

[0023] The first point cloud data is transformed into third point cloud data according to a preset rotation matrix and a preset translation matrix; wherein, the preset rotation matrix and the preset translation matrix are the rotation matrix and translation matrix between the lidar and the rear axle center of the vehicle body, and the third point cloud data is the point cloud data in the mine car coordinate system;

[0024] The location information in world coordinates is obtained based on the positioning information of the mining truck; the third point cloud data is converted into second point cloud information based on the location information; wherein the location information includes a first offset relative to the central meridian, a second offset relative to the equator, and a yaw angle.

[0025] In one possible design, after obtaining the boundary of the mining area road, the method further includes:

[0026] The boundary of the mining area road is mapped onto the second point cloud data to obtain a three-dimensional boundary based on the point cloud data.

[0027] Secondly, this application provides a mining area road boundary detection device, the device comprising:

[0028] The first module is used to collect first point cloud data of roads in the mining area, and to obtain a feature image based on high-level fusion features based on the first point cloud data; wherein, the feature value of the feature image is fused with the height information of the point cloud data;

[0029] The second module is used to input the feature image into the image segmentation model to obtain the mining area boundary image;

[0030] The third module is used to process the mining area boundary image according to the maximum inscribed circle algorithm to obtain the mining area road boundary.

[0031] Thirdly, this application provides an electronic device, including: a processor, and a memory communicatively connected to the processor;

[0032] The memory stores computer-executed instructions;

[0033] The processor executes computer execution instructions stored in the memory to implement the above-described method for detecting road boundaries in mining areas.

[0034] Fourthly, this application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the above-described method for detecting the boundary of a mining area road.

[0035] This application provides a method, device, medium, and product for detecting the boundary of a mining area road, comprising: collecting first point cloud data of the mining area road; obtaining a feature image based on high-level fusion features from the first point cloud data; wherein the feature values ​​of the feature image fuse the height information of the point cloud data; inputting the feature image into the image segmentation model to obtain a mining area boundary image; and processing the mining area boundary image according to the maximum inscribed circle algorithm to obtain the mining area road boundary; achieving the following technical effects:

[0036] It can combine image segmentation models and the maximum inscribed circle algorithm to determine the boundaries of mining roads based on point cloud data, thereby improving the accuracy of mining boundary detection and ensuring the safety of mining operations. Attached Figure Description

[0037] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0038] Figure 1 A flowchart illustrating a method for detecting road boundaries in a mining area, as provided in this application embodiment. Figure 1 ;

[0039] Figure 2 A flowchart illustrating a method for detecting road boundaries in a mining area, as provided in this application embodiment. Figure 2 ;

[0040] Figure 3 A flowchart illustrating step S206 provided in an embodiment of this application;

[0041] Figure 4 This is a schematic diagram of a mining area boundary image obtained according to an embodiment of this application;

[0042] Figure 5 This is a schematic diagram of multiple maximum inscribed circles obtained from a mining area boundary image obtained according to an embodiment of this application;

[0043] Figure 6 This is a schematic diagram of the boundary of a mining area road obtained according to an embodiment of this application;

[0044] Figure 7 This is a schematic diagram of the structure of a mining area road boundary detection device provided in an embodiment of this application;

[0045] Figure 8 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application.

[0046] The accompanying drawings have illustrated specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to specific embodiments. Detailed Implementation

[0047] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of devices and methods consistent with some aspects of this application as detailed in the appended claims, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without inventive effort are within the scope of protection of this invention.

[0048] It should be noted that, in the embodiments of this application, the terms "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design scheme described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0049] It should be noted that the terms "first," "second," "third," "fourth," etc. (if present) in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented, for example, in orders other than those illustrated or described herein.

[0050] It should be noted that "at the time of..." in the embodiments of this application can be either at the instant when a certain situation occurs, or for a period of time after the occurrence of a certain situation. The embodiments of this application do not make specific limitations on this.

[0051] This application provides a method for detecting the boundary of roads in mining areas, which aims to solve the problem of boundary detection accuracy in the prior art.

[0052] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will be described below with reference to the accompanying drawings.

[0053] Figure 1 A flowchart illustrating a method for detecting road boundaries in a mining area, as provided in this application embodiment. Figure 1 ;like Figure 1 As shown, the method includes:

[0054] S101. Collect the first point cloud data of the mining area road, and obtain a feature image based on the high-level fusion feature based on the first point cloud data; wherein, the feature value of the feature image fuses the height information of the point cloud data.

[0055] Specifically, the first point cloud data is obtained based on the point cloud data collected by the lidar on the mining truck during its operation;

[0056] As one embodiment, the first point cloud data is the point cloud data collected by the lidar on the mining truck. After the first point cloud data is converted into the second point cloud data in the world coordinate system, feature image conversion is performed to obtain the feature image. The feature value corresponding to the feature image is the feature value obtained by fusing the height information of the second point cloud data.

[0057] Specifically, the mining trucks can be autonomous mining trucks; the first point of cloud data can be collected through one or more mining trucks.

[0058] S102. Input the feature image into the image segmentation model to obtain the boundary image of the mining area;

[0059] Specifically, the image segmentation model is obtained by training a preset model based on a training set. The samples in the training set are feature images and corresponding label sequences. The labels in the label sequences are used to indicate whether the corresponding pixel is a mining area boundary or a non-mining area boundary.

[0060] Specifically, the preset model is a deep learning model based on image semantic segmentation.

[0061] S103. Process the mining area boundary image using the maximum inscribed circle algorithm to obtain the mining area road boundary.

[0062] Specifically, multiple largest inscribed circles that are sequentially tangent to each other and have a diameter not less than a preset value are obtained from the boundary image of the mining area, and the center of each circle is connected sequentially to obtain the boundary of the road in the mining area.

[0063] The method provided in this application embodiment collects first point cloud data of mining area roads, and obtains a feature image based on high-level fusion features from the first point cloud data; wherein, the feature values ​​of the feature image fuse the height information of the point cloud data; the feature image is input into an image segmentation model to obtain a mining area boundary image; the mining area boundary image is processed according to the maximum inscribed circle algorithm to obtain the line connecting the centers of multiple inscribed circles as the boundary of the mining area roads. This method can achieve the following technical effects:

[0064] The method for detecting road boundaries in mining areas according to the embodiments of this application can combine image segmentation models and the maximum inscribed circle algorithm to determine the road boundaries in mining areas based on point cloud data, thereby improving the accuracy of mining area boundary detection and ensuring the safety of mining operations.

[0065] Figure 2A flowchart illustrating a method for detecting road boundaries in a mining area, as provided in this application embodiment. Figure 2 ;like Figure 2 As shown, the method includes:

[0066] S201. Collect the first point cloud data of the mining area road, and transform the first point cloud data into the third point cloud data according to the preset rotation matrix and preset translation matrix; wherein, the preset rotation matrix and preset translation matrix are the rotation matrix and translation matrix between the lidar and the rear axle center of the vehicle body, and the third point cloud data is the point cloud data in the coordinate system of the mining vehicle.

[0067] Specifically, the first point cloud data is obtained by scanning the road information on the left and right sides of the direction of travel of the mining truck using a lidar installed on the front side of the vehicle. The first point cloud data includes the x, y, and z values ​​of each point in the three-dimensional point cloud in the lidar coordinate system, where the positive x-axis is the direction of travel of the vehicle, the positive y-axis is the left side of the direction of travel of the vehicle, and the positive z-axis is vertically upward.

[0068] Specifically, the first point of cloud data can be collected using one or more mining trucks;

[0069] As one embodiment, a preset rotation matrix and a preset translation matrix are determined based on the rotation matrix R1 and translation matrix T1 between the lidar and the center of the rear axle of the mining truck; then the third point cloud data can be obtained according to the following formula:

[0070] P vehicle = R1 × P lidar + T1 (1)

[0071] Where P vehicle This represents the third point cloud data, i.e., the point cloud in the minecart coordinate system, P. lidar This represents the first point cloud data, i.e., the point cloud in the lidar coordinate system.

[0072] S202. Obtain the position information in world coordinates based on the positioning information of the mining truck; convert the third point cloud data into second point cloud information based on the position information; the position information includes the first offset relative to the central meridian, the second offset relative to the equator, and the yaw angle;

[0073] Specifically, the positioning information of the mine car is obtained through a positioning antenna installed above the driver's cab and a positioning box inside the cab. The mine car's position in the UTM world coordinate system is obtained by using the center of its rear axle as the positioning center and conducting a figure-eight circling test.

[0074] Specifically, the location information mainly includes the utm_east, utm_north, and yaw values ​​of the minecart in the UTM world coordinate system. UTM_east represents the first offset of the position relative to the central meridian, with east being positive; utm_north represents the second offset of the position relative to the equator, with north being positive; and yaw represents the heading angle, with true north as the reference.

[0075] As one embodiment, the second point cloud data can be obtained according to formulas (2) to (4):

[0076] x q = (x vehicle ×sinyaw)- (y vehicle ×cosyaw)+utm_east (2)

[0077] y q =(x vehicle ×cosyaw)+(y vehicle ×sinyaw)+utm_north(3)

[0078] z q = z vehicle (4)

[0079] Where, x q This represents the x-value of the point cloud in the world coordinate system. vehicle Represents the x-values ​​and y-values ​​of the point cloud in the minecart coordinate system. q This represents the y-value of the point cloud in the world coordinate system. vehicle This represents the y-value and z-value of the point cloud in the minecart coordinate system. q This represents the z-value of a point cloud in world coordinates. vehicle This represents the z-value of the point cloud in the mine car coordinate system.

[0080] Specifically, the first point cloud data is converted into point cloud data in the world coordinate system as the second point cloud data. The second point cloud data is more in line with people's understanding of maps and is easier to process.

[0081] Furthermore, if multiple mining trucks are used to collect the first point cloud data, and then the data is uniformly converted into the second point cloud data in the world coordinate system before being stitched and fused, using multiple mining trucks for collection and fusion can be faster or can obtain more mining area data.

[0082] Furthermore, for ease of calculation, x can be... q -x1 is used as the x-value of the second point cloud data, and y q-y1 is the y value of the second point cloud data, where x1 represents the x value of the origin of the mining area in the world coordinate system, and y1 represents the y value of the origin of the mining area in the world coordinate system; the obtained second point cloud data is the coordinate data with the left point (x1, y1) as the origin.

[0083] S203. Perform gridding processing on the planar coordinate information in the second point cloud data to obtain the grid matrix;

[0084] Specifically, the second point cloud data is rasterized according to a preset raster size 'a' (length and width of the raster). Then, the second point cloud data is mapped to a raster matrix (column, row), and the values ​​of column and row are:

[0085]

[0086]

[0087] The raster matrix uses a Cartesian coordinate system with the bottom left corner as the origin. The positive direction of the column axis is the direction of the matrix extending to the right, and the positive direction of the row axis is the direction of the matrix extending upwards. In the formula, `round` represents a rounding operation, rounding the coordinate transformation result to ensure that continuous floating-point coordinates (x, y) are transformed to discrete integer raster coordinates (column, row). In the formula, x... min Represents the minimum value in the x-axis direction of point cloud data, y min 'a' represents the minimum value along the y-axis in the second point cloud data, and 'a' represents the length and width of each grid in the raster matrix.

[0088] S204. Obtain feature images by highly fusion processing the second point cloud data corresponding to each grid in the grid matrix;

[0089] Specifically, point cloud data is divided into grids, where each grid includes one or more point clouds; each grid corresponds to a pixel in the feature image.

[0090] Specifically, for each grid, the height information of the second point cloud data in the grid is obtained, and the maximum height, maximum height difference, and average height are obtained. The height fusion feature value is obtained based on the weighted average of the maximum height, maximum height difference, and average height. The grid matrix based on the height fusion feature value is then used as a feature map.

[0091] As one embodiment, the height fusion feature value corresponding to each grid is represented by the following formula:

[0092] δ z = m×Z max +n×ΔZ+c×Z avg (7)

[0093] Where δ z This represents the height feature of the second point cloud data within this raster, with a value ranging from 0 to 255. m, n, and c are weighting coefficients, and m + n + c = 1. The weighting coefficients can be set according to specific circumstances. Z max ΔZ represents the maximum value of the second point cloud data within this grid along the Z-axis of the mining area coordinate system, and ΔZ represents the maximum difference between the point cloud clusters within this grid along the Z-axis of the mining area coordinate system. avg This represents the average value of the point cloud cluster within this grid along the Z-axis of the mining area coordinate system.

[0094] S205. Input the feature image into the image segmentation model to obtain the mining area boundary image; the image segmentation model is obtained by training a preset model based on the training set. The samples in the training set are feature images and the corresponding label sequences of the feature images. The labels in the label sequences are used to indicate whether the corresponding pixels are mining area boundaries or non-mining area boundaries.

[0095] Specifically, the preset model is trained based on the training set, and the image segmentation model is obtained after training is completed; the feature image is input into the image segmentation model to obtain the output mining area boundary image;

[0096] Specifically, the training set includes multiple samples, each of which includes a feature image and a corresponding label sequence or label matrix;

[0097] Specifically, the feature images in the sample are feature images obtained according to S201 to S204. Each pixel in the feature image corresponds to a label, which is used to indicate whether the pixel is a pixel on the boundary of the mining area or a pixel on the boundary of a non-mining area, and can be represented by 0 or 1.

[0098] As one example, the preset model is a deep learning model based on image semantic segmentation.

[0099] S206. The boundary image of the mining area is processed according to the maximum inscribed circle algorithm to obtain the line connecting the centers of multiple inscribed circles as the boundary of the mining area road.

[0100] Specifically, Figure 3 A flowchart of step S206 provided in an embodiment of this application; as shown Figure 3 As shown, S206 includes:

[0101] S2061. Using the mining area boundary image as the current segmentation image, obtain the maximum inscribed circle of the current segmentation image based on the preset starting point;

[0102] Specifically, its preset starting point is on the largest inscribed circle;

[0103] S2062. Segment the current segmented image according to the maximum inscribed circle segmentation to obtain a new current segmented image; obtain a new maximum inscribed circle according to the new current segmented image; the new maximum inscribed circle is tangent to the maximum inscribed circle.

[0104] Specifically, the current segmented image is segmented based on the edge of the obtained largest inscribed circle. Among the obtained sub-images, the sub-image farthest from the preset starting point is taken as the new current segmented image.

[0105] Specifically, the inscribed circle in the current segmented image that is tangent to the previous largest inscribed circle and has the largest diameter is taken as the new largest inscribed circle, and the diameter of the largest inscribed circle is not less than a preset value.

[0106] Specifically, the method for obtaining the maximum inscribed circle can be implemented using the inner_circle operator in vision software.

[0107] S2063. Repeat the step of segmenting the current image based on the maximum inscribed circle until the diameter of the new maximum inscribed circle is less than the preset value.

[0108] Specifically, after obtaining the new maximum inscribed circle, it is determined whether the diameter of the maximum inscribed circle is less than the preset value. If it is not less than the preset value, return to S2062; if it is less than the preset value, execute step S2064.

[0109] S2064. Connect the centers of the circles in sequence according to the acquisition time of the largest inscribed circle to obtain the boundary of the mining area road.

[0110] Specifically, the polyline containing the center of all the largest inscribed circles with diameters not less than the preset value is the road boundary of the mining area.

[0111] As one example, Figure 4 This is a schematic diagram of a mining area boundary image obtained according to an embodiment of this application; Figure 5 This is a schematic diagram of multiple maximum inscribed circles obtained from a mining area boundary image obtained according to an embodiment of this application; Figure 6 This is a schematic diagram of the boundary of a mining area road obtained according to an embodiment of this application.

[0112] S207. Map the boundary of the mining area road onto the second point cloud data to obtain the three-dimensional boundary based on the point cloud data.

[0113] Specifically, the precise location information of the mining area road boundary is obtained based on the second point of cloud data.

[0114] The mining area road boundary detection method provided in this application can achieve the following technical effects:

[0115] Using lidar technology to acquire ground point cloud data, lidar is unaffected by factors such as lighting and weather, and can provide stable position and shape information under various conditions;

[0116] This application uses a feature image that integrates highly distinctive features as the feature image for image segmentation, which is more in line with the basic characteristics of the mining area, avoids feature redundancy, and improves the accuracy of model segmentation.

[0117] The maximum inscribed circle algorithm is used to further determine the edge of the mining area, improve the accuracy of boundary detection, and effectively enhance the safety of mining operations.

[0118] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily essential to this application.

[0119] It should be further noted that although the steps in the flowchart are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowchart may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0120] In this embodiment of the invention, electronic devices or main control devices can be divided into functional modules according to the above method examples. For example, each function can be divided into its own functional modules, or two or more functions can be integrated into one processing unit. The integrated unit can be implemented in hardware or as a software functional module. It should be noted that the module division in this embodiment of the invention is illustrative and only represents one logical functional division; other division methods may be used in actual implementation.

[0121] Figure 7 This application provides a schematic diagram of the structure of a mining area road boundary detection device according to an embodiment of the present application; as shown below. Figure 7 As shown, device 70 includes:

[0122] The first module 701 is used to collect first point cloud data of the mining area road and obtain a feature image based on high-level fusion features based on the first point cloud data; wherein, the feature value of the feature image is fused with the height information of the point cloud data.

[0123] The second module 702 is used to input the feature image into the image segmentation model to obtain the boundary image of the mining area;

[0124] The third module 703 is used to process the mining area boundary image according to the maximum inscribed circle algorithm to obtain the mining area road boundary.

[0125] Furthermore, the third module 703 is specifically used for:

[0126] Use the boundary image of the mining area as the current segmentation image, and obtain the maximum inscribed circle of the current segmentation image based on the preset starting point;

[0127] The current segmented image is segmented based on the maximum inscribed circle to obtain a new current segmented image; a new maximum inscribed circle is obtained based on the new current segmented image; the new maximum inscribed circle is tangent to the maximum inscribed circle.

[0128] Repeat the step of segmenting the current image based on the maximum inscribed circle until the diameter of the new maximum inscribed circle is smaller than the preset value;

[0129] The boundaries of the mining area roads are obtained by connecting the centers of the largest inscribed circles in the order of their acquisition time.

[0130] Furthermore, the first module 701 is specifically used for:

[0131] Transform the first point cloud data into the second point cloud data in the world coordinate system;

[0132] The planar coordinate information in the second point cloud data is processed into a grid matrix to obtain the grid matrix.

[0133] Each grid in the grid matrix is ​​treated as a pixel; the height information in the grid is fused to obtain the height fusion feature value, and the height fusion feature value is used as the pixel value of the pixel.

[0134] Furthermore, the first module 701 is specifically used for:

[0135] For each grid, obtain the height information of the second point cloud data in the grid, and obtain the maximum height, maximum height difference and average height.

[0136] The height fusion feature value is obtained by weighting the maximum height, the maximum height difference, and the average height.

[0137] Furthermore, the first module 701 is specifically used for:

[0138] The first point cloud data is transformed into the third point cloud data according to the preset rotation matrix and preset translation matrix; wherein, the preset rotation matrix and preset translation matrix are the rotation matrix and translation matrix between the lidar and the rear axle center of the vehicle body, and the third point cloud data is the point cloud data in the mine car coordinate system;

[0139] The location information in world coordinates is obtained based on the positioning information of the mining truck; the third point cloud data is converted into the second point cloud information based on the location information; the location information includes the first offset relative to the central meridian, the second offset relative to the equator, and the yaw angle.

[0140] Furthermore, the third module 703 is specifically used for:

[0141] The boundaries of the mining area roads are mapped onto the second point cloud data to obtain the three-dimensional boundaries based on the point cloud data.

[0142] The mining area road boundary detection device provided in this embodiment can perform the above-mentioned mining area road boundary detection method. Its implementation principle and technical effect are similar, and will not be described again in this embodiment.

[0143] In the aforementioned specific implementation of a mining area road boundary detection device, each module can be implemented as a processor. The processor can execute computer execution instructions stored in the memory, thereby enabling the processor to execute the aforementioned mining area road boundary detection method.

[0144] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 8 As shown, the electronic device 80 includes at least one processor 801 and a memory 802. The electronic device 80 also includes a communication component 803. The processor 801, memory 802, and communication component 803 are connected via a bus 804.

[0145] In the specific implementation process, at least one processor 801 executes computer execution instructions stored in memory 802, causing at least one processor 801 to execute a mining area road boundary detection method as executed by the above electronic device.

[0146] The specific implementation process of processor 801 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0147] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0148] The memory may include high-speed RAM, and may also include non-volatile storage (NVM), such as at least one disk storage.

[0149] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0150] The above description of the functions implemented by electronic devices and main control devices has introduced the solutions provided by the embodiments of the present invention. It is understood that, in order to implement the above functions, the electronic device or main control device includes hardware structures and / or software modules corresponding to the execution of each function. By combining the units and algorithm steps of the various examples described in the embodiments of the present invention, the embodiments of the present invention can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed by hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the technical solutions of the embodiments of the present invention.

[0151] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-described method for detecting road boundaries in mining areas.

[0152] The aforementioned computer-readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0153] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in an electronic device or a host device.

[0154] This application also provides a computer program product, comprising: a computer program stored in a readable storage medium, wherein at least one processor of an electronic device can read the computer program from the readable storage medium, and the at least one processor executes the computer program to cause the electronic device to perform the scheme provided in any of the above embodiments.

[0155] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0156] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting road boundaries in mining areas, characterized in that, include: First point cloud data of the mining area road is collected, and a feature image based on high-level fusion features is obtained based on the first point cloud data; wherein, the feature value of the feature image is fused with the height information of the point cloud data; The feature image is input into an image segmentation model to obtain the boundary image of the mining area; Using the boundary image of the mining area as the current segmentation image, the maximum inscribed circle of the current segmentation image is obtained according to a preset starting point; The current segmented image is segmented according to the maximum inscribed circle segmentation to obtain a new current segmented image; a new maximum inscribed circle is obtained according to the new current segmented image; the new maximum inscribed circle is tangent to the maximum inscribed circle. Repeat the steps of segmenting the current segmented image based on the maximum inscribed circle until the diameter of the new maximum inscribed circle is less than a preset value; The boundaries of the mining area road are obtained by connecting the centers of the largest inscribed circles in the order of their acquisition time.

2. The method according to claim 1, characterized in that, The step of obtaining a feature image based on highly fused features from the first point cloud data includes: Transform the first point cloud data into the second point cloud data in the world coordinate system; The planar coordinate information in the second point cloud data is processed into a grid matrix. Each grid in the grid matrix is ​​treated as a pixel; the height information in the grid is fused to obtain a height fusion feature value, and the height fusion feature value is used as the pixel value of the pixel.

3. The method according to claim 2, characterized in that, The step of fusing the height information in the grille to obtain height fusion feature values ​​includes: For each of the grids, the height information of the second point cloud data in the grid is obtained, and the maximum height, maximum height difference, and average height are obtained from the height information; The height fusion feature value is obtained by weighting the maximum height value, the maximum height difference value, and the average height value.

4. The method according to claim 2, characterized in that, The first point cloud data is point cloud data collected by the lidar on the mining truck. The step of converting the first point cloud data into second point cloud data in the world coordinate system includes: The first point cloud data is transformed into third point cloud data according to a preset rotation matrix and a preset translation matrix; wherein, the preset rotation matrix and the preset translation matrix are the rotation matrix and translation matrix between the lidar and the rear axle center of the vehicle body, and the third point cloud data is the point cloud data in the mine car coordinate system; The location information in world coordinates is obtained based on the positioning information of the mining truck; the third point cloud data is converted into second point cloud information based on the location information; wherein the location information includes a first offset relative to the central meridian, a second offset relative to the equator, and a yaw angle.

5. The method according to claim 2, characterized in that, After obtaining the road boundaries of the mining area, the method further includes: The boundary of the mining area road is mapped onto the second point cloud data to obtain a three-dimensional boundary based on the point cloud data.

6. A mining area road boundary detection device, characterized in that, include: The first module is used to collect first point cloud data of roads in the mining area, and to obtain a feature image based on high-level fusion features based on the first point cloud data; wherein, the feature value of the feature image is fused with the height information of the point cloud data; The second module is used to input the feature image into an image segmentation model to obtain the boundary image of the mining area. The third module is used to take the mining area boundary image as the current segmentation image and obtain the maximum inscribed circle of the current segmentation image according to a preset starting point. The current segmented image is segmented according to the maximum inscribed circle segmentation to obtain a new current segmented image; a new maximum inscribed circle is obtained according to the new current segmented image; the new maximum inscribed circle is tangent to the maximum inscribed circle. Repeat the steps of segmenting the current segmented image based on the maximum inscribed circle until the diameter of the new maximum inscribed circle is less than a preset value; The boundaries of the mining area road are obtained by connecting the centers of the largest inscribed circles in the order of their acquisition time.

7. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1 to 5.

9. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the method as described in any one of claims 1 to 5.

Images