Sparse perception-based strip mine area online mapping and boundary line fitting method

Abstract

The invention relates to an open-pit mine area online mapping and boundary line fitting method based on sparse perception, belongs to the technical field of open-pit mine area automatic driving, and solves the problems that in the prior art, a step-by-step processing mode is used in an open-pit mine area environment perception method, information transmission loss exists, and environment perception is incomplete. Extracting a multi-scale feature map; s2, decoding the multi-scale feature map to obtain a dynamic instance and a static instance; s3, constructing a local vector map, and dynamically updating the local vector map; s4, key points of road boundaries are extracted from the local vector map, and matching point pairs meeting matching conditions are obtained; s5, carrying out breadth-first search on the matching point pairs to obtain an initial road boundary sequence; s6, screening the obtained initial road boundary sequence to obtain a screened road boundary sequence; and S7, re-projecting the screened road boundary sequence back to the original image to obtain a fitting road boundary.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving technology in open-pit mines, specifically to an online mapping and boundary line fitting method for open-pit mines based on sparse perception, which is used to solve the problems of track area identification and path planning in complex mining environments. Background Technology

[0002] Open-pit mine environments are characterized by unstructured roads, undulating terrain, diverse obstacles, and drastic changes in lighting. Traditional autonomous driving systems that rely on high-precision maps struggle to adapt to these dynamically changing environments. Existing mine perception systems often employ a modular design, such as first detecting obstacles, then constructing a local map, and finally planning the path. This step-by-step approach suffers from information loss during inter-module transmission, resulting in incomplete environmental perception; errors accumulate between modules, affecting the accuracy of the final planning result; the computational load is high, making it difficult to meet the real-time requirements of high-speed vehicle operation in mine areas; and it exhibits poor adaptability to complex scenarios such as multi-track intersections and dynamic obstacles.

[0003] In existing technologies, such as Chinese invention patent application CN110472527A, entitled "A Track Line Detection Method and System," a window-swiping detection method is used to fit a single track line, which cannot handle multi-track intersection scenarios. Chinese invention patent application CN110501719A, entitled "A Train Obstacle Detection Method Based on LiDAR," uses LiDAR for forward track detection, but the equipment is costly and has stringent environmental requirements.

[0004] Therefore, there is a need in this field for improved road boundary line fitting methods to achieve efficient and accurate track area identification and path planning in complex mining environments. Summary of the Invention

[0005] In view of the above problems, this invention provides an online mapping and boundary line fitting method for open-pit mines based on sparse sensing. Addressing the issues of error accumulation caused by the separation of mapping and boundary detection in existing mine road boundary line fitting methods, inaccurate fitting of boundary lines for multiple intersecting roads in complex scenarios, untimely online map updates under dynamic obstacle interference, and high system computational load, this invention proposes an online mapping and road boundary line fitting method suitable for the complex environment of open-pit mines. The integrated online mapping and track line fitting method based on sparse sensing proposed in this invention can significantly improve system efficiency and robustness while maintaining accuracy.

[0006] According to an embodiment of the present invention, an online mapping and boundary line fitting method for open-pit mines based on sparse sensing is provided, comprising: S1: Acquire multi-view mining area scene images as raw images, and extract multi-scale feature maps through an image encoder; S2: Establish a sparse perception module to uniformly decode multi-scale feature maps, and decode dynamic and static instances in 3D space from multi-scale feature maps through a deformable attention mechanism; S3: Construct a local vector map in the bird's-eye view coordinate system based on static and dynamic instances, and realize the dynamic update of the local vector map through temporal fusion; S4: Extract key points of road boundaries from the local vector map, and perform left and right boundary matching on the key points according to preset distance thresholds and slope thresholds to obtain matching point pairs that meet the matching conditions; S5: Perform a breadth-first search on the matching point pairs to obtain the initial road boundary sequence; S6: Filter the obtained initial road boundary sequence, remove road boundary sequences that do not meet the filtering conditions, and obtain the filtered road boundary sequence; S7: Reproject the filtered road boundary sequence back into the original image. Fit the key points in each road boundary sequence using the least squares method to obtain the fitted road boundary, and output it for subsequent path planning.

[0007] Optionally, in S1: multiple industrial cameras are used to collect images of the mining area environment. The industrial cameras are respectively deployed at the front, right front, left front, right rear, left rear and front rear of the vehicle for multi-view consistency verification.

[0008] Optionally, in S2: Dynamic instances are the state information of dynamic targets, including the 3D coordinates, motion attributes, target category and related confidence scores of the dynamic targets, where dynamic targets include mining vehicles and equipment; Static instances consist of polylines composed of multiple control points, used to represent map elements including road boundaries and parking areas, serving as the basis for boundary line fitting. This includes the 3D coordinates of all control points as the 3D coordinates of the static instance.

[0009] Optionally, in S3: Projecting the 3D coordinates of static instances onto the bird's-eye view coordinate plane from above creates a representation of road boundaries in the bird's-eye view space. Mark the area containing the 3D coordinates of dynamic targets in dynamic instances as a temporary obstacle zone, and filter out map elements located within this temporary obstacle zone from static instances.

[0010] Optionally, dynamic updates of local vector maps in S3 include: Set the length of the sequential memory queue to the number of frames; In each frame of the temporal memory queue, the detection confidence of the map element detected in real time is judged. When the confidence is greater than a set threshold, the map element is added to the current local vector map. The system performs position smoothing calculations on map elements that persist in the current local vector map, updates the positions of map elements in the local vector map, and achieves dynamic updates of the local vector map.

[0011] Optionally, in S3: Set the confidence threshold to 0.7 and perform position smoothing calculations on persistent map elements:

[0012] in, This indicates the current position of a map element in the temporal memory queue. This indicates the location of the newly detected map element in the current frame. This indicates the new location of the map element on the map after the update.

[0013] Optionally, S4 includes: In the bird's-eye view of the local vector map, extract a row of road boundary key points at set intervals along the direction of the vehicle's movement. Each row of road boundary key points includes a pair of key points on the left and right sides of the road. Perform left and right boundary matching on each row of key points, and retain the key points of each row that meet the matching conditions as matching point pairs; The matching conditions include distance threshold and slope threshold matching. The distance threshold matching condition is as follows:

[0014] in, This is the lower limit for the width of roads in the mining area. This is the upper limit for the width of roads in the mining area. This represents the distance between key points in a row to be matched. The average direction angle of the row containing the key points to be matched; The slope threshold matching condition is:

[0015] in, The preset slope tolerance threshold, This represents the slope of the local polyline segment containing the left boundary keypoint in the row of keypoints to be matched. This represents the slope of the local polyline segment containing the right boundary keypoint in the row of keypoints to be matched.

[0016] Optionally, S5 includes: Start searching from the row of matching points furthest from the vehicle and search towards the vehicle, setting the maximum number of rows for the search depth; Starting from the matching point pair in the farthest row, determine whether the matching point pairs in adjacent rows are connected. The connection conditions include the distance change being less than a set distance threshold and the direction change being less than a set angle threshold. Traverse all matching point pairs, save the matching point pairs that are determined to be connected, and form the initial road boundary sequence.

[0017] Optionally, S6 includes: S6.1: From the initial road boundary sequence, select two candidate road boundary sequences on the same side for comparison. Compare the coordinates of key points at the same row index position in the two candidate road boundary sequences. Key point pairs with different coordinates are considered different points. If the number of different points in the two candidate road boundary sequences is less than a preset threshold, then these two candidate road boundary sequences are determined to be highly repetitive sequences. S6.2: For two road boundary sequences with high repetition, calculate the local slope change at corresponding different points, compare and remove the road boundary sequence with the larger slope change, and keep the road boundary sequence with the smaller slope change. The above operation is performed on all road boundary sequences in the initial road boundary sequence to obtain the filtered road boundary sequence.

[0018] Compared with existing technologies, the online mapping and boundary line fitting method for open-pit mines based on sparse sensing provided by this invention has at least the following beneficial effects.

[0019] (1) To address the problem of separating mapping and boundary detection, a symmetric sparse perception architecture is proposed, unifying boundary detection and online mapping into a single model. This avoids information loss between modules through shared feature representations and interaction mechanisms. The core principle is that a unified sparse perception module directly decodes road boundaries (static instances) and dynamic obstacle information (dynamic instances) represented as 3D control point polylines from image features, achieving tight coupling between perception and geometric modeling. This effectively eliminates intermediate representation conversions and information loss in the traditional step-by-step process, improving boundary recognition accuracy while significantly increasing computational efficiency by avoiding steps such as generating dense BEV feature maps.

[0020] (2) To address the problem of adapting to the dynamic environment of the mining area, a method of separating dynamic and static instances is adopted. Dynamic instances handle moving obstacles to reduce interference, while static instances focus on fixed road boundary elements, improving the robustness of boundary line fitting. The principle of this method is to clearly distinguish and output the two types of instances within the model. In subsequent mapping, the 3D position information of dynamic instances is used to filter out candidate points of static map elements in areas occupied or interfered with by moving obstacles. This suppresses the interference of dynamic targets on the perception of static road structures from the source, making boundary line extraction more stable and reliable in complex scenarios with vehicles passing by.

[0021] (3) To address the issue of road boundary line attribution in multi-road intersection scenarios, a multi-boundary matching mechanism based on distance constraints and slope consistency is proposed. By setting reasonable distance thresholds and slope tolerance ranges, the problem of distinguishing road boundary lines can be effectively solved. The operating logic of this mechanism is to apply dual geometric constraints to the extracted left and right boundary candidate points in the bird's-eye view space: First, the distance between paired points must be within a reasonable road width range after direction correction; second, the local orientation (slope) on both sides of the paired points must be basically parallel. These two constraints together ensure that the successfully matched point pairs conform to the physical and geometric characteristics of real roads, thereby clearly distinguishing the boundaries between the current lane and adjacent lanes or non-road areas, effectively solving the attribution ambiguity in multi-track intersection scenarios.

[0022] (4) To address the issue of timely map updates, a temporal memory queue mechanism is designed to integrate multi-frame perception results and achieve incremental map updates, thereby improving the accuracy and stability of boundary line fitting. The principle is to use a queue that caches historical perception results for several frames to perform smooth incremental Bayesian filtering updates on map elements. New elements are added with high confidence, the positions of existing elements are smoothed using a weighted average of historical and current observations, and missing elements are removed promptly. This approach utilizes temporal information to smooth single-frame detection noise, improving map accuracy and temporal stability while ensuring real-time map performance with low computational overhead. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the embodiments will be briefly introduced below. The features and advantages of the present invention can be more clearly understood by referring to the accompanying drawings. The accompanying drawings are schematic and should not be construed as limiting the present invention in any way. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 A flowchart illustrating an online mapping and boundary line fitting method for open-pit mines based on sparse sensing, according to an embodiment of the present invention. Detailed Implementation

[0025] To better understand the above-mentioned objectives, features, and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments of the present invention and the features thereof can be combined with each other.

[0026] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein. Therefore, the scope of protection of the invention is not limited to the specific embodiments disclosed below.

[0027] The following describes in detail, with reference to the accompanying drawings, an online mapping and boundary line fitting method for open-pit mines based on sparse sensing, according to an embodiment of the present invention.

[0028] like Figure 1 As shown, an online mapping and boundary line fitting method for open-pit mines based on sparse sensing, according to an embodiment of the present invention, includes the following steps.

[0029] S1: Acquire multi-view images of the mining area as raw images, and extract multi-scale feature maps using an image encoder. Optionally, multi-view images of the mining area can be acquired using multiple imaging devices or methods such as cameras, video cameras, and webcams.

[0030] In one example, six 2-megapixel industrial cameras can be used to capture images of the mining environment. These cameras can be positioned directly in front of the vehicle, to the right front, left front, right rear, left rear, and directly behind, respectively, for multi-view consistency verification. The captured images can have a resolution of 1280×720 and a frame rate of 30fps. For example, the image encoder can use ResNet-101 as the backbone network to extract multi-scale feature maps. In this example, the scales of the four extracted multi-scale feature maps can be set to 1 / 4, 1 / 8, 1 / 16, and 1 / 32 of the input image, corresponding to resolutions of 320×180, 160×90, 80×45, and 40×22, respectively. It should be understood that in other implementations, the number of feature maps, resolution, frame rate, scale, etc., can be set separately as needed.

[0031] In this example, the resulting multi-scale feature map is represented as follows: , where the subscript of the symbol For the index of the feature map, Indicates the first Each feature map Indicates the first The height of each feature map, Indicates the first The width of each feature map, This represents the number of feature channels.

[0032] S2: Establish a sparse perception module to uniformly decode multi-scale feature maps. Through a deformable attention mechanism, directly decode dynamic and static instances in 3D space from the multi-scale feature maps, completing the implicit transformation from image space to 3D space. Simultaneously output the obtained dynamic and static instances to provide environmental elements for subsequent mapping and boundary line fitting.

[0033] The sparse perception module takes multi-scale feature maps as input and decodes dynamic and static instances in parallel through a deformable attention mechanism. Dynamic instances are used to identify and track moving obstacles (such as mining vehicles and equipment) that are dynamic targets. Each dynamic instance can include the dynamic target's state information, such as its 3D coordinates, velocity, motion attributes, target category, and related confidence level. By perceiving these dynamic targets in real time, the system can effectively eliminate their interference with road boundary line extraction in subsequent steps, improving the robustness of boundary fitting. Static instances can be used to directly represent map elements such as road boundaries and parking areas, serving as a direct basis for boundary line fitting. Each static instance, as a map element, can include a polyline composed of multiple control points, and can also include the 3D coordinates of all control points as the 3D coordinates of the static instance.

[0034] For example, in one example, the number of dynamic instances can be set. Each dynamic instance represents the state information of a dynamic target. The number of static instances can be set. Each static instance is represented as a polyline using 20 control points. This is used to accurately describe the geometry of road boundaries. It should be understood that the above data is exemplary and can be adjusted and set according to actual needs in practical use.

[0035] S3: Construct a local vector map based on static instances. This local vector map can be constructed in the bird's-eye view coordinate system and dynamically updated through temporal fusion.

[0036] Furthermore, by projecting the 3D coordinates of static instances onto the bird's-eye view coordinate plane, a road boundary representation in the bird's-eye view space can be naturally formed without generating computationally intensive BEV feature maps.

[0037] Furthermore, dynamic updates of the local vector map can be achieved through a temporal memory queue. For example, the length of the temporal memory queue can be set to 8 frames. Optionally, during the update process, in each frame of the temporal memory queue, newly detected map elements can be directly added to the current local vector map if the detection confidence is greater than a set threshold, such as 0.7. Position smoothing calculations can also be performed on persistent map elements in the current local vector map. ,in, This indicates the current position of the map element in the temporal memory queue. This indicates the location of the newly detected map element in the current frame. This indicates the new position of the map element in the local vector map after the update. Based on calculations... This updates the position of the map element in the local vector map, enabling dynamic updates of the local vector map. Furthermore, map elements that have disappeared for more than a set number of frames, such as 3 frames, can be removed from the map to ensure the map's real-time performance.

[0038] Furthermore, in this step, dynamic instances can also be considered when constructing the local vector map. Optionally, the system can also mark the region (such as the bounding box) of the 3D coordinates corresponding to the dynamic target in the dynamic instance as a temporary obstacle area, and filter out map elements located in this region from the static instance to avoid the dynamic target edge being misidentified as a road boundary.

[0039] S4: Extract key points of road boundaries from the local vector map in the constructed bird's-eye view coordinate system. The key points can be matched with the left and right boundaries according to preset distance and slope thresholds to obtain matching point pairs that meet the matching requirements.

[0040] Furthermore, for example, a row of road boundary key points can be extracted every 1 meter along the direction of vehicle travel in the bird's-eye view of the local vector map, and the confidence threshold for the key points can be set to 0.6. Each row of road boundary key points includes a corresponding pair of key points on the left and right sides of the road.

[0041] Next, perform left and right boundary matching on each row of key points. The distance between the matching points must meet the following requirements: ,in , These represent the lower and upper limits of road width in the mining area, indicating the range of road widths within the mining area. This represents the distance between key points in a row to be matched. This represents the average orientation angle of the row of key points to be matched. The slope difference between the left and right boundaries can be set to satisfy... ,in This is the slope tolerance threshold. This represents the slope of the local polyline segment where the key points on the left boundary are located. This represents the slope of the local polyline segment where the key point on the right boundary is located. During the matching process, a 3-point moving average can be used to smooth the slopes of the left and right boundaries.

[0042] S5: Perform a breadth-first search on the matching point pairs obtained in S4 to obtain the initial road boundary sequence.

[0043] In one example, the search starts from the row with the furthest matching point pair and proceeds towards the vehicle, with a maximum search depth of 20 rows. Optionally, starting from the furthest row's matching point pair, the search checks whether matching points in adjacent rows are connected, for example, determining... Matching point pairs of row pixels Whether the matched point pairs are connected can be determined by conditions including a distance change of less than 0.5m and a direction change of less than 20°. For the index of the matching point pair and , Indicates the first Line matching point pairs, This is the furthest matching pair, which in this instance is the pair in row 20. If the pair is not connected, return to search for other matching pairs; otherwise, connect them. Adding line matching point pairs to In the road boundary sequence where the row matching point pairs are located. Traverse all matching point pairs. If the current row's matching point pairs do not have connected matching point pairs, remove the matching point pair; and save the obtained connected matching point pairs to form the road boundary sequence. The connected matching point pairs obtained from the search form the initial road boundary sequence.

[0044] S6: Filter the obtained initial road boundary sequence, removing those that do not meet the requirements, to obtain the filtered road boundary sequence. The filtering criteria may include judging highly repeated sequences within the same side of the road boundary sequence, and judging slope changes at different points within highly repeated sequences. This step may further include the following steps.

[0045] S6.1: From the initial road boundary sequence, select two candidate road boundary sequences for the same side (left or right) and compare them. If the number of keypoint coordinates with significant differences at the same row index position in the two candidate road boundary sequences (i.e., pairs of keypoints with different coordinates at the same row index position) is less than a preset threshold, then these two sequences are determined to be highly repetitive sequences. Candidate road boundary sequences that do not meet the above conditions can be removed. Optionally, the coordinates of keypoints at the same row index position in the two candidate road boundary sequences can be compared. Keypoint pairs with different coordinates are considered different points. If the number of different points in the two candidate road boundary sequences is less than a preset threshold, then these two sequences are determined to be highly repetitive sequences.

[0046] S6.2: For two highly repetitive road boundary sequences, calculate their local slope changes at all "distinct points". Specifically, for each distinct point (index 1), calculate the slope changes at all "distinct points". ), calculate its relationship with the previous keypoint in its respective sequence ( ) and the next key point ( The slope angle of the line connecting the two sequences is used to obtain the slope change at that point (i.e., the absolute value of the difference between the two slope angles). The slope changes of the two sequences at corresponding different points are compared, and the sequence with the larger slope change is removed. Furthermore, if there is only one difference between two highly repetitive sequences, the sequence with the larger slope change at that point is directly removed; if there are multiple differences between the two highly repetitive sequences and the slope changes of both sequences do not exceed a set slope threshold, the sequence with the larger average slope change is removed after comprehensive comparison.

[0047] Furthermore, the local slope variation at different points can be calculated as follows: calculate the tilt angle between each different point and the two key points above and below the same road boundary sequence. and ,in, Indicates the first The first of the road boundary sequence The key point and the first The tilt angle between the key points Indicates the first The first of the road boundary sequence The key point and the first The tilt angle between the key points.

[0048] To illustrate, for example, in two highly repetitive road boundary sequences, road boundary sequence 1 and road boundary sequence 2, there is only one difference, which is key point 3. However, the slope of road boundary sequence 2 changes more at key point 3. Therefore, road boundary sequence 2 is removed.

[0049] The road boundary sequences in the initial road boundary sequence are filtered in the manner described above, and the filtered road boundary sequence is finally obtained.

[0050] S7: The filtered road boundary sequence is reprojected back into the original image obtained in S1 (i.e., the multi-view mining scene image). For the key points of each road boundary sequence, the least squares method is used to fit the road boundary to obtain the fitted road boundary. The obtained fitted road boundary can be provided to the path planning module for path planning in autonomous driving.

[0051] Furthermore, camera calibration parameters can be used during the reprojection process. For example, the Zhang Zhengyou calibration method can be used to obtain the camera intrinsic parameters and distortion coefficients.

[0052] Furthermore, the fitting process for the key points of each road boundary sequence obtained after screening can be carried out using the weighted least squares method, and the objective function can be established as follows: ,in This is the index of key points in the filtered road boundary sequence. This represents the total number of key points in the filtered road boundary sequence. The weights are... The confidence level is set based on the key points. Optionally, a third-order polynomial can be used for curve fitting. An L2 regularization term can also be added during the fitting process.

[0053] Furthermore, taking third-order polynomial fitting as an example, the fitting function is established as follows: Then the objective function is established as follows:

[0054] in, This represents the total number of key points in a filtered road boundary sequence. This refers to the index of key points in a filtered road boundary sequence. For the first The coordinates of the key points; Weights set based on the confidence level of key points; This is the regularization coefficient, which can be set to 0.01; denoted as the coefficients of the polynomial to be determined.

[0055] The above method can be used to fit all the road boundary sequences obtained after filtering, and finally obtain the fitted road boundaries.

[0056] In summary, the online mapping and boundary line fitting method for open-pit mines based on sparse perception provided by the above implementation method can fit road boundary lines in complex open-pit mine scenarios.

[0057] According to another embodiment of the present invention, an online mapping and boundary line fitting system for open-pit mines based on sparse sensing is provided, which performs the steps of the method described in the above embodiment, including: an imaging device for acquiring an original image; an image encoder module for preprocessing the original image to extract multi-scale feature maps; a sparse sensing module for converting image features into sparse instance representations, including dynamic instances and static instances; a local vector map construction module for constructing a local vector map based on dynamic instances and static instances, and dynamically updating it; a road boundary sequence generation module for performing key point extraction, left and right boundary matching, breadth-first search, and road boundary sequence filtering based on the local vector map to obtain a filtered road boundary sequence; and a road boundary fitting module for fitting the filtered road boundary sequence to obtain and output the fitted road boundary.

[0058] Furthermore, the online mapping and boundary line fitting system for open-pit mines based on sparse perception provided in the above embodiments can be used in the autonomous driving system of mining vehicles, and can cooperate with on-board and / or remote upstream and downstream modules to provide vehicle path planning schemes, autonomous driving schemes, etc.

[0059] According to another embodiment of the present invention, an online mapping and boundary line fitting method for open-pit mines based on sparse sensing is provided, comprising the following steps.

[0060] Step 1: Input multi-view mining area scene images and extract multi-scale feature maps through an image encoder.

[0061] The second step involves using a sparse perception module to convert image features into sparse instance representations, including dynamic and static instances. Dynamic instances are used to identify moving obstacles (such as mining vehicles) to eliminate their interference with boundary line fitting; static instances directly represent map elements such as road boundaries, presented as polylines.

[0062] Step 3: Construct a local vector map based on static instances, and achieve dynamic updates of the map through time-series fusion to ensure the real-time nature and stability of boundary line information.

[0063] Step 4: Extract key points of road boundaries in the bird's-eye view space, and match the left and right boundaries based on distance and slope thresholds.

[0064] Step 5: Perform a breadth-first search on the matching point pairs to obtain the initial road boundary sequence.

[0065] Step 6: Filter the initial road boundary sequence and remove sequences with abnormal slope changes.

[0066] Step 7: Fit the filtered road boundary sequence to obtain the final road boundary line and distinguish between this road and adjacent roads.

[0067] According to the above embodiments of the present invention, an online mapping and road boundary line fitting method based on sparse perception is provided, which has the advantages of high accuracy, high efficiency and strong adaptability, and is suitable for autonomous driving systems in complex environments of open-pit mines.

[0068] All of the above-mentioned optional technical solutions can be combined in any way to form optional embodiments of the present invention, and will not be described in detail here.

[0069] In the description of this application, it should be understood that if terms such as "center", "longitudinal", "lateral", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", "axial", "radial", "circumferential" appear, these terms indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application.

[0070] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order and method of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0071] It should be understood that the foregoing only illustrates some embodiments, and changes, modifications, additions, and / or variations can be made without departing from the scope and spirit of the disclosed embodiments. These embodiments are illustrative and not restrictive. Furthermore, the described embodiments relate to those currently considered most practical and preferred, and should be understood as not being limited to the disclosed embodiments, but rather intended to cover different modifications and equivalent arrangements included within the spirit and scope of those embodiments. Moreover, the various embodiments described above can be used in conjunction with other embodiments; for example, an aspect of one embodiment can be combined with an aspect of another embodiment to achieve yet another embodiment. Additionally, individual features or components of any given component can constitute another embodiment.

[0072] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention, and they should all be covered within the scope of the claims and specification of the present invention.

Claims

1. A method for online mapping and boundary line fitting of open-pit mines based on sparse sensing, characterized in that, include: S1: Acquire multi-view mining area scene images as raw images, and extract multi-scale feature maps through an image encoder; S2: Establish a sparse perception module to uniformly decode multi-scale feature maps, and decode dynamic and static instances in 3D space from multi-scale feature maps through a deformable attention mechanism; S3: Construct a local vector map in the bird's-eye view coordinate system based on static and dynamic instances, and realize the dynamic update of the local vector map through temporal fusion; S4: Extract key points of road boundaries from the local vector map, and perform left and right boundary matching on the key points according to preset distance thresholds and slope thresholds to obtain matching point pairs that meet the matching conditions; S5: Perform a breadth-first search on the matching point pairs to obtain the initial road boundary sequence; S6: Filter the obtained initial road boundary sequence, remove road boundary sequences that do not meet the filtering conditions, and obtain the filtered road boundary sequence; S7: Reproject the filtered road boundary sequence back into the original image. Fit the key points in each road boundary sequence using the least squares method to obtain the fitted road boundary, and output it for subsequent path planning.

2. The method for online mapping and boundary line fitting of open-pit mines based on sparse sensing according to claim 1, characterized in that, In S1: Multiple industrial cameras were used to collect images of the mining area. The industrial cameras were deployed at the front, right front, left front, right rear, left rear and front rear of the vehicle for multi-view consistency verification.

3. The method for online mapping and boundary line fitting of open-pit mines based on sparse sensing according to claim 1, characterized in that, In S2: Dynamic instances are the state information of dynamic targets, including the 3D coordinates, motion attributes, target category and related confidence scores of the dynamic targets, where dynamic targets include mining vehicles and equipment; Static instances consist of polylines composed of multiple control points, used to represent map elements including road boundaries and parking areas, serving as the basis for boundary line fitting. This includes the 3D coordinates of all control points as the 3D coordinates of the static instance.

4. The method for online mapping and boundary line fitting of open-pit mines based on sparse sensing according to claim 3, characterized in that, In S3: Projecting the 3D coordinates of static instances onto the bird's-eye view coordinate plane from above creates a representation of road boundaries in the bird's-eye view space. Mark the area containing the 3D coordinates of dynamic targets in dynamic instances as a temporary obstacle zone, and filter out map elements located within this temporary obstacle zone from static instances.

5. The online mapping and boundary line fitting method for open-pit mines based on sparse sensing according to claim 4, characterized in that, Dynamic updates of local vector maps in S3 include: Set the length of the sequential memory queue to the number of frames; In each frame of the temporal memory queue, the detection confidence of the map element detected in real time is judged. When the confidence is greater than a set threshold, the map element is added to the current local vector map. The system performs position smoothing calculations on map elements that persist in the current local vector map, updates the positions of map elements in the local vector map, and achieves dynamic updates of the local vector map.

6. The method for online mapping and boundary line fitting of open-pit mines based on sparse sensing according to claim 5, characterized in that, In S3: Set the confidence threshold to 0.7 and perform position smoothing calculations on persistent map elements: in, This indicates the current position of a map element in the temporal memory queue. This indicates the location of the newly detected map element in the current frame. This indicates the new location of the map element on the map after the update.

7. The online mapping and boundary line fitting method for open-pit mines based on sparse sensing according to claim 1, characterized in that, S4 include: In the bird's-eye view of the local vector map, extract a row of road boundary key points at set intervals along the direction of the vehicle's movement. Each row of road boundary key points includes a pair of key points on the left and right sides of the road. Perform left and right boundary matching on each row of key points, and retain the key points of each row that meet the matching conditions as matching point pairs; The matching conditions include distance threshold and slope threshold matching. The distance threshold matching condition is as follows: in, This is the lower limit for the width of roads in the mining area. This is the upper limit for the width of roads in the mining area. This represents the distance between key points in a row to be matched. The average direction angle of the row containing the key points to be matched; The slope threshold matching condition is: in, The preset slope tolerance threshold, This represents the slope of the local polyline segment containing the left boundary keypoint in the row of keypoints to be matched. This represents the slope of the local polyline segment containing the right boundary keypoint in the row of keypoints to be matched.

8. The online mapping and boundary line fitting method for open-pit mines based on sparse sensing according to claim 1, characterized in that, S5 include: Start searching from the row of matching points furthest from the vehicle and search towards the vehicle, setting the maximum number of rows for the search depth; Starting from the matching point pair in the farthest row, determine whether the matching point pairs in adjacent rows are connected. The connection conditions include the distance change being less than a set distance threshold and the direction change being less than a set angle threshold. Traverse all matching point pairs, save the matching point pairs that are determined to be connected, and form the initial road boundary sequence.

9. The online mapping and boundary line fitting method for open-pit mines based on sparse sensing according to claim 1, characterized in that, S6 include: S6.1: From the initial road boundary sequence, select two candidate road boundary sequences on the same side for comparison. Compare the coordinates of key points at the same row index position in the two candidate road boundary sequences. Key point pairs with different coordinates are considered different points. If the number of different points in the two candidate road boundary sequences is less than a preset threshold, then these two candidate road boundary sequences are determined to be highly repetitive sequences. S6.2: For two road boundary sequences with high repetition, calculate the local slope change at corresponding different points, compare and remove the road boundary sequence with the larger slope change, and keep the road boundary sequence with the smaller slope change. The above operation is performed on all road boundary sequences in the initial road boundary sequence to obtain the filtered road boundary sequence.

Images