Shovel point generation method and device, electronic equipment and medium

Abstract

The application provides a shovel point generation method and device, electronic equipment and medium. The generation method is based on target sensing information of a target site where a target stockpile is located to determine an identification result of the target site. The identification result of the target site includes an edge line of the target stockpile. Based on the edge line of the target stockpile, a plurality of candidate shovel points on the edge line are determined. Based on the identification result of the target site, each candidate shovel point is evaluated from a plurality of evaluation dimensions to determine a fusion evaluation result of each candidate shovel point. Based on the fusion evaluation result of each candidate shovel point, a target shovel point meeting a preset shovel requirement is screened from the plurality of candidate shovel points, so that a more reliable and accurate shovel point is generated, and the safety and efficiency of the process to the shovel point and the shovel process are ensured.

Description

Technical Field

[0001] This application relates to the field of unmanned loaders, and more specifically, to a method, apparatus, electronic device, and medium for generating a shovel point. Background Technology

[0002] When an unmanned loader is operating, a starting point and direction for shoveling at the edge of a material pile need to be input to ensure that there is material to shovel safely and reliably. Generating a reliable shoveling point provides a target point for the pre-shoveling planning stage and a basis for the direction of movement during shoveling; therefore, the generation and selection of the shoveling point is crucial for the unmanned loader operation. However, current solutions for shoveling perception have poor universality and safety, resulting in unreliable shoveling points. Summary of the Invention

[0003] In view of this, the purpose of this application is to provide a method, apparatus, electronic device and medium for generating a material shovel point, which generates a more reliable and accurate material shovel point, and ensures the safety and high efficiency of the process of going to the material shovel point and the material shovel process.

[0004] In a first aspect, embodiments of this application provide a method for generating a material shovel point, the method comprising:

[0005] Based on the target perception information of the target site where the target material pile is located, the identification result of the target site is determined; the identification result of the target site includes the edge line of the target material pile;

[0006] Based on the edge line of the target material pile, determine multiple candidate shovel points on the edge line;

[0007] Based on the identification results of the target site, each candidate material shovel point is evaluated from multiple evaluation dimensions to determine the fusion evaluation result of each candidate material shovel point; the fusion evaluation result represents the result of combining multiple single-dimensional evaluation results obtained from evaluating the candidate material shovel point from multiple evaluation dimensions respectively;

[0008] Based on the fusion evaluation results of each candidate material shovel point, target material shovel points that meet the preset material shovel requirements are selected from the multiple candidate material shovel points.

[0009] Secondly, embodiments of this application provide a material scraping point generation device, the generation device comprising:

[0010] The first determining module is used to determine the identification result of the target site based on the target perception information of the target site where the target material pile is located; the identification result of the target site includes the edge line of the target material pile.

[0011] The second determining module is used to determine multiple candidate shovel points on the edge line of the target material pile;

[0012] The third determination module is used to evaluate each candidate material shovel point from multiple evaluation dimensions based on the identification results of the target site, and determine the fusion evaluation result of each candidate material shovel point; the fusion evaluation result represents the result of combining multiple single-dimensional evaluation results obtained from evaluating the candidate material shovel point from multiple evaluation dimensions respectively;

[0013] The filtering module is used to filter out target shovel points that meet the preset shovel requirements from the multiple candidate shovel points based on the fusion evaluation results of each candidate shovel point.

[0014] Thirdly, embodiments of this application provide an electronic device, including: a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, the steps of the material scraping point generation method are performed.

[0015] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the material scraping point generation method.

[0016] This application provides a method, apparatus, electronic device, and medium for generating material shovel points. The generation method determines the identification result of the target site based on the target perception information of the target site where the target material pile is located. The identification result of the target site includes the edge line of the target material pile. Based on the edge line of the target material pile, multiple candidate material shovel points are determined on the edge line. Based on the identification result of the target site, each candidate material shovel point is evaluated from multiple evaluation dimensions to determine a fusion evaluation result of each candidate material shovel point that integrates the evaluation results of multiple evaluation dimensions. Based on the fusion evaluation result of each candidate material shovel point, a target material shovel point that meets the preset material shovel requirements is selected from the multiple candidate material shovel points. The method, while providing selectable candidate material shovel points, comprehensively considers the environmental factors of the target site and the safety factors of multiple evaluation dimensions, thus fully ensuring the reliability, safety, and material shovel efficiency of the generated material shovel points.

[0017] Furthermore, based on semantic elevation maps, the material surface is extracted and segmented, resulting in more accurate material pile edges and more precise selection of shovel points. On this basis, by comprehensively considering bucket torque balance, material pile concavity and convexity, distance from obstacles, and entry curvature, the safety of the shovel process is greatly ensured, avoiding many potential risks in the provision of shovel points. It also cleverly combines downstream planning distance indicators to ensure that the shovel points are reached with the highest efficiency during the journey. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 A flowchart of the material scraping point generation method described in an embodiment of this application is shown;

[0020] Figure 2 A flowchart of the method for determining the identification result of the target site according to an embodiment of this application is shown;

[0021] Figure 3 This illustration shows the effect of the semantic elevation map described in the embodiment of this application;

[0022] Figure 4 A schematic diagram of the loader path planning described in an embodiment of this application is shown;

[0023] Figure 5 A flowchart illustrating the method for determining the fusion evaluation result of each candidate material shovel point according to an embodiment of this application is shown.

[0024] Figure 6 This paper illustrates the relationship between the bucket torque balance index and the target torque difference index described in an embodiment of this application.

[0025] Figure 7 This illustration shows the relationship between the safety distance index and the closest target distance described in an embodiment of this application;

[0026] Figure 8 This diagram illustrates the relationship between the work efficiency index described in this application and the lengths of the longest and shortest candidate trajectories.

[0027] Figure 9 This illustration shows the relationship between the entry curvature index and the target entry curvature as described in an embodiment of this application.

[0028] Figure 10 This illustrates the effect of generating material scraping points on the entity during the test described in the embodiments of this application;

[0029] Figure 11 A schematic diagram of the material shovel point generation device according to an embodiment of this application is shown;

[0030] Figure 12 A schematic diagram of the structure of the electronic device described in an embodiment of this application is shown. Detailed Implementation

[0031] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. It should be understood that the accompanying drawings in this application are for illustrative and descriptive purposes only and are not intended to limit the scope of protection of this application. Furthermore, it should be understood that the schematic drawings are not drawn to scale. The flowcharts used in this application illustrate operations implemented according to some embodiments of this application. It should be understood that the operations in the flowcharts may not be implemented in sequence, and steps without logical contextual relationships may be reversed or implemented simultaneously. In addition, those skilled in the art, guided by the content of this application, may add one or more other operations to the flowcharts, or remove one or more operations from the flowcharts.

[0032] Furthermore, the described embodiments are merely some, not all, of the embodiments of this application. The components of the embodiments of this application described and illustrated herein can typically be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0033] It should be noted that the term "comprising" will be used in the embodiments of this application to indicate the presence of the features declared thereafter, but does not exclude the addition of other features.

[0034] When an unmanned loader is operating, a starting point and direction for shoveling at the edge of a material pile need to be input to ensure that there is material to shovel safely and reliably. Generating a reliable shoveling point provides a target point for the pre-shoveling planning stage and a basis for the direction of movement during shoveling; therefore, the generation and selection of the shoveling point is crucial for the unmanned loader operation. However, current solutions for shoveling perception have poor universality and safety, resulting in unreliable shoveling points.

[0035] Based on this, this application provides a method, apparatus, electronic device, and medium for generating material shovel points. The generation method determines the identification result of the target site based on the target perception information of the target site where the target material pile is located. The identification result of the target site includes the edge line of the target material pile. Based on the edge line of the target material pile, multiple candidate material shovel points are determined on the edge line. Based on the identification result of the target site, each candidate material shovel point is evaluated from multiple evaluation dimensions to determine the fusion evaluation result of each candidate material shovel point. The fusion evaluation result represents the result of combining multiple single-dimensional evaluation results obtained from evaluating the candidate material shovel point from multiple evaluation dimensions, which more comprehensively represents the performance of each candidate material shovel point in multiple dimensions. Based on the fusion evaluation result of each candidate material shovel point, target material shovel points that meet the preset material shovel requirements are selected from the multiple candidate material shovel points. The method, while providing selectable candidate material shovel points, comprehensively considers multiple dimensions of safety factors based on the environmental factors of the target site, thus fully ensuring the reliability, safety, and material shovel efficiency of the generated material shovel points.

[0036] Please refer to Figure 1 , Figure 1 A flowchart of the material scraping point generation method described in an embodiment of this application is shown; as follows: Figure 1 As shown, the generation method includes the following steps S101-S104:

[0037] S101. Based on the target perception information of the target site where the target material pile is located, determine the identification result of the target site; the identification result of the target site includes the edge line of the target material pile;

[0038] S102. Based on the edge line of the target material pile, determine multiple candidate shovel points on the edge line;

[0039] S103. Based on the identification results of the target site, each candidate material shovel point is evaluated from multiple evaluation dimensions to determine the fusion evaluation result of each candidate material shovel point; the fusion evaluation result represents the result of combining multiple single-dimensional evaluation results obtained from evaluating the candidate material shovel point from multiple evaluation dimensions respectively;

[0040] S104. Based on the fusion evaluation results of each candidate material shovel point, select the target material shovel point that meets the preset material shovel requirements from the multiple candidate material shovel points.

[0041] In step S101, the identification result of the target site is determined based on the target perception information of the target site where the target material pile is located; the identification result of the target site includes the edge line of the target material pile.

[0042] The target perception information of the target site where the target material pile is located is the target perception information of the target site where the target material pile is located collected by the sensing device.

[0043] The sensing devices include laser sensors, radar, cameras, etc.

[0044] The sensing device can be of various types, thereby acquiring various target sensing information.

[0045] The identification results of the target site also include the three-dimensional location information of the target material pile, obstacles, and ground.

[0046] In other words, in the material shovel point generation method described in this application, the outline of the target material pile can be clearly depicted by edge line recognition, providing high-precision boundary information, thereby improving the generation accuracy of candidate material shovel points; the three-dimensional position information of the target material pile, obstacles, and ground provides all-round spatial information of the site, which is used to realize the subsequent multi-dimensional and all-round evaluation and screening of candidate material shovel points.

[0047] The three-dimensional position information of the target material pile, obstacles, and ground is specifically a set of three-dimensional position information points, where each point in the set represents a sub-region of the site.

[0048] In some embodiments, the sub-region is determined based on the accuracy requirements generated by the target material shovel point.

[0049] In some embodiments, please refer to Figure 2 Based on the target perception information of the target material pile located at the target site collected by the sensing device, the identification result of the target site is determined, including the following steps S201-S203:

[0050] S201. Obtain target perception information of the target material pile located at the target site, collected by the sensing device installed at the target site; the target perception information includes three-dimensional position information and image information;

[0051] S202. Based on the image information and three-dimensional location information of the target site, identify the target material pile, obstacles, and ground in the target site, and determine the three-dimensional location information of the target material pile, obstacles, and ground;

[0052] S203. Based on the three-dimensional position information of the target material pile, obstacles, and ground, determine the edge line of the contact between the target material pile and the ground and the three-dimensional position information of the edge line.

[0053] The image information refers to image data of the target site collected by visual perception devices such as cameras, which contains rich visual features such as color, texture, and shape, and can be used to identify target material piles, obstacles, ground and other items; the other items are items other than target material piles, obstacles and ground.

[0054] The edge line refers to the boundary line formed by the part of the target material pile in contact with the ground.

[0055] For example, the three-dimensional location information of the target site where the target material pile is located, collected by the sensing device, can be point cloud data.

[0056] By fusing image information and 3D location information, more accurate identification of target material piles, obstacles, ground, and material pile edges can be achieved. Image information provides rich visual features, while 3D location information provides precise spatial positioning. The combination of the two can significantly improve the accuracy and reliability of identification.

[0057] In some embodiments, based on image information and three-dimensional location information of the target site, target material piles, obstacles, and ground in the target site are identified, and the three-dimensional location information of the target material piles, obstacles, and ground is determined; including:

[0058] By fusing the image information and three-dimensional location information of the target site, a semantic elevation map is obtained;

[0059] The semantic elevation map is semantically segmented to identify the target material pile, obstacles, and ground.

[0060] Based on the semantic elevation map, the three-dimensional location information of the target material pile, obstacles, and ground is determined.

[0061] The semantic elevation map displays the three-dimensional location information and semantic labels for each sub-region of the target site, and the semantic labels are color labels.

[0062] The semantic elevation map contains both height and semantic information of the target site and is updated synchronously during loader operation.

[0063] The semantic elevation map is a 2.5D grid map containing semantic information. It can be understood that each cell (i.e., sub-region) stores the height of the cell and the semantic label it represents.

[0064] In this embodiment of the application, the semantic tags include target material piles, obstacles, ground, and others.

[0065] In other words, the identification result of the target site includes a semantic segmentation result map of the semantic elevation map; the semantic elevation map is obtained by fusing the image information and the three-dimensional location information.

[0066] Specifically, the step of fusing the image information and three-dimensional location information of the target site to obtain a semantic elevation map includes: partitioning the image information of the target site to obtain an image comprising multiple sub-regions;

[0067] The semantic label of each sub-region in the image information of the target site is determined, and each sub-region is mapped to the target site based on the three-dimensional location information of the target site to obtain a semantic elevation map.

[0068] The semantic label of each sub-region in the image information of the target site is determined. Specifically, the semantic label of each sub-region can be determined based on a classification model. However, the accuracy of the semantic label is relatively low as it is determined only based on the image information. Therefore, after obtaining the semantic elevation map, the semantic elevation map is semantically segmented in combination with the three-dimensional position information in the semantic elevation map to identify the target material pile, obstacles and ground. The identification result is more accurate.

[0069] The semantic elevation map is semantically segmented by combining the three-dimensional position information in the semantic elevation map. Specifically, the semantic elevation map is semantically segmented by combining the height information in the semantic elevation map.

[0070] For example, please refer to Figure 3 The generated semantic elevation map of the material pile and its environment is as follows: Figure 3 As shown, blue represents semantic material pile labels, green represents semantic ground labels, red represents unknowns, and obstacles are represented in yellow.

[0071] After identifying the target material pile, obstacles, and ground in the target site based on semantic segmentation, the edge line of the material pile is extracted based on the contact relationship between the target material pile and the ground.

[0072] In step S102, multiple candidate shovel points are determined on the edge line of the target material pile.

[0073] Based on the edge line of the target material pile, multiple candidate shovel points are determined on the edge line, including:

[0074] The edge line of the target material pile is discretized to determine multiple candidate shovel points on the edge line.

[0075] In other words, after extracting the edge line of the material pile, the edge line is discretized to obtain all available material shoveling points.

[0076] In some embodiments, the edge lines are discretized based on the accuracy of the semantic elevation map.

[0077] Specifically, the semantic elevation map includes multiple grids, and the edge lines are discretized based on these grids. For example, one grid is discretized into a candidate material shovel point, 10 grids are discretized into a candidate material shovel point, and so on.

[0078] In step S103, each candidate material shovel point is evaluated from multiple evaluation dimensions based on the identification results of the target site, and the fusion evaluation result of each candidate material shovel point is determined.

[0079] The fusion evaluation result represents the combined result of multiple single-dimensional evaluation results obtained from evaluating the candidate material shovel point from multiple evaluation dimensions.

[0080] The fusion evaluation result refers to the single-dimensional evaluation result based on multiple dimensions (such as performance, or the result generated after integration through specific rules or algorithms).

[0081] In this embodiment of the application, the fusion evaluation result is specifically a unified quantitative result, which is the result calculated and integrated based on single-dimensional evaluation indicators of multiple dimensions.

[0082] To ensure the safety and efficiency of the material shoveling process, the feasibility of each shoveling point is assessed based on the semantic elevation map, and the optimal shoveling point is selected by comprehensively considering the risks of loader rollover, collision or damage, as well as operational efficiency.

[0083] In other words, the process of the loader traveling from its initial position to the material-shoveling point, the operational efficiency during the material-shoveling process at the material-shoveling point, and the risks that may cause the loader to overturn, collide, or be damaged are analyzed to determine the aforementioned multiple evaluation dimensions.

[0084] The multiple evaluation dimensions include: security evaluation dimension and efficiency evaluation dimension;

[0085] The safety assessment dimensions include: bucket torque balance, safe distance between the loader and obstacles, and entry curvature of the bucket.

[0086] The efficiency evaluation dimensions include: material pile unevenness and operating efficiency.

[0087] Please refer to Figure 4 Given the initial position of the loader and the material-shoveling point (x_i, y_i, delta_m), the loader needs to plan a path Gamma_{i,m} between the two points, travel along this path to the material-shoveling point (x_i, y_i), and complete the material-shoveling action at the entry angle delta_m. The material-shoveling point generation method described in this application simulates this process, calculates the scores s_j of each evaluation dimension, and obtains the weighted score s for that point.

[0088] The material shovel point generation method described in this application first extracts and segments the material surface based on a semantic elevation map, resulting in more accurate material pile edges and more precise selection of shovel points. On this basis, it comprehensively considers bucket torque balance, material pile concavity and convexity, distance from obstacles, and entry curvature, greatly ensuring the safety of the material shovel process and avoiding many potential risks in providing shovel points. Finally, it also cleverly combines downstream planning distance indicators to ensure that the shovel points are reached with the highest efficiency during the journey.

[0089] In some embodiments, please refer to Figure 5 The identification results based on the target site evaluate each candidate material shovel point from multiple evaluation dimensions to determine the fusion evaluation result of each candidate material shovel point, including the following steps S501-S502:

[0090] S501. For each candidate material shovel point, based on the three-dimensional position information of the target material pile, obstacles, and ground in the target site identification results, calculate the evaluation index of the candidate material shovel point in the evaluation dimension according to the evaluation rules of each evaluation dimension; the three-dimensional position information of the target material pile, obstacles, and ground is a collection of three-dimensional position information of multiple sub-regions.

[0091] S502. Integrate the evaluation indicators of the multiple evaluation dimensions to determine the integrated evaluation result of the candidate material shovel point.

[0092] In some embodiments, the bucket torque balance index of the candidate shoveling point in the bucket torque balance dimension is calculated based on the following steps:

[0093] Based on the three-dimensional position information of the target material pile, calculate the target torque difference index between the left and right sides when the loader bucket scoops material at the candidate scooping point;

[0094] Based on the comparison between the target torque difference index and the preset torque safety threshold, the bucket torque balance index of the candidate shoveling point in the bucket torque balance dimension is determined.

[0095] The material pile is not uniform in shape, so the two ends of the bucket will be subjected to different torques when shoveling. When the difference in torque between the two sides is too large, the steering structure of the loader may be damaged, and the vehicle may even be at risk of overturning.

[0096] In some embodiments, based on the three-dimensional position information of the target material pile, the target torque difference index between the left and right sides when the loader bucket scoops material at the candidate scooping point is calculated, including:

[0097] Based on the three-dimensional position information of the target material pile, the first area to be excavated by the loader bucket when shoveling material at the candidate shoveling point is determined; the first area to be excavated includes the left side area of ​​the bucket and the right side area of ​​the bucket; the left side area and the right side area of ​​the bucket include multiple sub-areas;

[0098] Obtain the height and lateral distance from the loader hopper of each sub-region;

[0099] Using the lateral distance as a weight for height, the weighted sum of heights for the left and right sides of the hopper is calculated separately;

[0100] The difference between the weighted sum of the heights of the left and right sides of the hopper is used as the target torque difference index.

[0101] In other words, the calculation of the overturning moment on the left and right sides of the hopper is transformed into a weighted sum of the heights of sub-regions in the semantic elevation map.

[0102] Specifically, the target torque difference index T C The simplified calculation formula is as follows:

[0103] T C =∑ (i,j)∈Ω h ij ·w i ; where h ij The height of the sub-region (i, j) represents the height of the grid in the semantic elevation map; w i The lateral distance of the sub-region (i, j) from the loader hopper is represented by the left direction as positive and the right direction as negative.

[0104] Based on this, and based on the comparison results between the target torque difference index and the preset torque safety threshold, the bucket torque balance index of the candidate shoveling point in the bucket torque balance dimension is determined, including:

[0105] When the target torque difference index is greater than or equal to the preset torque safety threshold, the bucket torque balance index is determined to be zero.

[0106] When the target torque difference index is less than the preset torque safety threshold, the bucket torque balance index is determined based on the ratio of the target torque difference index to the preset torque safety threshold.

[0107] Specifically, the bucket torque balance index S tor The calculation formula is as follows:

[0108]

[0109] Among them, T res Characterizes the preset torque safety threshold.

[0110] Please refer to Figure 6 , Figure 6 The bucket torque balance index S is shown. tor ( Figure 6 The score in the target torque difference index and the target torque difference index Figure 6 A schematic diagram showing the relationship between torque and force.

[0111] like Figure 6 As shown, when the target torque difference index (overturning moment) exceeds the preset torque safety threshold, there will be a significant risk of rollover, and this shoveling point is not recommended; within a certain range close to the threshold, the bucket torque balance index S tor The torque will decrease rapidly; when the torques on both sides are relatively balanced, the score is close to 1, indicating that the torque is not a key indicator to consider at this time.

[0112] In some embodiments, the safe distance index of the candidate shovel point in the dimension of safe distance between the loader and the obstacle is calculated based on the following steps:

[0113] Based on the loader's starting position and the candidate material-shoveling point, determine the target path area covered by the loader during its journey to the candidate material-shoveling point;

[0114] Based on the three-dimensional position information of the target path region and obstacles, determine the closest distance between the obstacles and the target path region;

[0115] Based on the comparison between the nearest distance to the target and the preset distance safety threshold, the safety distance index of the candidate shovel point in the dimension of the safety distance between the loader and the obstacle is determined.

[0116] The probability of a loader colliding with an obstacle during the material loading process is generally related to the distance between it and the obstacle. A certain safe distance should be maintained during the operation. Therefore, a safe distance index is calculated based on the closest distance between the obstacle and the target path area and a preset safe distance threshold.

[0117] Based on the comparison between the nearest distance to the target and the preset safe distance threshold, the safe distance index of the candidate shovel point in the dimension of safe distance between the loader and the obstacle is determined, including:

[0118] When the nearest distance to the target is greater than the preset distance safety threshold, the safety distance index of the candidate material shovel point is determined to be equal to 1;

[0119] When the nearest distance to the target is less than or equal to the preset distance safety threshold, the safety distance index is determined based on the ratio of the nearest distance to the preset distance safety threshold.

[0120] Specifically, the safety distance index S saf The calculation formula is as follows:

[0121]

[0122] Where, d minThe closest distance to the target represents the area between obstacles and the target path; d res This represents the preset distance safety threshold.

[0123] In other words, when the closest distance between an obstacle and the target area in the target path exceeds a preset safe distance threshold, the distance of the obstacle will no longer affect the material shoveling process. Therefore, regardless of the distance, the safe distance index S... saf All scores are 1; conversely, the closer the target is, the lower the score, thus assessing the reasonableness of the target material shovel point.

[0124] Please refer to Figure 7 , Figure 7 This diagram illustrates the relationship between the safety distance index and the closest target distance according to an embodiment of this application; where the horizontal axis (distance) represents the closest target distance; and the vertical axis (score) represents the safety distance index S. saf When the nearest distance to the target is less than or equal to the preset distance safety threshold, the material shoveling point is not recommended; when the nearest distance to the target is greater than the preset distance safety threshold, the material shoveling point is recommended.

[0125] In some embodiments, the concavity index of the candidate shovel point in the concavity dimension of the material pile is calculated based on the following steps:

[0126] Based on the three-dimensional position information of the target material pile, the second area to be excavated by the loader bucket when it scoops material at the candidate scooping point is determined;

[0127] Divide the second area to be excavated into three equal sections along the left and right direction of the hopper, and calculate the material volume of each section.

[0128] Based on the comparison of the material volume in the middle partition and the material volume in the remaining two partitions, the concavity index of the candidate shovel point in the concavity dimension of the material pile is determined.

[0129] In some embodiments, based on the comparison between the material volume of the middle partition and the material volumes of the two side partitions, the concavity index of the candidate shovel point in the concavity dimension of the material pile is determined, including:

[0130] If the material volume in the middle section is greater than or equal to the maximum material volume in the two side sections, then the candidate shovel point is determined to be convex.

[0131] If the material volume in the middle section is less than the maximum material volume in the two side sections, then the candidate shovel point is determined to be concave.

[0132] The unevenness of the material surface has a significant impact on the force exerted on the loader. To improve production efficiency, the loader should be inserted vertically into the material pile and shovel at the raised portion, as a concave material pile introduces greater bucket resistance and negatively affects the bucket fill rate. To estimate the unevenness of the shoveling point, the method described in this application uses a three-part bucket model. The second shoveling area of ​​the loader bucket at the candidate shoveling point is evenly divided into three parts according to the bucket width, resulting in a left section, a middle section, and a right section; the material volume of the left section is V. l The volume of material in the middle section is V. m The volume of material in the right partition is V. r If the volume of material in the middle partition is greater than the maximum volume on both sides, then the point is considered convex; otherwise, the point is considered concave.

[0133] Specifically, the concavity / convexity index S con The calculation formula is as follows:

[0134] Concavity index S con If it is 1, it is a bulge; if it is 0, it is a depression.

[0135] In some embodiments, the operational efficiency index of the candidate material shoveling point in the operational efficiency dimension is calculated based on the following steps:

[0136] Each candidate trajectory from the starting position of the loader to each candidate shovel point of the target stockpile is determined;

[0137] Based on the lengths of the longest and shortest candidate trajectories, and the length of the candidate trajectory corresponding to the candidate shovel point, the operational efficiency index of the candidate shovel point in the operational efficiency dimension is calculated.

[0138] The operational efficiency dimension characterizes the efficiency from the loader's starting position to each candidate shoveling point of the target stockpile.

[0139] Under the premise of ensuring safety, loaders generally tend to choose closer material-collecting points to improve operating efficiency. Based on the lengths of the longest and shortest candidate trajectories, and the length of the candidate trajectory corresponding to the candidate material-collecting point, the operating efficiency index of the candidate material-collecting point is calculated. This operating efficiency index is then normalized for subsequent comprehensive evaluation.

[0140] Each candidate shovel point from the starting position of the loader to the target material pile is determined. Specifically, for each candidate shovel point, multiple directions are evenly divided according to different angles; the candidate shovel point is determined in multiple directions.

[0141] Specifically, the work efficiency index S effCalculated using the following formula:

[0142] Among them, l i,m The length of the candidate trajectory of the i-th shovel point in direction m; l max l is the length of the longest candidate trajectory min is the length of the shortest candidate trajectory.

[0143] Please refer to Figure 8 , Figure 8 The work efficiency index 1 described in the embodiments of this application is shown. min ( Figure 8 A schematic diagram showing the relationship between the score in the test and the lengths of the longest and shortest candidate trajectories.

[0144] In some embodiments, the entry curvature index of the candidate shovel point in the entry curvature dimension is calculated based on the following steps:

[0145] Obtain the target entry curvature at the end of the trajectory corresponding to the candidate shovel point;

[0146] Based on the preset curvature threshold and the target entry curvature, the entry curvature index of the candidate shovel point in the entry curvature dimension is determined.

[0147] When a loader is shoveling material, the angle between the front and rear axles should not be too large. Otherwise, when the loader crashes into the material pile, the lateral pressure on the hinges will be greater, increasing the probability of structural damage and even potentially causing a rollover. Furthermore, the angle between the front and rear axles is related to the curvature K at the end of the trajectory. i，m Therefore, during production operations, loaders should minimize the curvature at the end of the track while avoiding exceeding the maximum permissible curvature to ensure the safety of the loader.

[0148] Based on this, the infeed curvature index S cur The calculation is performed using the following formula:

[0149]

[0150] Among them, K max Characterizes the preset curvature threshold, i.e., the maximum allowable curvature; K i，m The target entry curvature at the end of the candidate trajectory of the i-th shovel point in direction m represents the shovel curvature.

[0151] It should be noted that since curvature can be positive or negative (indicating different directions of bending), but here we only care about the magnitude of the bending, so we use the absolute value of curvature.

[0152] Please refer to Figure 9 , Figure 9The diagram illustrates the relationship between the entry curvature index and the target entry curvature as described in the embodiments of this application, wherein the vertical axis fraction represents the entry curvature index and the horizontal axis angle represents the target entry curvature.

[0153] In step S104, based on the fusion evaluation results of each candidate shovel point, target shovel points that meet the preset shovel requirements are selected from the multiple candidate shovel points.

[0154] Based on the fusion evaluation results of each candidate shovel point, target shovel points that meet the preset shovel requirements are selected from the multiple candidate shovel points, including:

[0155] The fusion evaluation results of each candidate material shoveling point are weighted and fused to determine the fusion evaluation index of each candidate material shoveling point;

[0156] By comparing the fusion evaluation indicators of multiple candidate material shovel points, the candidate material shovel point corresponding to the optimal fusion evaluation indicator is determined as the target material shovel point.

[0157] In some embodiments, the optimal fusion evaluation metric is the largest fusion evaluation metric.

[0158] By comparing the integrated evaluation indicators of multiple candidate material shoveling points, the target material shoveling point corresponding to the highest or best indicator is identified. This target material shoveling point performs well in multiple evaluation indicators, has high operating efficiency, stability and safety, and effectively avoids situations that may damage vehicles and threaten environmental safety, such as excessive overturning moment or collision.

[0159] Specifically, the calculation formula for the fusion evaluation index s is as follows:

[0160] S=∑ j w j ·S j ;

[0161] Among them, w j S represents the weight of the j-th evaluation indicator; j This represents the score of the j-th evaluation indicator; the evaluation indicators include: bucket torque balance index, safety distance index, concavity index, operation efficiency index, and entry curvature index.

[0162] as follows Figure 10 As shown, the actual material shovel point generation effect was tested on a real mixing plant site. The right area is the segmented material surface, the arrows are the top ten recommended material shovel points with the highest comprehensive scores, and the dots are the material shovel points with the highest scores. The output results meet the normal material shovel requirements and can improve efficiency while ensuring safety.

[0163] Based on the same inventive concept, this application also provides a shovel point generating device corresponding to the shovel point generating method. Since the principle of the device in this application is similar to the shovel point generating method described above in this application, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be described again.

[0164] Please refer to Figure 11 , Figure 11 A schematic diagram of the material shovel point generation device described in an embodiment of this application is shown, as follows: Figure 11 As shown, the generating apparatus includes:

[0165] The first determining module 1101 is used to determine the identification result of the target site based on the target perception information of the target site where the target material pile is located; the identification result of the target site includes the edge line of the target material pile.

[0166] The second determining module 1102 is used to determine multiple candidate shovel points on the edge line based on the edge line of the target material pile;

[0167] The third determining module 1103 is used to evaluate each candidate material shovel point from multiple evaluation dimensions based on the identification results of the target site, and determine the fusion evaluation result of each candidate material shovel point; the fusion evaluation result represents the result of combining multiple single-dimensional evaluation results obtained from evaluating the candidate material shovel point from multiple evaluation dimensions respectively;

[0168] The filtering module 1104 is used to filter out target shovel points that meet the preset shovel requirements from the multiple candidate shovel points based on the fusion evaluation results of each candidate shovel point.

[0169] In some embodiments, the identification results of the target site may also include the three-dimensional location information of the target material pile, obstacles, and ground.

[0170] In some embodiments, when the first determining module determines the identification result of the target site based on the target perception information of the target site where the target material pile is located, it is specifically used for:

[0171] Acquire target sensing information about the target material pile located at the target site, collected by sensing devices installed at the target site; the target sensing information includes three-dimensional location information and image information;

[0172] Based on the image information and three-dimensional location information of the target site, the target material pile, obstacles, and ground in the target site are identified, and the three-dimensional location information of the target material pile, obstacles, and ground is determined;

[0173] Based on the three-dimensional position information of the target material pile, obstacles, and ground, the edge line of the contact between the target material pile and the ground and the three-dimensional position information of the edge line are determined.

[0174] In some embodiments, when the first determining module identifies the target material pile, obstacles, and ground in the target site based on the image information and three-dimensional location information of the target site, and determines the three-dimensional location information of the target material pile, obstacles, and ground, it is specifically used for:

[0175] By fusing the image information and three-dimensional location information of the target site, a semantic elevation map is obtained;

[0176] The semantic elevation map is semantically segmented to identify the target material pile, obstacles, and ground.

[0177] Based on the semantic elevation map, the three-dimensional location information of the target material pile, obstacles, and ground is determined.

[0178] In some embodiments, when the second determining module determines multiple candidate shovel points on the edge line based on the edge line of the target material pile, it is specifically used for:

[0179] The edge line of the target material pile is discretized to determine multiple candidate shovel points on the edge line.

[0180] In some embodiments, the plurality of evaluation dimensions include: a security evaluation dimension and an efficiency evaluation dimension;

[0181] The safety assessment dimensions include: bucket torque balance, safe distance between the loader and obstacles, and entry curvature of the bucket.

[0182] The efficiency evaluation dimensions include: material pile unevenness and operating efficiency.

[0183] In some embodiments, the third determining module, when evaluating each candidate material-shoveling point from multiple evaluation dimensions based on the identification results of the target site and determining the fusion evaluation result of each candidate material-shoveling point, is specifically used for:

[0184] For each candidate material shovel point, based on the three-dimensional position information of the target material pile, obstacles, and ground in the target site identification results, the evaluation index of the candidate material shovel point in that evaluation dimension is calculated according to the evaluation rules of each evaluation dimension; the three-dimensional position information of the target material pile, obstacles, and ground is a collection of three-dimensional position information of multiple sub-regions;

[0185] By integrating the evaluation indicators of the multiple evaluation dimensions, the integrated evaluation result of the candidate material shovel point is determined.

[0186] In some embodiments, the third determining module is specifically used to calculate the bucket torque balance index of the candidate shoveling point in the bucket torque balance dimension based on the following steps:

[0187] Based on the three-dimensional position information of the target material pile, calculate the target torque difference index between the left and right sides when the loader bucket scoops material at the candidate scooping point;

[0188] Based on the comparison between the target torque difference index and the preset torque safety threshold, the bucket torque balance index of the candidate shoveling point in the bucket torque balance dimension is determined.

[0189] In some embodiments, the third determining module, when calculating the target torque difference index between the left and right sides of the loader bucket when shoveling material at the candidate shoveling point based on the three-dimensional position information of the target material pile, is specifically used for:

[0190] Based on the three-dimensional position information of the target material pile, the first area to be excavated by the loader bucket when shoveling material at the candidate shoveling point is determined; the first area to be excavated includes the left side area of ​​the bucket and the right side area of ​​the bucket; the left side area and the right side area of ​​the bucket include multiple sub-areas;

[0191] Obtain the height and lateral distance from the loader hopper of each sub-region;

[0192] Using the lateral distance as a weight for height, the weighted sum of heights for the left and right sides of the hopper is calculated separately;

[0193] The difference between the weighted sum of the heights of the left and right sides of the hopper is used as the target torque difference index.

[0194] In some embodiments, when the third determining module determines the bucket torque balance index of the candidate shoveling point in the bucket torque balance dimension based on the comparison result between the target torque difference index and the preset torque safety threshold, it is specifically used for:

[0195] When the target torque difference index is greater than or equal to the preset torque safety threshold, the bucket torque balance index is determined to be zero.

[0196] When the target torque difference index is less than the preset torque safety threshold, the bucket torque balance index is determined based on the ratio of the target torque difference index to the preset torque safety threshold.

[0197] In some embodiments, the third determining module is specifically used to calculate the safety distance index of the candidate shovel point in the dimension of the safety distance between the loader and the obstacle based on the following steps:

[0198] Based on the loader's starting position and the candidate material-shoveling point, determine the target path area covered by the loader during its journey to the candidate material-shoveling point;

[0199] Based on the three-dimensional position information of the target path region and obstacles, determine the closest distance between the obstacles and the target path region;

[0200] Based on the comparison between the nearest distance to the target and the preset distance safety threshold, the safety distance index of the candidate shovel point in the dimension of the safety distance between the loader and the obstacle is determined.

[0201] In some embodiments, when the third determining module determines the safe distance index of the candidate shovel point in the dimension of safe distance between the loader and the obstacle based on the comparison result of the target nearest distance and the preset distance safety threshold, it is specifically used for:

[0202] When the nearest distance to the target is greater than the preset distance safety threshold, the safety distance index of the candidate material shovel point is determined to be equal to 1;

[0203] When the nearest distance to the target is less than or equal to the preset distance safety threshold, the safety distance index is determined based on the ratio of the nearest distance to the preset distance safety threshold.

[0204] In some embodiments, the third determining module is specifically used to calculate the concavity index of the candidate shovel point in the concavity dimension of the material pile based on the following steps:

[0205] Based on the three-dimensional position information of the target material pile, the second area to be excavated by the loader bucket when it scoops material at the candidate scooping point is determined;

[0206] Divide the second area to be excavated into three equal sections along the left and right direction of the hopper, and calculate the material volume of each section.

[0207] Based on the comparison of the material volume in the middle partition and the material volume in the remaining two partitions, the concavity index of the candidate shovel point in the concavity dimension of the material pile is determined.

[0208] In some embodiments, when the third determining module determines the concavity index of the candidate shovel point in the concavity dimension of the material pile based on the comparison result of the material volume of the middle partition and the material volumes of the two side partitions, it is specifically used for:

[0209] If the material volume in the middle section is greater than or equal to the maximum material volume in the two side sections, then the candidate shovel point is determined to be convex.

[0210] If the material volume in the middle section is less than the maximum material volume in the two side sections, then the candidate shovel point is determined to be concave.

[0211] In some embodiments, the third determining module is specifically used to calculate the operational efficiency index of the candidate material shoveling point in the operational efficiency dimension based on the following steps:

[0212] Each candidate trajectory from the starting position of the loader to each candidate shovel point of the target stockpile is determined;

[0213] Based on the lengths of the longest and shortest candidate trajectories, and the length of the candidate trajectory corresponding to the candidate shovel point, the operational efficiency index of the candidate shovel point in the operational efficiency dimension is calculated.

[0214] In some embodiments, the third determining module is specifically used to calculate the entry curvature index of the candidate shovel point in the entry curvature dimension in the following steps:

[0215] Obtain the target entry curvature at the end of the trajectory corresponding to the candidate shovel point;

[0216] Based on the preset curvature threshold and the target entry curvature, the entry curvature index of the candidate shovel point in the entry curvature dimension is determined.

[0217] Based on the preset curvature threshold and the target entry curvature, the entry curvature index of the candidate shovel point in the entry curvature dimension is determined.

[0218] In some embodiments, when the filtering module filters out target shovel points that meet preset shovel requirements from the plurality of candidate shovel points based on the fusion evaluation results of each candidate shovel point, it is specifically used for:

[0219] The fusion evaluation results of each candidate material shoveling point are weighted and fused to determine the fusion evaluation index of each candidate material shoveling point;

[0220] By comparing the fusion evaluation indicators of multiple candidate material shovel points, the candidate material shovel point corresponding to the optimal fusion evaluation indicator is determined as the target material shovel point.

[0221] Based on the same inventive concept, this application also provides an electronic device corresponding to the material scraping point generation method. Since the principle of solving the problem by the electronic device in this application is similar to the above-mentioned material scraping point generation method in this application, the implementation of the electronic device can refer to the implementation of the method, and the repeated parts will not be described again.

[0222] Please refer to Figure 12 , Figure 12 A schematic diagram of the structure of the electronic device 1200 according to an embodiment of this application is shown. The electronic device 1200 includes a processor 1202, a memory 1201, and a bus. The memory 1201 stores machine-readable instructions executable by the processor 1202. When the electronic device 1200 is running, the processor 1202 communicates with the memory 1201 via the bus. When the machine-readable instructions are executed by the processor 1202, the processor 1202 executes the steps of the material scraping point generation method; specifically as follows:

[0223] Based on the target perception information of the target site where the target material pile is located, the identification result of the target site is determined; the identification result of the target site includes the edge line of the target material pile;

[0224] Based on the edge line of the target material pile, determine multiple candidate shovel points on the edge line;

[0225] Based on the identification results of the target site, each candidate material shovel point is evaluated from multiple evaluation dimensions to determine the fusion evaluation result of each candidate material shovel point; the fusion evaluation result represents the result of combining multiple single-dimensional evaluation results obtained from evaluating the candidate material shovel point from multiple evaluation dimensions respectively;

[0226] Based on the fusion evaluation results of each candidate material shovel point, target material shovel points that meet the preset material shovel requirements are selected from the multiple candidate material shovel points.

[0227] In some embodiments, the identification results of the target site may also include the three-dimensional location information of the target material pile, obstacles, and ground.

[0228] In some embodiments, when the processor performs the step of determining the identification result of the target site based on the collected target perception information of the target site where the target material pile is located, it specifically performs the following steps:

[0229] Acquire target sensing information about the target material pile located at the target site, collected by sensing devices installed at the target site; the target sensing information includes three-dimensional location information and image information;

[0230] Based on the image information and three-dimensional location information of the target site, the target material pile, obstacles, and ground in the target site are identified, and the three-dimensional location information of the target material pile, obstacles, and ground is determined;

[0231] Based on the three-dimensional position information of the target material pile, obstacles, and ground, the edge line of the contact between the target material pile and the ground and the three-dimensional position information of the edge line are determined.

[0232] In some embodiments, when the processor executes the step of determining the three-dimensional position information of the target material pile, obstacles, and ground in the target site based on the image information and three-dimensional position information of the target site, it specifically performs the following steps:

[0233] By fusing the image information and three-dimensional location information of the target site, a semantic elevation map is obtained;

[0234] The semantic elevation map is semantically segmented to identify the target material pile, obstacles, and ground.

[0235] Based on the semantic elevation map, the three-dimensional location information of the target material pile, obstacles, and ground is determined.

[0236] In some embodiments, when the processor performs the step of determining multiple candidate shovel points on the edge line based on the edge line of the target material pile, it specifically performs the following steps:

[0237] The edge line of the target material pile is discretized to determine multiple candidate shovel points on the edge line.

[0238] In some embodiments, the plurality of evaluation dimensions include: a security evaluation dimension and an efficiency evaluation dimension;

[0239] The safety assessment dimensions include: bucket torque balance, safe distance between the loader and obstacles, and entry curvature of the bucket.

[0240] The efficiency evaluation dimensions include: material pile unevenness and operating efficiency.

[0241] In some embodiments, when performing the step of evaluating each candidate material shovel point from multiple evaluation dimensions based on the identification results of the target site and determining the fusion evaluation result of each candidate material shovel point, the processor is specifically configured to:

[0242] For each candidate material shovel point, based on the three-dimensional position information of the target material pile, obstacles, and ground in the target site identification results, the evaluation index of the candidate material shovel point in that evaluation dimension is calculated according to the evaluation rules of each evaluation dimension; the three-dimensional position information of the target material pile, obstacles, and ground is a collection of three-dimensional position information of multiple sub-regions;

[0243] By integrating the evaluation indicators of the multiple evaluation dimensions, the integrated evaluation result of the candidate material shovel point is determined.

[0244] In some embodiments, the processor specifically performs the following steps to calculate the bucket torque balance index of the candidate shoveling point in the bucket torque balance dimension:

[0245] Based on the three-dimensional position information of the target material pile, calculate the target torque difference index between the left and right sides when the loader bucket scoops material at the candidate scooping point;

[0246] Based on the comparison between the target torque difference index and the preset torque safety threshold, the bucket torque balance index of the candidate shoveling point in the bucket torque balance dimension is determined.

[0247] In some embodiments, when the processor performs the step of calculating the target torque difference index between the left and right sides of the loader bucket when shoveling material at the candidate shoveling point based on the three-dimensional position information of the target material pile, it specifically performs the following steps:

[0248] Based on the three-dimensional position information of the target material pile, the first area to be excavated by the loader bucket when shoveling material at the candidate shoveling point is determined; the first area to be excavated includes the left side area of ​​the bucket and the right side area of ​​the bucket; the left side area and the right side area of ​​the bucket include multiple sub-areas;

[0249] Obtain the height and lateral distance from the loader hopper of each sub-region;

[0250] Using the lateral distance as a weight for height, the weighted sum of heights for the left and right sides of the hopper is calculated separately;

[0251] The difference between the weighted sum of the heights of the left and right sides of the hopper is used as the target torque difference index.

[0252] In some embodiments, when the processor executes the step of determining the bucket torque balance index of the candidate shoveling point in the bucket torque balance dimension based on the comparison result of the target torque difference index and the preset torque safety threshold, it specifically performs the following steps:

[0253] When the target torque difference index is greater than or equal to the preset torque safety threshold, the bucket torque balance index is determined to be zero.

[0254] When the target torque difference index is less than the preset torque safety threshold, the bucket torque balance index is determined based on the ratio of the target torque difference index to the preset torque safety threshold.

[0255] In some embodiments, the processor specifically performs the following steps: calculating the safety distance index of the candidate shovel point in the dimension of the safety distance between the loader and the obstacle:

[0256] Based on the loader's starting position and the candidate material-shoveling point, determine the target path area covered by the loader during its journey to the candidate material-shoveling point;

[0257] Based on the three-dimensional position information of the target path region and obstacles, determine the closest distance between the obstacles and the target path region;

[0258] Based on the comparison between the nearest distance to the target and the preset distance safety threshold, the safety distance index of the candidate shovel point in the dimension of the safety distance between the loader and the obstacle is determined.

[0259] In some embodiments, when the processor determines the safe distance index of the candidate shovel point in the dimension of safe distance between the loader and the obstacle based on the comparison result of the nearest distance to the target and a preset safe distance threshold, it specifically performs the following steps:

[0260] When the nearest distance to the target is greater than the preset distance safety threshold, the safety distance index of the candidate material shovel point is determined to be equal to 1;

[0261] When the nearest distance to the target is less than or equal to the preset distance safety threshold, the safety distance index is determined based on the ratio of the nearest distance to the preset distance safety threshold.

[0262] In some embodiments, the processor specifically performs the following steps: calculating the concavity index of the candidate shovel point in the concavity dimension of the material pile:

[0263] Based on the three-dimensional position information of the target material pile, the second area to be excavated by the loader bucket when it scoops material at the candidate scooping point is determined;

[0264] Divide the second area to be excavated into three equal sections along the left and right direction of the hopper, and calculate the material volume of each section.

[0265] Based on the comparison of the material volume in the middle partition and the material volume in the remaining two partitions, the concavity index of the candidate shovel point in the concavity dimension of the material pile is determined.

[0266] In some embodiments, when the processor determines the concavity index of the candidate shovel point in the concavity dimension of the material pile based on the comparison results of the material volume of the middle partition and the material volumes of the two side partitions, it specifically performs the following steps:

[0267] If the material volume in the middle section is greater than or equal to the maximum material volume in the two side sections, then the candidate shovel point is determined to be convex.

[0268] If the material volume in the middle section is less than the maximum material volume in the two side sections, then the candidate shovel point is determined to be concave.

[0269] In some embodiments, the processor specifically performs the following steps to calculate the operational efficiency index of the candidate material shoveling point in the operational efficiency dimension:

[0270] Each candidate trajectory from the starting position of the loader to each candidate shovel point of the target stockpile is determined;

[0271] Based on the lengths of the longest and shortest candidate trajectories, and the length of the candidate trajectory corresponding to the candidate shovel point, the operational efficiency index of the candidate shovel point in the operational efficiency dimension is calculated.

[0272] In some embodiments, the processor specifically performs the following steps: calculating the entry curvature index of the candidate shovel point in the entry curvature dimension:

[0273] Obtain the target entry curvature at the end of the trajectory corresponding to the candidate shovel point;

[0274] Based on the preset curvature threshold and the target entry curvature, the entry curvature index of the candidate shovel point in the entry curvature dimension is determined.

[0275] Based on the preset curvature threshold and the target entry curvature, the entry curvature index of the candidate shovel point in the entry curvature dimension is determined.

[0276] In some embodiments, when the processor performs the step of selecting target shovel points that meet preset shovel requirements from the plurality of candidate shovel points based on the fusion evaluation results of each candidate shovel point, it specifically performs the following steps:

[0277] The fusion evaluation results of each candidate material shoveling point are weighted and fused to determine the fusion evaluation index of each candidate material shoveling point;

[0278] By comparing the fusion evaluation indicators of multiple candidate material shovel points, the candidate material shovel point corresponding to the optimal fusion evaluation indicator is determined as the target material shovel point.

[0279] Based on the same inventive concept, this application also provides a computer-readable storage medium corresponding to the material scraping point generation method. Since the principle of the computer-readable storage medium in this application is similar to the above-mentioned material scraping point generation method in this application, the implementation of the computer-readable storage medium can refer to the implementation of the method, and the repeated parts will not be described again.

[0280] A computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor performs the steps of the material scraping point generation method described above, as follows:

[0281] Based on the target perception information of the target site where the target material pile is located, the identification result of the target site is determined; the identification result of the target site includes the edge line of the target material pile;

[0282] Based on the edge line of the target material pile, determine multiple candidate shovel points on the edge line;

[0283] Based on the identification results of the target site, each candidate material shovel point is evaluated from multiple evaluation dimensions to determine the fusion evaluation result of each candidate material shovel point; the fusion evaluation result represents the result of combining multiple single-dimensional evaluation results obtained from evaluating the candidate material shovel point from multiple evaluation dimensions respectively;

[0284] Based on the fusion evaluation results of each candidate material shovel point, target material shovel points that meet the preset material shovel requirements are selected from the multiple candidate material shovel points.

[0285] In some embodiments, the identification results of the target site may also include the three-dimensional location information of the target material pile, obstacles, and ground.

[0286] In some embodiments, when the processor performs the step of determining the identification result of the target site based on the collected target perception information of the target site where the target material pile is located, it specifically performs the following steps:

[0287] Acquire target sensing information about the target material pile located at the target site, collected by sensing devices installed at the target site; the target sensing information includes three-dimensional location information and image information;

[0288] Based on the image information and three-dimensional location information of the target site, the target material pile, obstacles, and ground in the target site are identified, and the three-dimensional location information of the target material pile, obstacles, and ground is determined;

[0289] Based on the three-dimensional position information of the target material pile, obstacles, and ground, the edge line of the contact between the target material pile and the ground and the three-dimensional position information of the edge line are determined.

[0290] In some embodiments, when the processor performs the step of identifying target material piles, obstacles, and ground in the target site based on image information and three-dimensional location information of the target site, and determining the three-dimensional location information of the target material piles, obstacles, and ground, it specifically performs the following steps:

[0291] By fusing the image information and three-dimensional location information of the target site, a semantic elevation map is obtained;

[0292] The semantic elevation map is semantically segmented to identify the target material pile, obstacles, and ground.

[0293] Based on the semantic elevation map, the three-dimensional location information of the target material pile, obstacles, and ground is determined.

[0294] In some embodiments, when the processor performs the step of determining multiple candidate shovel points on the edge line based on the edge line of the target material pile, it specifically performs the following steps:

[0295] The edge line of the target material pile is discretized to determine multiple candidate shovel points on the edge line.

[0296] In some embodiments, the plurality of evaluation dimensions include: a security evaluation dimension and an efficiency evaluation dimension;

[0297] The safety assessment dimensions include: bucket torque balance, safe distance between the loader and obstacles, and entry curvature of the bucket.

[0298] The efficiency evaluation dimensions include: material pile unevenness and operating efficiency.

[0299] In some embodiments, when performing the step of evaluating each candidate material shovel point from multiple evaluation dimensions based on the identification results of the target site and determining the fusion evaluation result of each candidate material shovel point, the processor is specifically configured to:

[0300] For each candidate material shovel point, based on the three-dimensional position information of the target material pile, obstacles, and ground in the target site identification results, the evaluation index of the candidate material shovel point in that evaluation dimension is calculated according to the evaluation rules of each evaluation dimension; the three-dimensional position information of the target material pile, obstacles, and ground is a collection of three-dimensional position information of multiple sub-regions;

[0301] By integrating the evaluation indicators of the multiple evaluation dimensions, the integrated evaluation result of the candidate material shovel point is determined.

[0302] In some embodiments, the processor specifically performs the following steps to calculate the bucket torque balance index of the candidate shoveling point in the bucket torque balance dimension:

[0303] Based on the three-dimensional position information of the target material pile, calculate the target torque difference index between the left and right sides when the loader bucket scoops material at the candidate scooping point;

[0304] Based on the comparison between the target torque difference index and the preset torque safety threshold, the bucket torque balance index of the candidate shoveling point in the bucket torque balance dimension is determined.

[0305] In some embodiments, when the processor performs the step of calculating the target torque difference index between the left and right sides of the loader bucket when shoveling material at the candidate shoveling point based on the three-dimensional position information of the target material pile, it specifically performs the following steps:

[0306] Based on the three-dimensional position information of the target material pile, the first area to be excavated by the loader bucket when shoveling material at the candidate shoveling point is determined; the first area to be excavated includes the left side area of ​​the bucket and the right side area of ​​the bucket; the left side area and the right side area of ​​the bucket include multiple sub-areas;

[0307] Obtain the height and lateral distance from the loader hopper of each sub-region;

[0308] Using the lateral distance as a weight for height, the weighted sum of heights for the left and right sides of the hopper is calculated separately;

[0309] The difference between the weighted sum of the heights of the left and right sides of the hopper is used as the target torque difference index.

[0310] In some embodiments, when the processor executes the step of determining the bucket torque balance index of the candidate shoveling point in the bucket torque balance dimension based on the comparison result of the target torque difference index and the preset torque safety threshold, it specifically performs the following steps:

[0311] When the target torque difference index is greater than or equal to the preset torque safety threshold, the bucket torque balance index is determined to be zero.

[0312] When the target torque difference index is less than the preset torque safety threshold, the bucket torque balance index is determined based on the ratio of the target torque difference index to the preset torque safety threshold.

[0313] In some embodiments, the processor specifically performs the following steps: calculating the safety distance index of the candidate shovel point in the dimension of the safety distance between the loader and the obstacle:

[0314] Based on the loader's starting position and the candidate material-shoveling point, determine the target path area covered by the loader during its journey to the candidate material-shoveling point;

[0315] Based on the three-dimensional position information of the target path region and obstacles, determine the closest distance between the obstacles and the target path region;

[0316] Based on the comparison between the nearest distance to the target and the preset distance safety threshold, the safety distance index of the candidate shovel point in the dimension of the safety distance between the loader and the obstacle is determined.

[0317] In some embodiments, when the processor determines the safe distance index of the candidate shovel point in the dimension of safe distance between the loader and the obstacle based on the comparison result of the nearest distance to the target and a preset safe distance threshold, it specifically performs the following steps:

[0318] When the nearest distance to the target is greater than the preset distance safety threshold, the safety distance index of the candidate material shovel point is determined to be equal to 1;

[0319] When the nearest distance to the target is less than or equal to the preset distance safety threshold, the safety distance index is determined based on the ratio of the nearest distance to the preset distance safety threshold.

[0320] In some embodiments, the processor specifically performs the following steps: calculating the concavity index of the candidate shovel point in the concavity dimension of the material pile:

[0321] Based on the three-dimensional position information of the target material pile, the second area to be excavated by the loader bucket when it scoops material at the candidate scooping point is determined;

[0322] Divide the second area to be excavated into three equal sections along the left and right direction of the hopper, and calculate the material volume of each section.

[0323] Based on the comparison of the material volume in the middle partition and the material volume in the remaining two partitions, the concavity index of the candidate shovel point in the concavity dimension of the material pile is determined.

[0324] In some embodiments, when the processor determines the concavity index of the candidate shovel point in the concavity dimension of the material pile based on the comparison results of the material volume of the middle partition and the material volumes of the two side partitions, it specifically performs the following steps:

[0325] If the material volume in the middle section is greater than or equal to the maximum material volume in the two side sections, then the candidate shovel point is determined to be convex.

[0326] If the material volume in the middle section is less than the maximum material volume in the two side sections, then the candidate shovel point is determined to be concave.

[0327] In some embodiments, the processor specifically performs the following steps to calculate the operational efficiency index of the candidate material shoveling point in the operational efficiency dimension:

[0328] Each candidate trajectory from the starting position of the loader to each candidate shovel point of the target stockpile is determined;

[0329] Based on the lengths of the longest and shortest candidate trajectories, and the length of the candidate trajectory corresponding to the candidate shovel point, the operational efficiency index of the candidate shovel point in the operational efficiency dimension is calculated.

[0330] In some embodiments, the processor specifically performs the following steps: calculating the entry curvature index of the candidate shovel point in the entry curvature dimension:

[0331] Obtain the target entry curvature at the end of the trajectory corresponding to the candidate shovel point;

[0332] Based on the preset curvature threshold and the target entry curvature, the entry curvature index of the candidate shovel point in the entry curvature dimension is determined.

[0333] Based on the preset curvature threshold and the target entry curvature, the entry curvature index of the candidate shovel point in the entry curvature dimension is determined.

[0334] In some embodiments, when the processor performs the step of selecting target shovel points that meet preset shovel requirements from the plurality of candidate shovel points based on the fusion evaluation results of each candidate shovel point, it specifically performs the following steps:

[0335] The fusion evaluation results of each candidate material shoveling point are weighted and fused to determine the fusion evaluation index of each candidate material shoveling point;

[0336] By comparing the fusion evaluation indicators of multiple candidate material shovel points, the candidate material shovel point corresponding to the optimal fusion evaluation indicator is determined as the target material shovel point.

[0337] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems and devices described above can be referred to the corresponding processes in the method embodiments, and will not be repeated here. In the several embodiments provided in this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed mutual coupling or direct coupling or communication connection can be through some communication interfaces; the indirect coupling or communication connection of devices or modules can be electrical, mechanical, or other forms.

[0338] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0339] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0340] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a platform server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0341] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for generating material shovel points, characterized in that, The generation method includes: Based on the target perception information of the target site where the target material pile is located, the identification result of the target site is determined; the identification result of the target site includes the edge line of the target material pile; Based on the edge line of the target material pile, determine multiple candidate material shoveling points on the edge line; Based on the identification results of the target site, each candidate material shovel point is evaluated from multiple evaluation dimensions to determine the fusion evaluation result of each candidate material shovel point; the fusion evaluation result represents the result of combining multiple single-dimensional evaluation results obtained from evaluating the candidate material shovel point from multiple evaluation dimensions respectively; Based on the fusion evaluation results of each candidate material shovel point, target material shovel points that meet the preset material shovel requirements are selected from the multiple candidate material shovel points.

2. The method for generating material shovel points according to claim 1, characterized in that, The identification results of the target site also include the three-dimensional location information of the target material pile, obstacles, and ground.

3. The method for generating material shovel points according to claim 1 or 2, characterized in that, Based on the target perception information collected about the target site where the target material pile is located, the identification result of the target site is determined, including: Acquire target sensing information about the target material pile located at the target site, collected by sensing devices installed at the target site; the target sensing information includes three-dimensional location information and image information; Based on the image information and three-dimensional location information of the target site, the target material pile, obstacles, and ground in the target site are identified, and the three-dimensional location information of the target material pile, obstacles, and ground is determined; Based on the three-dimensional position information of the target material pile, obstacles, and ground, the edge line of the contact between the target material pile and the ground and the three-dimensional position information of the edge line are determined.

4. The method for generating material scraping points according to claim 3, characterized in that, Based on the image information and three-dimensional location information of the target site, identify the target material pile, obstacles, and ground in the target site, and determine the three-dimensional location information of the target material pile, obstacles, and ground; including: By fusing the image information and three-dimensional location information of the target site, a semantic elevation map is obtained; The semantic elevation map is semantically segmented to identify the target material pile, obstacles, and ground. Based on the semantic elevation map, the three-dimensional location information of the target material pile, obstacles, and ground is determined.

5. The method for generating material shovel points according to claim 1 or 4, characterized in that, Based on the edge line of the target material pile, multiple candidate shovel points are determined on the edge line, including: The edge line of the target material pile is discretized to determine multiple candidate shovel points on the edge line.

6. The method for generating material shovel points according to claim 1 or 2, characterized in that, The multiple evaluation dimensions include: security evaluation dimension and efficiency evaluation dimension; The safety assessment dimensions include: bucket torque balance, safe distance between the loader and obstacles, and entry curvature of the bucket. The efficiency evaluation dimensions include: material pile unevenness and operating efficiency.

7. The method for generating material shovel points according to claim 1, 2, or 6, characterized in that, Based on the target site identification results, each candidate material shovel point is evaluated from multiple evaluation dimensions to determine the fusion evaluation result for each candidate material shovel point, including: For each candidate material shovel point, based on the three-dimensional position information of the target material pile, obstacles, and ground in the target site identification results, the evaluation index of the candidate material shovel point in that evaluation dimension is calculated according to the evaluation rules of each evaluation dimension; the three-dimensional position information of the target material pile, obstacles, and ground is a collection of three-dimensional position information of multiple sub-regions; By integrating the evaluation indicators of the multiple evaluation dimensions, the integrated evaluation result of the candidate material shovel point is determined.

8. The method for generating material scraping points according to claim 7, characterized in that, The following steps are used to calculate the bucket torque balance index of the candidate shoveling point in the bucket torque balance dimension: Based on the three-dimensional position information of the target material pile, calculate the target torque difference index between the left and right sides when the loader bucket scoops material at the candidate scooping point; Based on the comparison between the target torque difference index and the preset torque safety threshold, the bucket torque balance index of the candidate shoveling point in the bucket torque balance dimension is determined.

9. The method for generating material scraping points according to claim 8, characterized in that, Based on the three-dimensional position information of the target material pile, the target torque difference index between the left and right sides when the loader bucket scoops material at the candidate scooping point is calculated, including: Based on the three-dimensional position information of the target material pile, the first area to be excavated by the loader bucket when shoveling material at the candidate shoveling point is determined; the first area to be excavated includes the left side area of ​​the bucket and the right side area of ​​the bucket; the left side area and the right side area of ​​the bucket include multiple sub-areas; Obtain the height and lateral distance from the loader hopper of each sub-region; Using the lateral distance as a weight for height, the weighted sum of heights for the left and right sides of the hopper is calculated separately; The difference between the weighted sum of the heights of the left and right sides of the hopper is used as the target torque difference index.

10. The method for generating material scraping points according to claim 8, characterized in that, Based on the comparison between the target torque difference index and the preset torque safety threshold, the bucket torque balance index of the candidate shoveling point in the bucket torque balance dimension is determined, including: When the target torque difference index is greater than or equal to the preset torque safety threshold, the bucket torque balance index is determined to be zero. When the target torque difference index is less than the preset torque safety threshold, the bucket torque balance index is determined based on the ratio of the target torque difference index to the preset torque safety threshold.

11. The method for generating a material shovel point according to claim 7, characterized in that, The following steps are used to calculate the safety distance index of the candidate shovel point in the dimension of the safety distance between the loader and the obstacle: Based on the loader's starting position and the candidate material-shoveling point, determine the target path area covered by the loader during its journey to the candidate material-shoveling point; Based on the three-dimensional position information of the target path region and obstacles, determine the closest distance between the obstacles and the target path region; Based on the comparison between the nearest distance to the target and the preset distance safety threshold, the safety distance index of the candidate shovel point in the dimension of the safety distance between the loader and the obstacle is determined.

12. The method for generating material shovel points according to claim 11, characterized in that, Based on the comparison between the nearest distance to the target and the preset safe distance threshold, the safe distance index of the candidate shovel point in the dimension of safe distance between the loader and the obstacle is determined, including: When the nearest distance to the target is greater than the preset distance safety threshold, the safety distance index of the candidate material shovel point is determined to be equal to 1; When the nearest distance to the target is less than or equal to the preset distance safety threshold, the safety distance index is determined based on the ratio of the nearest distance to the preset distance safety threshold.

13. The method for generating material scraping points according to claim 7, characterized in that, The following steps are used to calculate the concavity index of the candidate shovel point in the concavity dimension of the stockpile: Based on the three-dimensional position information of the target material pile, the second area to be excavated by the loader bucket when it scoops material at the candidate scooping point is determined; Divide the second area to be excavated into three equal sections along the left and right direction of the hopper, and calculate the material volume of each section. Based on the comparison of the material volume in the middle partition and the material volume in the remaining two partitions, the concavity index of the candidate shovel point in the concavity dimension of the material pile is determined.

14. The method for generating material shovel points according to claim 13, characterized in that, Based on the comparison of material volumes in the middle and side zones, the concavity / convexity index of the candidate shovel point in the concavity / convexity dimension of the material pile is determined, including: If the material volume in the middle section is greater than or equal to the maximum material volume in the two side sections, then the candidate shovel point is determined to be convex. If the material volume in the middle section is less than the maximum material volume in the two side sections, then the candidate shovel point is determined to be concave.

15. The method for generating a material shovel point according to claim 7, characterized in that, The following steps are used to calculate the operational efficiency index of the candidate material shoveling point in terms of operational efficiency: Each candidate trajectory from the starting position of the loader to each candidate shovel point of the target stockpile is determined; Based on the lengths of the longest and shortest candidate trajectories, and the length of the candidate trajectory corresponding to the candidate shovel point, the operational efficiency index of the candidate shovel point in the operational efficiency dimension is calculated.

16. The method for generating material scraping points according to claim 7, characterized in that, The following steps are used to calculate the entry curvature index of the candidate shovel point in the entry curvature dimension: Obtain the target entry curvature at the end of the trajectory corresponding to the candidate shovel point; Based on the preset curvature threshold and the target entry curvature, the entry curvature index of the candidate shovel point in the entry curvature dimension is determined. Based on the preset curvature threshold and the target entry curvature, the entry curvature index of the candidate shovel point in the entry curvature dimension is determined.

17. The method for generating material scraping points according to claim 1, characterized in that, Based on the fusion evaluation results of each candidate shovel point, target shovel points that meet the preset shovel requirements are selected from the multiple candidate shovel points, including: The fusion evaluation results of each candidate material shoveling point are weighted and fused to determine the fusion evaluation index of each candidate material shoveling point; By comparing the fusion evaluation indicators of multiple candidate material shovel points, the candidate material shovel point corresponding to the optimal fusion evaluation indicator is determined as the target material shovel point.

18. A material shovel point generation device, characterized in that, The generating apparatus includes: The first determining module is used to determine the identification result of the target site based on the target perception information of the target site where the target material pile is located; the identification result of the target site includes the edge line of the target material pile. The second determining module is used to determine multiple candidate shovel points on the edge line of the target material pile; The third determination module is used to evaluate each candidate material shovel point from multiple evaluation dimensions based on the identification results of the target site, and determine the fusion evaluation result of each candidate material shovel point; the fusion evaluation result represents the result of combining multiple single-dimensional evaluation results obtained from evaluating the candidate material shovel point from multiple evaluation dimensions respectively; The filtering module is used to filter out target shovel points that meet the preset shovel requirements from the multiple candidate shovel points based on the fusion evaluation results of each candidate shovel point.

19. An electronic device, characterized in that, include: The device includes a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, the steps of the material shovel point generation method as described in any one of claims 1 to 17 are performed.

20. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of the material scraping point generation method as described in any one of claims 1 to 18.

Images