Method, device and equipment for sensing stock volume of stock bin and storage medium

By moving the three-dimensional lidar in the work site and establishing an elevation diagram to calculate the material storage volume, the problems of high installation cost of perceptual equipment and complex coordinate calibration in medium and large-scale work sites are solved, and the accuracy and popularity of material storage volume detection are improved. .

CN118897301APending Publication Date: 2024-11-05NETEASE LINGDONG (HANGZHOU) TECHNOLOGY CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411069895.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-05
Publication Date
2024-11-05

AI Technical Summary

Technical Problem

In medium and large operation sites, the inspection of warehousing volume requires the installation of multiple sets of sensing equipment, which leads to high deployment costs and complex coordinate calibration, which affects the accuracy of detection.

Method used

The movable device drives the three-dimensional lidar to move within the work site, perform laser positioning, establish an elevation diagram, determine the average height and area of ​​the material stacking area, and calculate the material storage volume.

Benefits of technology

It significantly reduces the installation cost of perceived equipment, reduces coordinate calibration errors, improves the accuracy of estimating material storage volumes, and promotes the popularization of medium and large-scale operation locations.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118897301A_ABST
    Figure CN118897301A_ABST
Patent Text Reader

Abstract

The invention provides a stock bin reserve sensing method, device and equipment and a storage medium, and the sensing method drives a three-dimensional laser radar to move in a working site through a movable device, and can carry out the laser positioning of the working site in the moving process of the three-dimensional laser radar. The installation cost of the sensing equipment in a working site is greatly reduced, popularization and promotion of the stock bin reserve sensing method in medium and large working sites are facilitated, and meanwhile calculation deviation generated due to the fact that coordinate calibration needs to be carried out between the sensing equipment and the stock bin position can be reduced. And the accuracy of the pre-estimation result of the stock bin reserves is effectively improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of intelligent detection technology, and more specifically, to a method, device, equipment and storage medium for sensing storage capacity in a silo. Background Art

[0002] At present, when detecting the storage capacity of silos in a work site (such as a mixing plant), it is usually necessary to install sensing devices such as cameras or laser sensors in the work site to sense and calculate the storage capacity of the silos. At least one set of sensing devices needs to be installed for each silo or two adjacent silos. In addition, the sensing devices such as cameras or laser sensors installed in the work site also need to be calibrated with the silo position. Therefore, the more silos there are in the work site, the more sensing devices need to be installed, and the more complicated the calibration process will be, resulting in a high cost for deploying the sensing devices, which is not conducive to popularization in medium and large work sites (generally, medium-sized mixing plants will have about ten silos). Summary of the invention

[0003] In view of this, the present application provides a method, device, equipment and storage medium for sensing the storage capacity of a silo. A movable device is used to drive a three-dimensional laser radar to move within a work site, and laser positioning of the work site can be performed during the movement of the three-dimensional laser radar, thereby greatly reducing the installation cost of sensing equipment within the work site. This is conducive to the popularization and promotion of the silo storage capacity sensing method in medium and large work sites, and can also reduce the calculation deviation caused by the need to perform coordinate calibration between the sensing equipment and the silo position, effectively improving the accuracy of the estimated results of the silo storage capacity.

[0004] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings.

[0005] In a first aspect, an embodiment of the present application provides a method for sensing storage capacity of a silo, the sensing method comprising:

[0006] The three-dimensional laser radar is driven to move in the work site by a movable device, and during the movement, the laser positioning data of each point in the work site is collected according to the three-dimensional laser radar;

[0007] Establishing an elevation map of the work site from a global perspective based on the collected laser positioning data;

[0008] According to the target local elevation map corresponding to the target silo in the elevation map, a material stacking area whose height information is higher than the ground height is determined from the target local elevation map, and the material reserves in the target silo are determined according to the average height and area of ​​the material stacking area in the target local elevation map; wherein the target silo represents a silo located in the work site.

[0009] In a second aspect, an embodiment of the present application provides a device for sensing storage capacity in a silo, the sensing device comprising:

[0010] A collection module, used to drive the three-dimensional laser radar to move in the work site through a movable device, and during the movement, collect laser positioning data of each point in the work site according to the three-dimensional laser radar;

[0011] A mapping module, used to establish an elevation map of the work site from a global perspective based on the collected laser positioning data;

[0012] A perception module is used to determine a material storage area whose height information is higher than the ground height from the target local elevation map according to the target local elevation map corresponding to the target silo in the elevation map, and to determine the material reserve in the target silo according to the average height and area of ​​the material storage area in the target local elevation map; wherein the target silo represents a silo located in the work site.

[0013] In a third aspect, an embodiment of the present application provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned method for sensing storage capacity in a silo when executing the computer program.

[0014] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the above-mentioned method for sensing the storage capacity of a silo are executed.

[0015] The technical solution provided by the embodiments of the present application may have the following beneficial effects:

[0016] The embodiments of the present application provide a method, device, equipment and storage medium for sensing the storage capacity of a silo. A movable device drives a three-dimensional laser radar to move within a work site, and the work site can be laser positioned during the movement of the three-dimensional laser radar, thereby greatly reducing the installation cost of the sensing equipment within the work site. This is conducive to the popularization and promotion of the silo storage capacity sensing method in medium and large-scale work sites, and can also reduce the calculation deviation caused by the need to perform coordinate calibration between the sensing equipment and the silo position, thereby effectively improving the accuracy of the estimated results of the silo storage capacity. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 A schematic diagram of a process of sensing a storage capacity of a silo provided in an embodiment of the present application is shown;

[0019] Figure 2a A schematic diagram of a work site provided in an embodiment of the present application is shown;

[0020] Figure 2b A schematic diagram of an elevation map of a work site from a global perspective provided by an embodiment of the present application is shown;

[0021] Figure 3 A schematic diagram of calculating the material storage amount in a target silo provided in an embodiment of the present application is shown;

[0022] Figure 4 A schematic diagram of a process of a high-level version of a silo storage capacity sensing method provided in an embodiment of the present application is shown;

[0023] Figure 5 A schematic diagram of an elevation map fused with entity distribution recognition results provided by an embodiment of the present application is shown;

[0024] Figure 6 A schematic flow chart of a method for updating a storage estimation result in a silo provided in an embodiment of the present application is shown;

[0025] Figure 7 A schematic diagram of the structure of a device for sensing storage capacity of a silo provided in an embodiment of the present application is shown;

[0026] Figure 8 A schematic diagram of the structure of an electronic device 800 provided in an embodiment of the present application. DETAILED DESCRIPTION

[0027] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the drawings in the present application only serve the purpose of explanation and description and are not used to limit the scope of protection of the present application. In addition, it should be understood that the schematic drawings are not drawn in real proportion. The flowchart used in this application shows the operations implemented according to some embodiments of the present application. It should be understood that the operations of the flowchart can be implemented out of sequence, and the steps without logical context can be reversed in order or implemented simultaneously. In addition, those skilled in the art can add one or more other operations to the flowchart under the guidance of the content of the present application, or remove one or more operations from the flowchart.

[0028] In addition, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0029] It should be noted that the term "comprising" will be used in the embodiments of the present application to indicate the existence of the features declared thereafter, but does not exclude the addition of other features.

[0030] At present, when detecting the storage capacity of silos in a work site, it is usually necessary to install sensing devices such as cameras or laser sensors in the work site to sense and calculate the storage capacity of the silos. At least one set of sensing devices needs to be installed for each silo or two adjacent silos. In addition, the sensing devices such as cameras or laser sensors installed in the work site also need to be calibrated with the position of the silo. Therefore, the more silos there are in the work site, the more sensing devices need to be installed, and the more complicated the calibration process will be, resulting in a high deployment cost of the sensing equipment, which is not conducive to popularization in medium and large work sites.

[0031] Based on this, the embodiments of the present application provide a method, device, equipment and storage medium for sensing the storage capacity of a silo. The three-dimensional laser radar is driven by a movable device to move within the work site, and the work site can be laser positioned during the movement of the three-dimensional laser radar, which greatly reduces the installation cost of the sensing equipment within the work site, and is conducive to the popularization and promotion of the silo storage capacity sensing method in medium and large work sites. At the same time, it can also reduce the calculation deviation caused by the need to calibrate the coordinates between the sensing equipment and the silo position, effectively improving the accuracy of the estimated results of the silo storage capacity.

[0032] In one embodiment of the present application, a method for sensing the storage capacity of a silo can be run on a terminal device or a server. The terminal device can be a local terminal device. When the method for sensing the storage capacity of a silo is run on a server, the method can be implemented and executed based on a cloud interaction system, wherein the cloud interaction system includes a server and a client device (i.e., a terminal device).

[0033] To facilitate understanding of the embodiments of the present application, a method, device, equipment and storage medium for sensing storage capacity in a silo provided in the embodiments of the present application are introduced in detail below.

[0034] Reference Figure 1 As shown, Figure 1 A schematic flow chart of a method for sensing storage capacity of a silo provided in an embodiment of the present application is shown, wherein the sensing method comprises steps S101-S103; specifically:

[0035] S101, driving a three-dimensional laser radar to move in a work site by a movable device, and collecting laser positioning data of each point in the work site according to the three-dimensional laser radar during the movement.

[0036] S102: Establishing an elevation map of the work site from a global perspective based on the collected laser positioning data.

[0037] S103, according to the target local elevation map corresponding to the target silo in the elevation map, determine the material stacking area whose height information is higher than the ground height from the target local elevation map, and determine the material storage amount in the target silo according to the average height and area of ​​the material stacking area in the target local elevation map.

[0038] The above-mentioned silo storage capacity sensing method provided in the embodiment of the present application drives the three-dimensional laser radar to move in the work site through a movable device, and can perform laser positioning on the work site during the movement of the three-dimensional laser radar, thereby greatly reducing the installation cost of the sensing equipment in the work site, which is conducive to the popularization and promotion of the silo storage capacity sensing method in medium and large-scale work sites. At the same time, it can also reduce the calculation deviation caused by the need to perform coordinate calibration between the sensing equipment and the silo position, effectively improving the accuracy of the estimated results of the silo storage capacity.

[0039] The following is an exemplary description of each step in the above-mentioned method for sensing the storage capacity of the silo provided in the embodiment of the present application:

[0040] S101, driving a three-dimensional laser radar to move in a work site by a movable device, and collecting laser positioning data of each point in the work site according to the three-dimensional laser radar during the movement.

[0041] Here, the above-mentioned movable device can be a work vehicle that can move freely in the work site (for example, the movable device can be a loader), and the above-mentioned movable device can also be a transmission device installed in the work site that can drive objects to move (for example, the movable device can also be a crane that is horizontally mounted above each silo in the work site to lift materials); the embodiment of the present application does not impose any limitation on the specific device type of the above-mentioned movable device.

[0042] In an embodiment of the present application, taking the above-mentioned movable device as an example of a loader in the work site, when executing the above-mentioned step S101, during the historical moving operation of the loader, the laser positioning data of each point in the work site can be collected according to the three-dimensional laser radar installed on the loader. The three-dimensional laser radar is mainly used for the global positioning of the loader and the establishment of an elevation map that can represent the environmental information in the work site; among them, in order to save the installation cost of the sensing equipment in the work site to the greatest extent, only one three-dimensional laser radar can be installed on the loader, and in order to reduce the field of view obstruction of the three-dimensional laser radar during the moving operation of the loader, the three-dimensional laser radar can also be preferably installed on the top of the loader.

[0043] It should be noted that one or more three-dimensional laser radars can be installed on the loader, wherein the three-dimensional laser radar can be installed on the top of the loader or at other locations on the loader. The embodiments of the present application do not impose any limitation on the number of three-dimensional laser radars installed on the loader and the specific installation locations of the three-dimensional laser radars on the loader.

[0044] Here, the above-mentioned historical mobile operation period characterizes the mobile operation process of the loader in the work site at a historical moment (i.e., the moment before the current moment); wherein the above-mentioned mobile operation process includes but is not limited to: the moving driving process of the loader in the work site, the loader shoveling operation process, etc.

[0045] Specifically, according to the three-dimensional laser radar installed on the loader, the laser positioning data of each point in the work site (including the plane position coordinates and spatial height information at each position point) can be collected during one historical mobile operation of the loader, and the laser positioning data of each point in the work site can also be collected during multiple historical mobile operations of the loader; wherein, the embodiment of the present application does not impose any limitation on the number of historical collection times corresponding to the above-mentioned laser positioning data.

[0046] It should be noted that, considering that the positioning laser emitted by the three-dimensional laser radar usually has only 16 or 32 lines (that is, 16 laser lines or 32 laser lines can be emitted simultaneously), therefore, when the area of ​​the work site is large (that is, greater than a certain area threshold), multiple sets of laser positioning data collected by the three-dimensional laser radar for each point in the work site during the loader's multiple historical mobile operations can be obtained (which is also equivalent to driving the three-dimensional laser radar to move multiple times in the work site through a movable device, and obtaining multiple sets of laser positioning data collected by the three-dimensional laser radar during the multiple movements) to avoid the point cloud data formed according to the laser positioning data (used to subsequently establish an elevation map of the work site) being too sparse.

[0047] It should be noted that the above-mentioned work site is a work site that includes silos. For example, the work site may be a concrete mixing station, an asphalt mixing station, etc. The embodiment of the present application does not impose any limitation on the specific site type to which the above-mentioned work site belongs and the specific number of silos included in the above-mentioned work site.

[0048] S102: Establishing an elevation map of the work site from a global perspective based on the collected laser positioning data.

[0049] here, Figure 2a A schematic diagram of a work site provided in an embodiment of the present application is shown. Figure 2a As shown, the work site may include entities such as the ground, walls, fences (used to divide the vehicle entrance and exit of each silo), beams (used to support the roof and can be used as a boundary between two adjacent silos), work vehicles (such as loaders), materials stacked in the silo (i.e., piles of materials), etc., wherein, taking the movable device in step S101 as an example that the loader is a loader, during the moving operation of the loader, the three-dimensional laser radar can emit a positioning laser in the work site, and when the positioning laser scans the surface of any of the above entities, the above-mentioned laser positioning data reflected back will include the coordinate information of each point on the surface of the entity in the three-dimensional space (also equivalent to the plane position coordinates and spatial height information), and the combination of these points is the point cloud data obtained by scanning the three-dimensional laser radar (also equivalent to the above-mentioned laser positioning data).

[0050] Specifically, an elevation map (i.e., an elevation map of the work site from a global perspective) can be generated based on the above-mentioned point cloud data (i.e., the above-mentioned laser positioning data) through a variety of methods. For example, the above-mentioned point cloud data can be converted into an elevation map through a DEM (Digital Elevation Model) model, or the above-mentioned point cloud data can be converted into an elevation map through a DTM (digital elevation model) model, etc. The specific tools for realizing the conversion between the point cloud data (i.e., the above-mentioned laser positioning data) and the elevation map (the elevation map of the work site from a global perspective) in step S102 are not limited in any way in the embodiments of the present application.

[0051] It should be noted that the elevation map is an important form used in the embodiments of the present application to represent the environment in the work site. Figure 1 It is generally used for real-time perception of the surrounding environment of a legged robot, and has the ability to fuse historical information from multiple sensors. However, in the embodiment of the present application, unlike general robot applications, when establishing an elevation map of the work site from a global perspective, the present application sets the coordinate system of the elevation map to the world coordinate system (i.e., the world coordinate system positioned in the above-mentioned laser positioning data), and sets the size information of the elevation map to the global size information of the work site (i.e., the global size information of the work site) rather than the surrounding area of ​​the robot.

[0052] An exemplary description is given with Figure 2a Take the work site shown as an example. Figure 2b A schematic diagram of an elevation map of a work site from a global perspective provided by an embodiment of the present application is shown, such as Figure 2b As shown, the elevation map can be used to represent the environmental information within the work site, wherein, from the elevation map, the plane position coordinates (i.e., equivalent to the plane position coordinates of the ground within the work site) and the height information of each spatial position point (i.e., the position point with height information) scanned by the three-dimensional laser radar within the work site can be determined.

[0053] S103, according to the target local elevation map corresponding to the target silo in the elevation map, determine the material stacking area whose height information is higher than the ground height from the target local elevation map, and determine the material storage amount in the target silo according to the average height and area of ​​the material stacking area in the target local elevation map.

[0054] Here, the target silo represents a silo located in the work site, that is, for any silo in the work site, the material storage capacity in the silo (that is, the material stacking volume in the silo) can be estimated in the manner described in step S103.

[0055] Specifically, Figure 3A schematic diagram of calculating the material storage amount in a target silo provided in an embodiment of the present application is shown. Figure 3 As shown, the coordinates of the four vertices of the target silo ABCD can be obtained from Figure 2b Determined in the elevation map shown (ref. Figure 2b The four vertices ABCD shown in the figure are shown in the figure. If it is assumed that the opening of the target silo (i.e., the vehicle entrance and exit) is on the AD side, then on the BC side, multiple points can be discretized from left to right according to the resolution of the elevation map grid (the resolution of the elevation map grid refers to the physical size of each grid in the DEM dataset, that is, the actual geographic space size represented by each grid on the surface). For each grid s (i.e., grid s), extend and visit each elevation map grid (i.e., elevation map grid) along the opening direction of AD in a route parallel to the AB side. If the grid height exists (the non-existent grid is equivalent to the grid area whose height information is less than or equal to the above ground height), continue to extend and visit the next one. If the next grid height does not exist, stop and calculate the distance sf between the last visited grid f and the starting grid s. After traversing all the grids on the BC side in the above manner, the material stacking volume V in the target silo ABCD (i.e., the material storage in the target silo) can be estimated as:

[0056]

[0057] Among them, h represents the material stacking area (i.e. Figure 3 The average height of each grid area under the curve in the target silo ABCD in the target local elevation map;

[0058] r is the side length of the elevation map grid (i.e., the elevation map grid resolution);

[0059] |BC| represents the length of the BC side in the target silo ABCD;

[0060] n represents the number of elevation map grids contained on the BC side of the target silo ABCD.

[0061] It should be noted that for other objects that may exist in the target silo (such as temporarily parked vehicles or fences that may appear in the work area), the above calculation method can be used to filter the non-material storage area belonging to the above other objects based on the continuity assumption of the material; for example, Figure 3 As shown, the grid corresponding to the obstacle O is discontinuous with the grid f, so when estimating the material stacking volume V in the target silo ABCD according to the above calculation method, the grid corresponding to the above obstacle O can be effectively filtered out, thereby improving the accuracy of the estimated result of the silo storage capacity.

[0062] Taking the above-mentioned movable device as a loader in the working site as an example, the specific implementation process of each of the above-mentioned steps in the embodiment of the present application is described in detail below:

[0063] With regard to the specific implementation process of the above step S102, when the three-dimensional laser radar collects laser positioning data, the emitted laser rays may scan the loader body or bucket (equivalent to scanning a position point belonging to the mobile device body but not to the work site), resulting in the presence of erroneous height information corresponding to the loader body or bucket part in the final collected laser positioning data (such as Figure 2b As shown in the figure, due to the laser beam erroneously scanning the loader body, erroneous height information framed in a white rectangular box is generated on the loader's travel route. At this time, as an optional embodiment, the above step S102 can be performed according to the method shown in the following steps a1-a3, and the above elevation map is established using the laser positioning data after filtering out the above erroneous height information. Specifically:

[0064] Step a1: According to the relative distance between the installation position of the three-dimensional laser radar and the edge of the loader body, the laser positioning data of each point on the loader body is filtered out from the laser positioning data to obtain target laser positioning data representing the environmental information in the work site.

[0065] Specifically, as an optional embodiment, the laser positioning data corresponding to the position points whose distance from the installation position of the three-dimensional laser radar is less than or equal to the above-mentioned relative distance can be filtered out from the laser positioning data (which is also equivalent to deleting the above-mentioned position points from the point cloud data). That is, when the distance between a position point and the installation position of the three-dimensional laser radar is less than or equal to the above-mentioned relative distance, it can be determined that the position point belongs to the position point on the loader body (that is, the position point that needs to be filtered out).

[0066] Step a2: Create a target elevation map whose coordinate system is the same as the world coordinate system and whose size information is the same as the global size information according to the world coordinate system corresponding to the target laser positioning data and the global size information of the work site.

[0067] Here, the specific implementation of step a2 can refer to the specific implementation of the aforementioned step S102. It is only necessary to replace the laser positioning data in step S102 with the above-mentioned target laser positioning data after filtering out the erroneous height information (i.e., the laser positioning data of each point on the loader body). The repetitions will not be repeated here.

[0068] Step a3: determining the target elevation map as the elevation map of the work site from a global perspective.

[0069] In the embodiment of the present application, considering that the estimation of the silo reserves does not need to ensure that the estimation result has a very high accuracy (the estimation result of the silo reserves is mainly used to help the management users in the work site to reasonably formulate storage and management plans for the materials according to the current reserves of the materials in the silo), therefore, as an optional embodiment, the method described in the above steps a1-a3 can be used to establish a more accurate elevation map using the more accurate target laser positioning data after filtering out the above erroneous height information. As another optional embodiment, the method of the above step S102 can be used to directly use the collected laser positioning data to establish a slightly rough elevation map. The embodiment of the present application does not impose any limitation on the specific method for establishing the above elevation map and the specific laser positioning data used to establish the above elevation map.

[0070] Here, based on the method described in the above steps a1-a3, considering that the three-dimensional laser radar is installed on the loader, the three-dimensional laser radar will inevitably lack some data collection perspectives (that is, generate some data collection blind spots) when following the loader to move. At this time, as an optional embodiment, after executing the above step a2, the hole values ​​generated by the above data collection blind spots in the above target elevation map can be filled by executing the following step b1 to obtain a more accurate elevation map, specifically:

[0071] Step b1, filling the hole values ​​belonging to the target grid area in the target elevation map through an interpolation algorithm, and obtaining the filled target elevation map as the elevation map of the work site in a global perspective.

[0072] It should be noted that the specific algorithm type to which the above interpolation algorithm belongs is not unique. For example, the above interpolation algorithm can be a point-by-point comparison interpolation algorithm, or a digital integral interpolation algorithm, etc. The embodiment of the present application does not impose any limitation on the specific algorithm type to which the above interpolation algorithm belongs.

[0073] Here, the above-mentioned target grid area can be determined based on the data collection blind area of ​​the three-dimensional laser radar during the movement of the loader. For example, the above-mentioned data collection blind area can be the upper surface area of ​​the material pile in the silo (that is, the three-dimensional laser radar can collect the front stacking information and side stacking information of the material pile during the movement of the loader, but cannot collect the upper surface information of the material pile due to the lack of a top-down perspective).

[0074] Specifically, taking the above-mentioned data collection blind area as the upper surface area of ​​the material pile in the silo as an example, the above-mentioned target grid area can be the grid area adjacent to the highest point of the material pile in the target elevation map. At this time, the height information of the target grid area (that is, the hole value with a height of zero) can be filled with the height information of the adjacent grid area through an interpolation algorithm, and the filled target elevation map is obtained as the elevation map of the work site from a global perspective.

[0075] Regarding the specific implementation process of the above step S103, considering that the elevation map can only represent the coordinate position and height information of each point in the work site, but cannot accurately identify the specific entity category to which each point belongs (for example, the location points belonging to the ground area cannot be accurately identified from the elevation map), therefore, as an optional embodiment, when executing the above step S103, the ground height of the work site can be determined according to the method described in the following steps c1-c3, specifically:

[0076] Step c1, according to the local idle area in front of the silo in the work site, determine the local elevation map corresponding to the local idle area from the elevation map.

[0077] Here, based on the compliance operation mode, the main driving routes within the work site (such as Figure 2b As shown, no debris is allowed to be piled up on the main road in front of each silo. Therefore, as an optional embodiment, the area in the work site that is located in front of the silo and belongs to the main road can be used as the above-mentioned local free area where no debris is piled up, and the local elevation map corresponding to the above-mentioned local free area can be determined from the elevation map.

[0078] Step c2: determining the average height of the local idle area in the local elevation map as the ground height.

[0079] Here, based on the height information of each point in the above-mentioned local elevation map, the average height of the above-mentioned local elevation map can be calculated, and the calculated average height can be used as the replacement result of the actual ground height in the work site (that is, the above-mentioned ground height).

[0080] Step c3: Determine from the target local elevation map that an area whose height information is higher than the ground height belongs to the material stacking area.

[0081] Specifically, after the ground height is determined, it can be determined from the target local elevation map that an area with height information higher than the ground height belongs to a material storage area where materials are stored in the target silo.

[0082] In the embodiment of the present application, in order to improve the recognition accuracy of the material stacking area and obtain a more accurate silo storage estimation result, in an optional implementation manner, Figure 4 A schematic diagram of a high-level version of a silo storage capacity sensing method provided in an embodiment of the present application is shown. Figure 4 As shown, the method includes steps S401-S404, specifically:

[0083] S401, driving a camera sensor to move in the work site by means of the movable device, and collecting an environmental image in the work site according to the camera sensor during the movement.

[0084] Here, similar to the above step S101, when the above movable device is a loader in the work site, when executing the above step S401, during the historical moving operation of the loader, the environmental image in the work site can be collected according to the camera sensor installed on the loader.

[0085] In the embodiment of the present application, taking the above-mentioned movable device as an example, which is a loader in the work site, similar to the installation position of the three-dimensional laser radar, in order to save the installation cost of the sensing equipment in the work site to the greatest extent, only one camera sensor can be installed on the loader, and in order to reduce the obstruction of the camera sensor's field of view during the movement of the loader, the camera sensor can also be preferably installed on the top of the loader.

[0086] It should be noted that one camera sensor or multiple camera sensors can be installed on the loader. The camera sensor can be installed on the top of the loader or at other locations on the loader. The embodiments of the present application do not impose any limitations on the number of camera sensors installed on the loader and the specific installation locations of the camera sensors on the loader.

[0087] S402, identifying target entities from the environment image through a semantic segmentation algorithm to obtain entity distribution recognition results in the work site.

[0088] Here, the semantic segmentation algorithm can specifically be a pre-trained image segmentation model. The above-mentioned environmental image is input into the image segmentation model, and the image segmentation model can output an image recognition result for the environmental image (that is, the recognition results of the above-mentioned entities respectively), wherein the image recognition result includes: the specific entity category to which the above-mentioned target entity belongs and the specific position of the above-mentioned target entity in the environmental image.

[0089] Specifically, the above-mentioned target entities include but are not limited to: the ground, work vehicles, material piles, fences (placed in front of the silo, and work vehicles can enter the silo from the fences) and other entities that may appear in the work site; the embodiment of the present application does not impose any limitation on the scope of entity categories included in the above-mentioned target entities.

[0090] S403, according to the entity distribution recognition result in the work site, determine from the target local elevation map the grid area whose entity category belongs to the material pile as the material stacking area.

[0091] It should be noted that in order to enable the above-mentioned entity distribution recognition results to be integrated with the elevation map, the camera sensor and the three-dimensional lidar can be calibrated with external parameters in advance so that the above-mentioned entity distribution recognition results can be integrated with the elevation map, and then the specific entity category to which each grid area belongs can be determined from the target local elevation map, so as to identify the grid area belonging to the material pile as the material stacking area.

[0092] Exemplary description, Figure 5 A schematic diagram of an elevation map fused with entity distribution recognition results provided by an embodiment of the present application is shown, such as Figure 5 As shown, compared to Figure 2b The elevation map shown in FIG. 1 can accurately identify the local area 501 of the fence in front of each silo and the working vehicle 502 temporarily parked in the silo (in the elevation map after being fused with the entity distribution recognition result). Figure 5 The rectangular frame is used to frame it), thereby effectively improving the accuracy of the estimated results of the storage in each silo in the work site.

[0093] S404: Determine the material storage amount in the target silo according to the average height and area of ​​the material storage area in the target local elevation map.

[0094] Here, the specific implementation of step S404 is the same as the specific implementation of the aforementioned step S103, and the repeated parts are not repeated here.

[0095] In the embodiment of the present application, in the method described in the above steps c1-c3, since the ground height is replaced by the average value of the height in the local free area in front of each silo, in special circumstances (such as the occurrence of non-compliant operations), if there are debris piled in the above local free area, the ground height calculated in the above steps c1-c3 will have a large error, resulting in a decrease in the accuracy of the final estimated silo storage result. In steps S401-S404, after adding the above camera sensor, the ground area can be accurately identified from the elevation map according to the above entity distribution recognition result. Therefore, as an optional embodiment, after adding the above camera sensor, when executing the above step S103, the ground height can also be determined according to the method described in the following steps d1-d3, specifically:

[0096] Step d1, according to the entity distribution recognition result in the work site, determine from the elevation map the grid area whose entity category belongs to the ground as the ground area in the work site.

[0097] Here, referring to the method described in the above steps S401-S404, the grid area belonging to the ground (ie, the ground area) can be accurately identified from the elevation map without using the local free area in front of the silo as a substitute as described in the above steps c1-c3.

[0098] Step d2: determining the average height of the ground area in the elevation map as the ground height.

[0099] Here, after the above-mentioned ground area is identified, the average height of the above-mentioned ground area in the elevation map can be calculated according to the height information of each point in the grid area belonging to the ground as the ground height in the work site.

[0100] Step d3: Determine from the target local elevation map that an area whose height information is higher than the ground height belongs to the material stacking area.

[0101] Here, the specific implementation of step d3 is the same as the specific implementation of the aforementioned step c3, and the repeated parts are not repeated here.

[0102] In the embodiment of the present application, considering that the method based on the above steps S101-S103 requires the loader to pass through the silo in the work site before generating new laser positioning data to refresh the perception result (i.e., the estimated result) of the storage in the silo, therefore, in an optional implementation, Figure 6 A schematic diagram of a method for updating the estimated storage result in a silo provided in an embodiment of the present application is shown. Figure 6 As shown, the method includes steps S601-S604, specifically:

[0103] S601, during the current moving operation of the loader, current laser positioning data of each point in the work site is collected according to the three-dimensional laser radar installed on the loader.

[0104] Here, the specific implementation of step S601 is the same as the specific implementation of the aforementioned step S101, and the repeated parts are not repeated here.

[0105] S602: Update the elevation map according to the current laser positioning data to obtain an updated version of the elevation map.

[0106] Here, according to the above-mentioned current laser positioning data, the laser scanning time and height information of each point in the elevation map can be updated to obtain an updated version of the elevation map.

[0107] S603: Determine the number of updated grid areas in the target local elevation map according to the new version of the target local elevation map corresponding to the target silo in the updated version of the elevation map.

[0108] Here, the updated grid area in the target local elevation map is the grid area where the height information in the target silo changes.

[0109] S604, when it is determined that the number of updated grid areas is greater than or equal to a preset number threshold, an updated material stacking area is determined from the new version of the target local elevation map, and the updated material storage amount in the target silo is determined based on the average height and area of ​​the updated material stacking area in the new version of the target local elevation map.

[0110] Here, the specific implementation method of determining the updated material storage in the target silo in step S604 is the same as the specific implementation method of determining the material storage in the target silo in the aforementioned step S103. It is only necessary to replace the material stacking area in step S103 with the updated material stacking area, and the repetitions are not repeated here.

[0111] Specifically, in steps S601-S604, when the update ratio of the elevation map grid in the target silo reaches the set threshold number (that is, the number of updated grid areas is greater than or equal to the preset threshold number), the calculation of the material reserves in the target silo will be triggered and the update time of the current material reserves will be released. At the same time, the update time of all grids in the target silo can also be reset. This method can reduce the computational cost of material reserve estimation, and only needs to be recalculated when the number of grid refreshes reaches the threshold. At the same time, the user or the upper-level application can know the effective update time of the capacity information of all silos in the work site (that is, the material reserves in all silos). For silos that have not updated their capacity information for a long time, it will automatically trigger or suggest that users go to the silo for perception update, so as to avoid the situation where the capacity perception information is outdated and inconsistent due to not going to the silo area for a long time.

[0112] Based on the above-mentioned silo storage capacity perception method provided in the embodiment of the present application, the three-dimensional laser radar is driven by a movable device to move in the work site, and the work site can be laser positioned during the movement of the three-dimensional laser radar, which greatly reduces the installation cost of the sensing equipment in the work site, and is conducive to the popularization and promotion of the silo storage capacity perception method in medium and large-scale work sites. At the same time, it can also reduce the calculation deviation caused by the need to perform coordinate calibration between the sensing equipment and the silo position, effectively improving the accuracy of the estimated results of the silo storage capacity.

[0113] Based on the same inventive concept, the present application also provides a sensing device corresponding to the above-mentioned method for sensing the silo storage capacity. Since the principle of solving the problem by the sensing device in the embodiment of the present application is similar to the above-mentioned method for sensing the silo storage capacity in the embodiment of the present application, the implementation of the sensing device can refer to the implementation of the above-mentioned sensing method, and the repeated parts will not be repeated.

[0114] Reference Figure 7 As shown, Figure 7 The schematic diagram of the structure of a silo storage capacity sensing device provided in an embodiment of the present application is shown, wherein the silo storage capacity sensing device comprises:

[0115] The acquisition module 701 is used to drive the three-dimensional laser radar to move in the work site through a movable device, and during the movement, collect laser positioning data of each point in the work site according to the three-dimensional laser radar;

[0116] A mapping module 702 is used to establish an elevation map of the work site from a global perspective based on the collected laser positioning data;

[0117] The perception module 703 is used to determine the material storage area whose height information is higher than the ground height from the target local elevation map according to the target local elevation map corresponding to the target silo in the elevation map, and determine the material reserves in the target silo according to the average height and area of ​​the material storage area in the target local elevation map; wherein the target silo represents a silo located in the work site.

[0118] In an optional embodiment, the movable device includes a loader, and the acquisition module 701 is used to collect laser positioning data of each point in the work site according to a three-dimensional laser radar installed on the loader during the historical mobile operation of the loader.

[0119] In an optional implementation, when establishing an elevation map of the work site from a global perspective based on the collected laser positioning data, the mapping module 702 is used to:

[0120] According to the relative distance between the installation position of the three-dimensional laser radar and the edge of the body of the loader, the laser positioning data of each point on the body of the loader is filtered out from the laser positioning data to obtain target laser positioning data representing the environmental information in the work site;

[0121] Creating a target elevation map whose coordinate system is the same as the world coordinate system and whose size information is the same as the global size information according to the world coordinate system corresponding to the target laser positioning data and the global size information of the work site;

[0122] The target elevation map is determined as an elevation map of the work site from a global perspective.

[0123] In an optional implementation, after creating the target elevation map whose coordinate system is the same as the world coordinate system and whose size information is the same as the global size information, the mapping module 702 is further used to:

[0124] Through the interpolation algorithm, the hole values ​​belonging to the target grid area in the target elevation map are filled, and the filled target elevation map is obtained as the elevation map of the work site from a global perspective; wherein, the target grid area is determined according to the data collection blind area of ​​the three-dimensional laser radar during the movement of the loader.

[0125] In an optional implementation, when determining the material storage area whose height information is higher than the ground height from the target local elevation map, the perception module 703 is used to:

[0126] According to the local idle area in front of the silo in the work site, determining a local elevation map corresponding to the local idle area from the elevation map;

[0127] Determine the average height of the local idle area in the local elevation map as the ground height;

[0128] It is determined from the target local elevation map that an area whose height information is higher than the ground height belongs to the material stacking area.

[0129] In an optional implementation, the sensing device further includes: a backup sensing module, wherein the backup sensing module is used to:

[0130] The movable device drives the camera sensor to move in the work site, and in the process of movement, the camera sensor collects an environmental image in the work site;

[0131] Using a semantic segmentation algorithm, target entities are identified from the environment image to obtain entity distribution recognition results in the work site; wherein the target entities include: the ground, work vehicles, and material piles;

[0132] According to the entity distribution recognition result in the work site, a grid area whose entity category belongs to the material pile is determined from the target local elevation map as the material stacking area;

[0133] The material storage capacity in the target silo is determined according to the average height and area of ​​the material storage area in the target local elevation map.

[0134] In an optional implementation, when determining the material storage area whose height information is higher than the ground height from the target local elevation map, the perception module 703 is further used to:

[0135] According to the entity distribution recognition result in the work site, determine from the elevation map a grid area whose entity category belongs to the ground as a ground area in the work site;

[0136] Determine the average height of the ground area in the elevation map as the ground height;

[0137] It is determined from the target local elevation map that an area whose height information is higher than the ground height belongs to the material stacking area.

[0138] In an optional implementation, the sensing device further includes: an updating module, wherein the updating module is used to:

[0139] During the current mobile operation of the loader, collecting current laser positioning data of each point in the work site according to the three-dimensional laser radar installed on the loader;

[0140] According to the current laser positioning data, the elevation map is updated to obtain an updated version of the elevation map;

[0141] Determine the number of grid areas updated in the target local elevation map according to the new version of the target local elevation map corresponding to the target silo in the updated version of the elevation map;

[0142] When it is determined that the number of updated grid areas is greater than or equal to the preset number threshold, the updated material stacking area is determined from the new version of the target local elevation map, and the updated material storage capacity in the target silo is determined based on the average height and area of ​​the updated material stacking area in the new version of the target local elevation map.

[0143] Based on the above-mentioned silo storage capacity sensing device provided in the embodiment of the present application, the three-dimensional laser radar is driven by a movable device to move in the work site, and the work site can be laser positioned during the movement of the three-dimensional laser radar, which greatly reduces the installation cost of the sensing equipment in the work site, is conducive to the popularization and promotion of the silo storage capacity sensing method in medium and large-scale work sites, and can also reduce the calculation deviation caused by the need to perform coordinate calibration between the sensing equipment and the silo position, effectively improving the accuracy of the estimated results of the silo storage capacity.

[0144] Based on the same inventive concept, the present application also provides an electronic device corresponding to the above-mentioned method for sensing the silo storage capacity. Since the principle of solving the problem by the electronic device in the embodiment of the present application is similar to the above-mentioned method for sensing the silo storage capacity in the embodiment of the present application, the implementation of the electronic device can refer to the implementation of the above-mentioned sensing method, and the repeated parts will not be repeated.

[0145] Figure 8A schematic diagram of the structure of an electronic device 800 provided in an embodiment of the present application includes: a processor 801, a memory 802 and a bus 803, wherein the memory 802 stores machine-readable instructions executable by the processor 801. When the electronic device runs a method for sensing the storage amount of a silo in an embodiment, the processor 801 communicates with the memory 802 via the bus 803, and the processor 801 executes the machine-readable instructions. When the processor 801 executes the machine-readable instructions, the following steps are implemented, specifically:

[0146] The three-dimensional laser radar is driven to move in the work site by a movable device, and during the movement, the laser positioning data of each point in the work site is collected according to the three-dimensional laser radar;

[0147] Establishing an elevation map of the work site from a global perspective based on the collected laser positioning data;

[0148] According to the target local elevation map corresponding to the target silo in the elevation map, a material stacking area whose height information is higher than the ground height is determined from the target local elevation map, and the material reserves in the target silo are determined according to the average height and area of ​​the material stacking area in the target local elevation map; wherein the target silo represents a silo located in the work site.

[0149] In an optional implementation, the movable device includes a loader, and during the movement, when collecting laser positioning data of each point in the work site according to the three-dimensional laser radar, the processor 801 is used to:

[0150] During the historical moving operation of the loader, laser positioning data of each point in the work site is collected based on the three-dimensional laser radar installed on the loader.

[0151] In an optional implementation, when establishing an elevation map of the work site from a global perspective based on the collected laser positioning data, the processor 801 is used to:

[0152] According to the relative distance between the installation position of the three-dimensional laser radar and the edge of the body of the loader, the laser positioning data of each point on the body of the loader is filtered out from the laser positioning data to obtain target laser positioning data representing the environmental information in the work site;

[0153] Creating a target elevation map whose coordinate system is the same as the world coordinate system and whose size information is the same as the global size information according to the world coordinate system corresponding to the target laser positioning data and the global size information of the work site;

[0154] The target elevation map is determined as an elevation map of the work site from a global perspective.

[0155] In an optional implementation, after creating the target elevation map whose coordinate system is the same as the world coordinate system and whose size information is the same as the global size information, the processor 801 is further configured to:

[0156] Through the interpolation algorithm, the hole values ​​belonging to the target grid area in the target elevation map are filled, and the filled target elevation map is obtained as the elevation map of the work site from a global perspective; wherein, the target grid area is determined according to the data collection blind area of ​​the three-dimensional laser radar during the movement of the loader.

[0157] In an optional implementation, when determining the material storage area whose height information is higher than the ground height from the target local elevation map, the processor 801 is used to:

[0158] According to the local idle area in front of the silo in the work site, determining a local elevation map corresponding to the local idle area from the elevation map;

[0159] Determine the average height of the local idle area in the local elevation map as the ground height;

[0160] It is determined from the target local elevation map that an area whose height information is higher than the ground height belongs to the material stacking area.

[0161] In an optional implementation, the processor 801 is further configured to:

[0162] The movable device drives the camera sensor to move in the work site, and in the process of movement, the camera sensor collects an environmental image in the work site;

[0163] Using a semantic segmentation algorithm, target entities are identified from the environment image to obtain entity distribution recognition results in the work site; wherein the target entities include: the ground, work vehicles, and material piles;

[0164] According to the entity distribution recognition result in the work site, a grid area whose entity category belongs to the material pile is determined from the target local elevation map as the material stacking area;

[0165] The material storage capacity in the target silo is determined according to the average height and area of ​​the material storage area in the target local elevation map.

[0166] In an optional implementation, when determining the material storage area whose height information is higher than the ground height from the target local elevation map, the processor 801 is further configured to:

[0167] According to the entity distribution recognition result in the work site, determine from the elevation map a grid area whose entity category belongs to the ground as a ground area in the work site;

[0168] Determine the average height of the ground area in the elevation map as the ground height;

[0169] It is determined from the target local elevation map that an area whose height information is higher than the ground height belongs to the material stacking area.

[0170] In an optional implementation, the processor 801 is further configured to:

[0171] During the current mobile operation of the loader, collecting current laser positioning data of each point in the work site according to the three-dimensional laser radar installed on the loader;

[0172] According to the current laser positioning data, the elevation map is updated to obtain an updated version of the elevation map;

[0173] Determine the number of grid areas updated in the target local elevation map according to the new version of the target local elevation map corresponding to the target silo in the updated version of the elevation map;

[0174] When it is determined that the number of updated grid areas is greater than or equal to the preset number threshold, the updated material stacking area is determined from the new version of the target local elevation map, and the updated material storage capacity in the target silo is determined based on the average height and area of ​​the updated material stacking area in the new version of the target local elevation map.

[0175] Through the above-mentioned electronic device provided in the embodiment of the present application, the three-dimensional laser radar is driven to move in the work site through a movable device, and the work site can be laser positioned during the movement of the three-dimensional laser radar, which greatly reduces the installation cost of the sensing equipment in the work site, is conducive to the popularization and promotion of the silo storage sensing method in medium and large-scale work sites, and can also reduce the calculation deviation caused by the need to perform coordinate calibration between the sensing equipment and the silo position, effectively improving the accuracy of the estimated results of the silo storage capacity.

[0176] Based on the same inventive concept, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. The computer program is executed when a processor is running, and the processor performs the following steps:

[0177] The three-dimensional laser radar is driven to move in the work site by a movable device, and during the movement, the laser positioning data of each point in the work site is collected according to the three-dimensional laser radar;

[0178] Establishing an elevation map of the work site from a global perspective based on the collected laser positioning data;

[0179] According to the target local elevation map corresponding to the target silo in the elevation map, a material stacking area whose height information is higher than the ground height is determined from the target local elevation map, and the material reserves in the target silo are determined according to the average height and area of ​​the material stacking area in the target local elevation map; wherein the target silo represents a silo located in the work site.

[0180] In an optional embodiment, the movable device includes a loader, and during the movement, when collecting laser positioning data of each point in the work site according to the three-dimensional laser radar, the processor is used to:

[0181] During the historical moving operation of the loader, laser positioning data of each point in the work site is collected based on the three-dimensional laser radar installed on the loader.

[0182] In an optional implementation, when establishing an elevation map of the work site from a global perspective based on the collected laser positioning data, the processor is used to:

[0183] According to the relative distance between the installation position of the three-dimensional laser radar and the edge of the body of the loader, the laser positioning data of each point on the body of the loader is filtered out from the laser positioning data to obtain target laser positioning data representing the environmental information in the work site;

[0184] Creating a target elevation map whose coordinate system is the same as the world coordinate system and whose size information is the same as the global size information according to the world coordinate system corresponding to the target laser positioning data and the global size information of the work site;

[0185] The target elevation map is determined as an elevation map of the work site from a global perspective.

[0186] In an optional implementation, after creating the target elevation map whose coordinate system is the same as the world coordinate system and whose size information is the same as the global size information, the processor is further configured to:

[0187] Through the interpolation algorithm, the hole values ​​belonging to the target grid area in the target elevation map are filled, and the filled target elevation map is obtained as the elevation map of the work site from a global perspective; wherein, the target grid area is determined according to the data collection blind area of ​​the three-dimensional laser radar during the movement of the loader.

[0188] In an optional implementation, when determining the material storage area whose height information is higher than the ground height from the target local elevation map, the processor is used to:

[0189] According to the local idle area in front of the silo in the work site, determining a local elevation map corresponding to the local idle area from the elevation map;

[0190] Determine the average height of the local idle area in the local elevation map as the ground height;

[0191] It is determined from the target local elevation map that an area whose height information is higher than the ground height belongs to the material stacking area.

[0192] In an optional implementation, the processor is further configured to:

[0193] The movable device drives the camera sensor to move in the work site, and in the process of movement, the camera sensor collects an environmental image in the work site;

[0194] Using a semantic segmentation algorithm, target entities are identified from the environment image to obtain entity distribution recognition results in the work site; wherein the target entities include: the ground, work vehicles, and material piles;

[0195] According to the entity distribution recognition result in the work site, a grid area whose entity category belongs to the material pile is determined from the target local elevation map as the material stacking area;

[0196] The material storage capacity in the target silo is determined according to the average height and area of ​​the material storage area in the target local elevation map.

[0197] In an optional implementation, when determining the material storage area whose height information is higher than the ground height from the target local elevation map, the processor is further used to:

[0198] According to the entity distribution recognition result in the work site, determine from the elevation map a grid area whose entity category belongs to the ground as a ground area in the work site;

[0199] Determine the average height of the ground area in the elevation map as the ground height;

[0200] It is determined from the target local elevation map that an area whose height information is higher than the ground height belongs to the material stacking area.

[0201] In an optional implementation, the processor is further configured to:

[0202] During the current mobile operation of the loader, collecting current laser positioning data of each point in the work site according to the three-dimensional laser radar installed on the loader;

[0203] According to the current laser positioning data, the elevation map is updated to obtain an updated version of the elevation map;

[0204] Determine the number of grid areas updated in the target local elevation map according to the new version of the target local elevation map corresponding to the target silo in the updated version of the elevation map;

[0205] When it is determined that the number of updated grid areas is greater than or equal to the preset number threshold, the updated material stacking area is determined from the new version of the target local elevation map, and the updated material storage capacity in the target silo is determined based on the average height and area of ​​the updated material stacking area in the new version of the target local elevation map.

[0206] Through the above-mentioned computer-readable storage medium provided in the embodiment of the present application, the three-dimensional laser radar is driven to move in the work site by a movable device, and the work site can be laser positioned during the movement of the three-dimensional laser radar, which greatly reduces the installation cost of the sensing equipment in the work site, is conducive to the popularization and promotion of the silo storage sensing method in medium and large-scale work sites, and can also reduce the calculation deviation caused by the need to perform coordinate calibration between the sensing equipment and the silo position, effectively improving the accuracy of the estimated results of the silo storage capacity.

[0207] In an embodiment of the present application, the computer-readable storage medium can also execute other machine-readable instructions when run by the processor to execute the silo storage capacity perception method as described in other embodiments. For the specific execution steps and principles of the perception method, please refer to the description of the method side embodiment, which will not be repeated here.

[0208] In the embodiments provided in the present application, it should be understood that the disclosed systems and methods can be implemented in other ways. The system embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, indirect coupling or communication connection of systems or units, which can be electrical, mechanical or other forms.

[0209] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0210] In addition, each functional unit in the embodiments provided in the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0211] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0212] It should be noted that similar numbers and letters represent similar items in the following figures. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. In addition, the terms "first", "second", "third", etc. are only used to distinguish the description and are not to be understood as indicating or implying relative importance.

[0213] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solutions of the present application, rather than to limit them. The protection scope of the present application is not limited thereto. Although the present application is described in detail with reference to the above-mentioned embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-mentioned embodiments within the technical scope disclosed in the present application, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application. They should all be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A method for sensing storage capacity of a silo, characterized in that: The sensing method comprises: The three-dimensional laser radar is driven to move in the work site by a movable device, and during the movement, the laser positioning data of each point in the work site is collected according to the three-dimensional laser radar; Establishing an elevation map of the work site from a global perspective based on the collected laser positioning data; According to the target local elevation map corresponding to the target silo in the elevation map, a material stacking area whose height information is higher than the ground height is determined from the target local elevation map, and the material reserves in the target silo are determined according to the average height and area of ​​the material stacking area in the target local elevation map; wherein the target silo represents a silo located in the work site.

2. The sensing method according to claim 1, characterized in that: The movable device includes a loader, and during the movement, the laser positioning data of each point in the working site is collected according to the three-dimensional laser radar, including: During the historical moving operation of the loader, laser positioning data of each point in the work site is collected based on the three-dimensional laser radar installed on the loader.

3. The sensing method according to claim 2, characterized in that: The step of establishing an elevation map of the work site from a global perspective based on the collected laser positioning data includes: According to the relative distance between the installation position of the three-dimensional laser radar and the edge of the body of the loader, the laser positioning data of each point on the body of the loader is filtered out from the laser positioning data to obtain target laser positioning data representing the environmental information in the work site; Creating a target elevation map whose coordinate system is the same as the world coordinate system and whose size information is the same as the global size information according to the world coordinate system corresponding to the target laser positioning data and the global size information of the work site; The target elevation map is determined as an elevation map of the work site from a global perspective.

4. The sensing method according to claim 3, characterized in that: After creating the target elevation map whose coordinate system is the same as the world coordinate system and whose size information is the same as the global size information, the perception method further includes: Through the interpolation algorithm, the hole values ​​belonging to the target grid area in the target elevation map are filled, and the filled target elevation map is obtained as the elevation map of the work site from a global perspective; wherein, the target grid area is determined according to the data collection blind area of ​​the three-dimensional laser radar during the movement of the loader.

5. The sensing method according to claim 1, characterized in that: The step of determining a material storage area having height information higher than the ground height from the target local elevation map includes: According to the local idle area in front of the silo in the work site, determining a local elevation map corresponding to the local idle area from the elevation map; Determine the average height of the local idle area in the local elevation map as the ground height; It is determined from the target local elevation map that an area whose height information is higher than the ground height belongs to the material stacking area.

6. The sensing method according to claim 1, characterized in that: The sensing method further comprises: The movable device drives the camera sensor to move in the work site, and during the movement, the camera sensor collects an environmental image in the work site; Using a semantic segmentation algorithm, target entities are identified from the environment image to obtain entity distribution recognition results in the work site; wherein the target entities include: the ground, work vehicles, and material piles; According to the entity distribution recognition result in the work site, a grid area whose entity category belongs to the material pile is determined from the target local elevation map as the material stacking area; The material storage capacity in the target silo is determined according to the average height and area of ​​the material storage area in the target local elevation map.

7. The sensing method according to claim 6, characterized in that: The step of determining a material storage area having height information higher than the ground height from the target local elevation map further includes: According to the entity distribution recognition result in the work site, determine from the elevation map a grid area whose entity category belongs to the ground as a ground area in the work site; Determine the average height of the ground area in the elevation map as the ground height; It is determined from the target local elevation map that an area whose height information is higher than the ground height belongs to the material stacking area.

8. The sensing method according to claim 2, characterized in that: The sensing method further comprises: During the current mobile operation of the loader, collecting current laser positioning data of each point in the work site according to the three-dimensional laser radar installed on the loader; According to the current laser positioning data, the elevation map is updated to obtain an updated version of the elevation map; Determine the number of grid areas updated in the target local elevation map according to the new version of the target local elevation map corresponding to the target silo in the updated version of the elevation map; When it is determined that the number of updated grid areas is greater than or equal to the preset number threshold, the updated material stacking area is determined from the new version of the target local elevation map, and the updated material storage capacity in the target silo is determined based on the average height and area of ​​the updated material stacking area in the new version of the target local elevation map.

9. A device for sensing storage capacity of a silo, characterized in that: The sensing device comprises: A collection module, used to drive the three-dimensional laser radar to move in the work site through a movable device, and during the movement, collect laser positioning data of each point in the work site according to the three-dimensional laser radar; A mapping module, used to establish an elevation map of the work site from a global perspective based on the collected laser positioning data; A perception module is used to determine a material storage area whose height information is higher than the ground height from the target local elevation map according to the target local elevation map corresponding to the target silo in the elevation map, and to determine the material reserve in the target silo according to the average height and area of ​​the material storage area in the target local elevation map; wherein the target silo represents a silo located in the work site.

10. An electronic device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate via the bus, and when the machine-readable instructions are executed by the processor, the steps of the method for sensing the storage amount of a silo as described in any one of claims 1 to 8 are performed.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method for sensing the storage amount of a silo as claimed in any one of claims 1 to 8 are executed.