Material loading control method, device, equipment, medium and product
Through real-time sensing equipment, the position information of the material loading area is obtained, and the loading path is generated and sent, which solves the problem of difficulty in position matching of unmanned transportation equipment during material loading and transportation, and improves loading and transportation efficiency.
Patent Information
- Application Number
- CN202411376377.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-29
- Publication Date
- 2025-01-07
AI Technical Summary
During the material loading and transportation process, unmanned transportation equipment cannot accurately match the location of the shovel equipment due to material changes and obstacles, resulting in a decrease in loading efficiency and transportation efficiency.
Preset perception equipment is used to obtain the location information of target transportation equipment, shovel equipment, materials and obstacles in the real-time material loading area. Through the loading path model analysis, the target loading path is generated and sent to guide the transportation equipment to accurately load and transport.
It realizes accurate loading and transportation of materials, improves loading and transportation efficiency, and ensures accurate matching and safe transportation of transportation equipment with shovel equipment.
Smart Images
Figure CN119270721A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the technical field of material transportation, and in particular to a material loading control method, a material loading control device, an electronic device, a computer-readable storage medium, and a computer program product. Background Art
[0002] At present, in the process of loading and transporting materials, with the changes in materials and the influence of factors such as obstacles in the transportation process (such as scattered materials, etc.), the unmanned transportation equipment cannot accurately match the position of the shovel loading equipment. For example, there is a deviation between the parking position of the transportation equipment and the parking position of the shovel loading equipment, so that the materials unloaded by the shovel loading equipment cannot be quickly and accurately transported out by the transportation equipment, reducing the loading and transportation efficiency of the materials. Summary of the invention
[0003] To this end, the present disclosure provides a material loading control method, a material loading control device, an electronic device, a computer-readable storage medium and a computer program product to solve the problem of how to improve the efficiency of material loading and transportation.
[0004] In a first aspect, the present disclosure provides a material loading control method, the method comprising: acquiring real-time perception information, the real-time perception information being information obtained by a preset perception device sensing the real-time position of at least one of a target transportation equipment, a target shoveling equipment, materials to be loaded and obstacles in a material loading area; inputting the real-time perception information into a loading path model for processing to obtain a target loading path; and sending the target loading path to the target transportation equipment, the target loading path being used to guide the target transportation equipment to load and transport the materials unloaded by the target shoveling equipment.
[0005] In a second aspect, the present disclosure provides a material loading control device, which includes: an acquisition module for acquiring real-time perception information, the real-time perception information is information obtained by a preset perception device sensing the real-time position of at least one of a target transportation equipment, a target shoveling equipment, materials to be loaded and obstacles in a material loading area; a processing module for inputting the real-time perception information into a loading path model for processing to obtain a target loading path; a sending module for sending the target loading path to the target transportation equipment, the target loading path is used to guide the target transportation equipment to load and transport the materials unloaded by the target shoveling equipment.
[0006] In a third aspect, an embodiment of the present disclosure provides an electronic device, comprising: a memory and a processor; the memory stores a computer program that can be executed by the processor, and when the computer program is executed by the processor, it is a loading control method for any material in the embodiments of the present disclosure.
[0007] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements any material loading control method in the embodiments of the present disclosure.
[0008] In a fifth aspect, an embodiment of the present disclosure provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, it implements any one of the material loading control methods in the embodiments of the present disclosure.
[0009] The material loading control method, material loading control device, electronic device, computer-readable storage medium and computer program product in the embodiments of the present disclosure use a preset sensing device to sense the real-time position of at least one of the target transportation equipment, target shoveling equipment, materials to be loaded and obstacles in the material loading area, and obtain real-time sensing information to facilitate clarifying the position of each equipment, material and obstacle in the material loading area; then, the real-time sensing information is input into the loading path model, so that the loading path model can analyze the real-time position information of each object in the real-time sensing information, thereby obtaining an accurate target loading path that can guide the movement of the target transportation equipment. When the target transportation equipment receives the target loading path, it can load and transport the materials unloaded by the target shoveling equipment according to the target loading path, thereby realizing accurate loading and transportation of materials and improving the loading and transportation efficiency of materials. BRIEF DESCRIPTION OF THE DRAWINGS
[0010] The accompanying drawings are used to provide a further understanding of the embodiments of the present disclosure and constitute a part of the specification. Together with the embodiments of the present disclosure, they are used to explain the present disclosure and do not constitute a limitation of the present disclosure. The above and other features and advantages will become more apparent to those skilled in the art by describing detailed example embodiments with reference to the accompanying drawings, in which:
[0011] Figure 1 A schematic diagram showing a flow chart of a material loading control method provided by an embodiment of the present disclosure;
[0012] Figure 2 A block diagram showing a material loading control device provided by an embodiment of the present disclosure;
[0013] Figure 3 A block diagram showing a material loading control system provided by an embodiment of the present disclosure is shown;
[0014] Figure 4 A schematic diagram showing a sensing area provided by an embodiment of the present disclosure;
[0015] Figure 5A schematic flow chart showing a material loading control method implemented by a material loading control system provided by an embodiment of the present disclosure;
[0016] Figure 6 A schematic diagram showing a target shovel loading device provided by an embodiment of the present disclosure is shown;
[0017] Figure 7 A schematic diagram showing a target parking area provided by an embodiment of the present disclosure;
[0018] Figure 8 A schematic diagram of determining a target parking point provided by an embodiment of the present disclosure is shown;
[0019] Fig. 9 A block diagram of a composition of an electronic device provided by an embodiment of the present disclosure is shown. DETAILED DESCRIPTION
[0020] The specific embodiments of the present disclosure are described in detail below in conjunction with the accompanying drawings. It should be understood that the specific embodiments described herein are only used to illustrate and explain the present disclosure and are not intended to limit the present disclosure. For those skilled in the art, the present disclosure can be implemented without the need for some of these specific details. The following description of the embodiments is only intended to provide a better understanding of the present disclosure by illustrating examples of the present disclosure.
[0021] In order to make the objectives, technical solutions and advantages of the present disclosure more clear, the embodiments of the present disclosure will be further described in detail below with reference to the accompanying drawings.
[0022] During the loading and transportation of materials, in order to achieve the automation goal of the entire process of "collecting-transporting-unloading", unmanned transportation equipment will be used to load and transport materials.
[0023] However, with the changes in materials and the influence of factors such as obstacles in the transportation process (such as scattered materials, etc.), the unmanned transportation equipment may not be able to accurately match the position of the shovel loading equipment. For example, there is a deviation between the parking position of the transportation equipment and the parking position of the shovel loading equipment, so that the materials unloaded by the shovel loading equipment cannot be quickly and accurately transported out by the transportation equipment, reducing the loading and transportation efficiency of the materials.
[0024] The embodiments of the present disclosure provide a material loading control method, a material loading control device, an electronic device, a computer-readable storage medium and a computer program product, which can determine the optimal target loading path according to the acquired real-time perception information, so that the target transportation equipment can load and transport the materials unloaded by the target shoveling equipment, thereby realizing the automated production of loaded materials and improving the loading and transportation efficiency of materials.
[0025] In a first aspect, an embodiment of the present disclosure provides a material loading control method.
[0026] Figure 1 A schematic flow chart of a material loading control method provided by an embodiment of the present disclosure is shown. The material loading control method can be applied to a material loading control device, and the material loading control device can be set on an edge cloud server.
[0027] like Figure 1 As shown, the material loading control method includes but is not limited to the following steps.
[0028] Step S101, obtaining real-time perception information.
[0029] Among them, the real-time sensing information is the information obtained by the preset sensing device sensing at least one of the real-time position of the target transportation equipment in the material loading area, the real-time position of the target shoveling equipment, the real-time position of the material to be loaded, and the real-time position of obstacles (such as other parked vehicles, scattered large pieces of material, etc.).
[0030] In some embodiments, the preset sensing device includes at least one of the following: a radar sensing device, a laser sensing device, and an image acquisition device.
[0031] Step S102: input the real-time sensing information into the loading path model for processing to obtain the target loading path.
[0032] Among them, the loading path model is a model obtained by training a neural network model based on multiple sample perception information.
[0033] The loading path model will analyze the real-time position of the target transport equipment, the real-time position of the target shovel loader, the real-time position of the material to be loaded in the material loading area, and the real-time position of the obstacles in the input real-time perception information, and clarify the path that the target transport equipment should travel to avoid various items in the material loading area, prevent the target transport equipment from colliding with other items, and enable the target transport equipment to accurately receive the materials unloaded by the target shovel loader and safely transport the materials to the destination.
[0034] Step S103: sending the target loading path to the target transportation equipment.
[0035] The target loading path is used to guide the target transport equipment to load and transport the materials unloaded by the target shovel loading equipment.
[0036] When the target transport equipment receives the target loading path, it will move according to the instructions of the target loading path and stop at a suitable position to accurately receive the materials unloaded by the target shovel loading equipment; and then transport the materials to the destination according to the instructions of the target loading path.
[0037] In this embodiment, a preset sensing device is used to sense the real-time position of at least one of the target transportation equipment, the target shoveling equipment, the materials to be loaded and the obstacles in the material loading area, and real-time sensing information is obtained to facilitate clarifying the positions of each equipment, material and obstacle in the material loading area; then, the real-time sensing information is input into the loading path model, so that the loading path model can analyze the real-time position information of each object in the real-time sensing information, so as to obtain an accurate target loading path that can guide the target transportation equipment to move. When the target transportation equipment receives the target loading path, it can load and transport the materials unloaded by the target shoveling equipment according to the target loading path, thereby realizing accurate loading and transportation of materials and improving the loading and transportation efficiency of materials.
[0038] In some exemplary embodiments, before executing step S102 of inputting the real-time perception information into the loading path model for processing to obtain the target loading path, the method also includes: acquiring sample perception information, the sample perception information including sample position information of multiple sample shoveling devices, sample material position information and sample obstacle information; training a neural network model based on the sample position information, sample material position information and sample obstacle information of the multiple sample shoveling devices to obtain the loading path model.
[0039] Among them, the sample sensing information includes sample position information of multiple sample shoveling equipment, sample material position information and sample obstacle information.
[0040] The Neural Networks (NN) model is described based on the mathematical model of neurons. A neural network is a complex network system formed by a large number of simple processing units (called neurons) that are widely interconnected. It reflects many basic characteristics of human brain functions and is a highly complex nonlinear dynamic learning system. Neural networks have large-scale parallel, distributed storage and processing, self-organization, self-adaptation and self-learning capabilities, and are particularly suitable for processing imprecise and fuzzy information processing problems that require considering many factors and conditions at the same time.
[0041] By matching and analyzing the sample position information and sample material position information of multiple sample shoveling devices in the sample perception information, the target parking area information of the sample transport equipment can be clarified, that is, when the sample transport equipment is parked in the target parking area, it can accurately receive the materials unloaded by the sample shoveling equipment. Then, the target parking area information and sample obstacle information of the sample transport equipment are comprehensively considered to clarify which sample obstacles the sample transport equipment needs to avoid in the process of transporting materials, and the target parking area information of the sample transport equipment is determined so that it can drive to the target parking area as quickly as possible. Furthermore, the target parking area information and sample obstacle information of the sample transport equipment are used as sample data to train the neural network model, so that the neural network model can learn how to avoid obstacles to obtain the optimal target loading path, so that the obtained loading path model can accurately analyze the real-time perception information input therein, and quickly output the target loading path for use by the target transport equipment.
[0042] In some exemplary embodiments, obtaining the real-time position information of the target shoveling equipment includes: determining the center point information of the target shoveling equipment according to the obtained three-dimensional size of the target shoveling equipment; and determining the real-time position information of the target shoveling equipment according to the center point information of the target shoveling equipment.
[0043] The three-dimensional size of the target shovel loading equipment includes the length, width and height of the target shovel loading equipment. By constructing a three-dimensional stereoscopic model through the length, width and height of the target shovel loading equipment, a three-dimensional model corresponding to the target shovel loading equipment can be obtained, and the center point of the three-dimensional model can be determined.
[0044] By determining the center point information of the target shovel loader by means of three-dimensional modeling, the space occupied by the target shovel loader can be determined intuitively and three-dimensionally, so that the center point of the target shovel loader can be used as the real-time position of the target shovel loader, thereby improving the accuracy of determining the real-time position of the target shovel loader.
[0045] In some exemplary embodiments, the real-time location information of the target transportation equipment includes approach point information of the target transportation equipment and departure point information of the target transportation equipment.
[0046] Inputting the real-time sensing information into the loading path model for processing in step S102 to obtain the target loading path includes:
[0047] Determine the travel distance information of the target transport equipment based on the approach point information of the target transport equipment, the departure point information of the target transport equipment and the real-time position information of the target shovel-loading equipment; determine the travel direction information of the target transport equipment based on the approach direction of the target transport equipment, the preset standard direction and the departure point information of the target transport equipment; determine the position information of the target parking point based on the travel distance information and the travel direction information; determine the target loading path based on the position information of the target parking point, the approach point and departure point of the target transport equipment.
[0048] The entry point information indicates the location information of the entry point where the target transport equipment enters the material loading area; the departure point information indicates the location information of the exit point where the target transport equipment drives out of the material loading area. The target parking point is the point in the target parking area that is closest to the entry point and departure point of the target transport equipment.
[0049] By analyzing the real-time position information of the target shoveling equipment, it is determined that the target transport equipment needs to travel to the vicinity of the target shoveling equipment to receive the materials unloaded by the target shoveling equipment. The distance required to travel from the approach point of the target transport equipment to the vicinity of the target shoveling equipment and then to the departure point of the target transport equipment is the travel distance information of the target transport equipment.
[0050] In addition, during the movement of the target transport equipment, it is also necessary to determine the travel direction of the transport equipment, that is, through the approach direction of the target transport equipment and the preset standard direction (such as due north, or due south, etc.), the direction in which the target transport equipment enters the material loading area is clarified, and through the departure point information of the target transport equipment and the preset standard direction, the direction in which the target transport equipment leaves the material loading area is clarified, thereby determining the travel direction information of the target transport equipment.
[0051] The travel distance information and travel direction information determined in the above manner can accurately determine the position information of the target parking point, that is, the specific position information where the target transport equipment should be parked around the target shoveling equipment; then, based on the position information of the target parking point, the approach point and the departure point of the target transport equipment, the target loading path is determined, so that the target transport equipment can travel to the target parking point according to the target loading path, accurately receive the surroundings of the target shoveling equipment, thereby obtaining the unloaded materials, and transporting the materials out, thereby improving the transportation efficiency of the materials.
[0052] In some exemplary embodiments, the traveling direction information includes the moving direction of the target transport equipment relative to the target parking point; determining the traveling direction information of the target transport equipment according to the approach direction of the target transport equipment, the preset standard direction and the departure point information of the target transport equipment may be implemented in the following manner:
[0053] Determine the direction of travel of the target transport equipment relative to the preset parking point as its approach direction, and determine a first angle between the approach direction of the target transport equipment and the preset standard direction; determine a second angle between a line between the preset parking point and the departure point of the target transport equipment and the preset standard direction; and when it is determined that the difference between the first angle and the second angle is less than or equal to a preset angle threshold, determine the preset parking point as the target parking point.
[0054] The preset parking point is a parking point in the target parking area; the movement direction of the target transport equipment relative to the preset parking point is the movement direction of the target transport equipment relative to the target parking point.
[0055] By selecting any parking point in the target parking area as the preset parking point, and determining the first angle based on the relative direction between the preset parking point and the target transport equipment, and a preset standard direction (such as due north, due south, etc.), the direction in which the target transport equipment enters the material loading area can be clarified; then, a line is connected between the preset parking point and the departure point of the target transport equipment, and the angle between the connection line and the preset standard direction is used as the second angle to clarify the direction in which the target transport equipment leaves the material loading area.
[0056] When the difference between the first angle and the second angle is less than or equal to the preset angle threshold, it indicates that the moving direction of the target transportation equipment is accurate, and it can be determined that the preset parking point is the target parking point.
[0057] For example, if the preset standard direction is set to the north direction, the first angle is set to θ′, which means the angle between the line between point A and point M and the north direction; the second angle is μ, which means the angle between the line between point M and point B and the north direction;
[0058] Then the difference between the first angle and the second angle can be determined as θ = lim θ′→μ |θ′-μ|, if θ≤ε, it is determined that the transportation direction of the target transportation equipment is correct.
[0059] Wherein, ε represents the preset angle threshold.
[0060] Through the above analysis of the first angle and the second angle, the transport direction of the target transport equipment can be adjusted in time so that the target transport equipment can accurately drive to the target parking point and leave the material loading area from the target parking point, thereby realizing accurate reception and transportation of materials.
[0061] In some exemplary embodiments, before determining the travel distance information of the target transport equipment based on the approach point information of the target transport equipment, the departure point information of the target transport equipment and the real-time position information of the target shoveling equipment, the method also includes: obtaining the mechanical arm length of the target shoveling equipment and the length of the target transport equipment; determining the initial parking area based on the length of the target transport equipment, the real-time position information of the target shoveling equipment and the length of its mechanical arm; matching the real-time position of the material to be loaded with the initial parking area to determine the target parking area corresponding to the target transport equipment.
[0062] The target parking area includes the location information of the target parking point. The initial parking area can be rectangular, square or circular, so as to facilitate the parking of the target transport equipment and accurately receive the materials unloaded by the target shovel loading equipment.
[0063] For example, if the real-time position information of the target shoveling equipment is taken as the center of the circle and the sum of the target shoveling equipment's own length and the length of its mechanical arm is taken as the radius, the delivery area for the materials unloaded by the target shoveling equipment can be delineated; then, taking the real-time position information of the target shoveling equipment as the center of the circle and the sum of the length of the target transport equipment, the target shoveling equipment's own length and the length of its mechanical arm as the radius, the area where the target transport equipment can be parked can be obtained, so that the target transport equipment can accurately receive the materials unloaded by the target shoveling equipment.
[0064] For another example, the initial parking area is a circular area, the dot of the circular area is the center point of the target shovel loading equipment, and the radius difference corresponding to the circular area is the length of the target transportation equipment.
[0065] Among them, the radius difference corresponding to the circular area is the difference between the first radius and the second radius. The first radius is determined according to the length of the target transportation equipment, the three-dimensional size of the target shoveling equipment and the length of its mechanical arm. The second radius is determined according to the three-dimensional size of the target shoveling equipment and the length of its mechanical arm.
[0066] Furthermore, the real-time position of the material to be loaded is matched with the initial parking area, and the target parking area corresponding to the target transportation equipment is determined, so that the target shoveling equipment can grab the material more quickly from the real-time position of the material to be loaded, and put the grabbed material into the target transportation equipment parked in the target parking area, thereby improving the loading efficiency of the material.
[0067] In some exemplary embodiments, after executing step S103 of sending the target loading path to the target transportation equipment, the method also includes: determining the presence of an intruding object in response to environmental perception information reported by a preset perception device; generating alarm information based on the environmental perception information; and sending the alarm information to the operation and maintenance terminal.
[0068] The warning information includes at least one of the following: warning time, warning type, warning level, road condition information, vehicle information, and traffic participant information. The above warning information is only an example, and can be specifically set according to actual needs. Other unspecified warning information is also within the scope of protection of this disclosure and will not be repeated here.
[0069] The environmental perception information is information obtained by the preset perception device from perceiving objects in the surrounding environment. For example, the environmental perception information includes information about intruding objects, which may include movement trajectories of pedestrians, movement trajectories of other working vehicles, and the like.
[0070] When it is determined that there is an intruding object around the preset sensing device, an alarm message is generated based on the environmental sensing information so that the relevant characteristics of the intruding object can be reflected through the alarm message. For example, whether the motion trajectory of the intruding object coincides with the motion trajectory of the target transport equipment (or, the target shovel loading equipment) to determine the road condition information; or, if the intruding object is other transport equipment related to the type of the target transport equipment (i.e., a follower vehicle), the follower vehicle information indicates the transport equipment that needs to load and transport materials after the current target transport equipment.
[0071] The traffic participant information includes at least one of the following: target transport equipment, other transport equipment related to the type of the target transport equipment, target shovel loading equipment, and pedestrians.
[0072] By generating alarm information based on environmental perception information and sending the alarm information to the operation and maintenance terminal, the operation and maintenance terminal can promptly feedback the situation of the intruding object to the operation and maintenance personnel, which is conducive to the operation and maintenance personnel to deal with the intruding object in time and ensure the normal loading and transportation of materials by the target transportation equipment.
[0073] In a second aspect, an embodiment of the present disclosure provides a material loading control device.
[0074] Figure 2 FIG. 1 is a block diagram showing a material loading control device provided by an embodiment of the present disclosure. Figure 2 As shown, the material loading control device 200 includes but is not limited to the following modules.
[0075] The acquisition module 201 is configured to acquire real-time perception information.
[0076] The real-time sensing information is information obtained by a preset sensing device sensing the real-time position of at least one of a target transport device, a target shoveling device, materials to be loaded and obstacles in the material loading area.
[0077] The processing module 202 is configured to input the real-time perception information into the loading path model for processing to obtain a target loading path.
[0078] Among them, the loading path model is a model obtained by training a neural network model based on multiple sample perception information.
[0079] The sending module 203 is configured to send the target loading path to the target transportation equipment.
[0080] The target loading path is used to guide the target transport equipment to load and transport the materials unloaded by the target shovel loading equipment.
[0081] It should be noted that the material loading control device 200 can implement any material loading control method disclosed in the present invention.
[0082] In this embodiment, the acquisition module uses a preset sensing device to sense the real-time position of at least one of the target transportation equipment, the target shoveling equipment, the materials to be loaded and the obstacles in the material loading area, and obtains real-time sensing information to facilitate clarifying the positions of each equipment, material and obstacle in the material loading area; then, the processing module is used to input the real-time sensing information into the loading path model, so that the loading path model can analyze the real-time position information of each object in the real-time sensing information, thereby obtaining an accurate target loading path that can guide the movement of the target transportation equipment, and the sending module is used to send the target loading path to the target transportation equipment. When the target transportation equipment receives the target loading path, it can load and transport the materials unloaded by the target shoveling equipment according to the target loading path, thereby realizing accurate loading and transportation of materials and improving the loading and transportation efficiency of materials.
[0083] It is worth mentioning that all modules involved in this embodiment are logic modules. In practical applications, a logic unit can be a physical unit, a part of a physical unit, or a combination of multiple physical units. In addition, in order to highlight the innovative part of the present disclosure, this embodiment does not introduce units that are not closely related to solving the technical problems proposed by the present disclosure, but this does not mean that there are no other units in this embodiment.
[0084] In a third aspect, an embodiment of the present disclosure provides a material loading control system.
[0085] Figure 3 The block diagram of a material loading control system provided by an embodiment of the present disclosure is shown. The material loading control system can be applied in scenarios such as mining areas or docks, and can also be applied in other scenarios where material loading or transportation is required, and the present disclosure does not limit this.
[0086] like Figure 3 As shown, the material loading control system includes but is not limited to the following equipment: target shoveling equipment 301 (such as an excavator, etc.), target transportation equipment 302 (such as an unmanned truck, etc.), preset sensing equipment 303 (such as an image acquisition device, or a laser sensing device, or a radar sensing device, etc.), and an edge cloud server 304.
[0087] The target shovel loading equipment 301 is used to load the materials in the material loading area onto the target transportation equipment 302 .
[0088] The target transport equipment 302 is used to obtain the material unloaded by the target shovel loading equipment 301 and transport the material from the material loading area to the material unloading area.
[0089] The preset sensing device 303 is used to sense the real-time positions of target transportation equipment, target shoveling equipment and various items in the material loading area to obtain real-time sensing information; and send the real-time sensing information to the edge cloud server 304.
[0090] In some embodiments, the preset sensing device 303 is a mobile trolley, which can adjust its position according to the change of the position of the loaded material. The trolley is equipped with a rod that can be automatically raised and lowered, and the maximum height of the rod can be set according to the range of the material loading area to be sensed, for example, the height of the rod is set to 4.5 meters to 7.5 meters.
[0091] Among them, a laser radar sensing device and three image acquisition devices are installed on the top of the rod of the car. The overlapping area of the scanning range of the laser radar sensing device and the coverage area of the image acquisition device is the target sensing area.
[0092] For example, Figure 4 FIG. 1 is a schematic diagram showing a sensing area provided by an embodiment of the present disclosure. Figure 4 As shown in the figure, the black solid circle indicates the location of the preset sensing device, the three triangle areas formed by the dotted lines from the black solid circle are the sensing areas of the image acquisition device, the upper semicircle with the black solid circle as the center indicates the sensing area of the laser radar, and the overlapping area between the sensing area of the image acquisition device and the sensing area of the laser radar is the target sensing area. In the target sensing area, the material loading area (i.e., Figure 4 The lower semicircle area with the target shovel loading equipment and the target transport equipment as the center is shown in ).
[0093] The edge cloud server 304 is used to input the real-time perception information into the loading path model for processing to obtain the target loading path; and send the target loading path to the target transportation equipment 302.
[0094] Among them, the loading path model is a model obtained by training the neural network model according to multiple sample perception information; the target loading path is used to guide the target transportation equipment 302 to load and transport the materials unloaded by the target shovel loading equipment 301, that is, the target loading path is a path for the target transportation equipment 302 to park, load materials, and then transport the loaded materials from the material loading area to the material unloading area.
[0095] In some embodiments, the material loading control system may also include an access gateway (not shown in the figure), a base station (not shown in the figure) and other equipment.
[0096] The access gateway is used to transmit the real-time perception information collected by the preset perception device 303 to the base station, and the base station forwards the real-time perception information to the edge cloud server 304.
[0097] Among them, a visualization and scheduling supervision platform, or an emergency remote takeover system runs on the edge cloud server 304.
[0098] The visualization and scheduling supervision platform is used to control each device in the material loading control system and / or display alarm information for remote monitoring by operation and maintenance personnel.
[0099] The emergency remote takeover system is used to remotely control the target transport equipment 302 when a control module of the target transport equipment 302 itself fails.
[0100] In some embodiments, the edge cloud server 304 includes an automatic positioning and calibration module, a first information collection module for constructing three-dimensional information of the target shoveling equipment 301, a second information collection module for collecting feature information of the target transportation equipment 302, a material pile detection module, a state information perception module of the target shoveling equipment, a ground spillage and pit detection module, an optimal area determination module, and an optimal path calculation module, etc.
[0101] In some embodiments, before using the above-mentioned material loading control system to control the loading of materials, the edge cloud server 304 also needs to train and obtain a loading path model.
[0102] Figure 5 FIG. 1 is a flow chart showing a material loading control method implemented by a material loading control system provided by an embodiment of the present disclosure. Figure 5 As shown, the material loading control method implemented by the system includes but is not limited to the following steps:
[0103] Step S501: the edge cloud server 304 obtains real-time perception information.
[0104] The real-time sensing information is information obtained by a preset sensing device sensing the real-time position of at least one of a target transport device, a target shoveling device, materials to be loaded and obstacles in the material loading area.
[0105] In some embodiments, the real-time sensing information is information obtained by the preset sensing device 303 sensing the real-time position of the target transportation equipment, the real-time position of the target shoveling equipment, the real-time position of the material to be loaded, and the real-time position of obstacles (such as other parked vehicles, scattered large pieces of material, etc.) in the material loading area.
[0106] For example, the preset sensing device 303 is fixedly set on a rod of a trolley in the material loading area. When it is determined that the position of the trolley is fixed, the automatic positioning module of the trolley is started to obtain the latitude and longitude information of the trolley's position and the height information of the preset sensing device 303 relative to the ground.
[0107] Since the rod on the trolley has an automatic lifting function, the height of the preset sensing device 303 on the rod relative to the ground can be set accordingly according to the range of the material loading area, so as to more accurately sense the equipment in the material loading area.
[0108] By establishing a three-dimensional coordinate system with the trolley where the preset sensing device 303 is located as the origin, the position information of each device in the material loading area relative to the preset sensing device 303 can be accurately measured, thereby facilitating the subsequent processing of the positions of multiple objects in the material loading area.
[0109] In some embodiments, the target shoveling equipment can be scanned by a preset sensing device 303 to determine the three-dimensional size of the target shoveling equipment (e.g., the length, width, and height of the shovel), and then a three-dimensional model of the target shoveling equipment is constructed based on the three-dimensional size to determine the center point information of the three-dimensional model.
[0110] The three-dimensional dimensions of the target shovel-loading equipment do not include the length of the mechanical arm of the target shovel-loading equipment.
[0111] For example, Figure 6 A schematic diagram of a target shovel loading device provided by an embodiment of the present disclosure is shown. Figure 6 As shown, the three-dimensional model of the target shoveling equipment is represented as a cube, and the center point of the cube can represent the real-time position information of the target shoveling equipment.
[0112] The length of the mechanical arm of the target shovel loading equipment is L1+L2; L1 represents the length of the movable arm of the target shovel loading equipment, and L2 represents the straight line length from the center point of the target shovel loading equipment to the end of its movable arm.
[0113] It should be noted that the position within a preset distance range (eg, 2.5 meters to 3.5 meters) to the left, right or rear of the target shovel loading equipment can be identified as the position closest to the target shovel loading equipment.
[0114] If the center point of the target shovel loader is taken as the circle point and the first circular area with R1 as the radius is the unloading range of the target shovel loader (that is, the target transport equipment needs to be parked outside the first circular area); the center point of the target shovel loader is taken as the circle point and the second circular area with R2 as the radius, then the annular area between the first circular area and the second circular area is the initial parking area.
[0115] Among them, R1 = the mechanical arm length of the target shovel loading equipment + the physical length of the target shovel loading equipment, and the physical length of the target shovel loading equipment is the longest distance between the center point of the target shovel loading equipment and an edge of the target shovel loading equipment. R2 = R1 + the length of the target transport equipment.
[0116] In some embodiments, a method for acquiring real-time location information of an obstacle includes:
[0117] Obtaining the flatness information of the ground through which the target transport equipment passes; when it is determined that the flatness information includes a pit, and the width of the pit is greater than a preset width threshold, and the depth of the pit is greater than a preset depth threshold, determining that an obstacle exists, thereby obtaining real-time position information of the obstacle;
[0118] And / or, when it is determined that the ground flatness information includes a protruding object and the three-dimensional size of the protruding object is outside a preset three-dimensional size range, it is determined that an obstacle exists, thereby obtaining real-time position information of the obstacle.
[0119] For example, if the preset sensing device 303 detects that there is a pit on the ground where the target transport equipment passes, and the width of the pit is greater than 50 centimeters (cm) and the depth of the pit is greater than 30 cm, it can be determined that there is an obstacle on the path where the target transport equipment passes.
[0120] For another example, if the preset sensing device 303 detects that there are protruding objects (for example, scattered large materials, or stones, etc.) on the ground through which the target transport equipment passes, and the three-dimensional dimensions (such as length, width and height) of the protruding object exceed 30cm*30cm*30cm, it can be determined that there are obstacles on the path through which the target transport equipment passes.
[0121] By detecting obstacle information in the above manner, the target transport equipment can avoid obstacles during its movement, thereby reducing the possibility of the target transport equipment rolling over obstacles.
[0122] In step S502 , the edge cloud server 304 determines a target parking area where the target transportation equipment 302 needs to be parked.
[0123] The target parking area is used to park the target transport equipment 302, so that the target transport equipment 302 can accurately receive the abbreviated materials of the target shovel loading equipment 301. The target parking area includes a target parking point.
[0124] For example, Figure 7 A schematic diagram of a target parking area provided by an embodiment of the present disclosure is shown. Figure 7 As shown in the figure, the center point of the target shovel loading equipment is taken as the circle point, and R1 and R2 are taken as the radius to obtain two circular areas. The circular area between the two circular areas is the initial parking area. By matching the initial parking area with the location where the materials are stacked, the target parking area can be obtained, that is, Figure 7 The partial ring area shown in .
[0125] When the target transport equipment is parked in the target parking area, it can accurately receive the materials unloaded by the target shovel loading equipment.
[0126] In step S503 , the edge cloud server 304 determines the travel distance information of the target transportation equipment 302 according to the approach point information of the target transportation equipment 302 , the departure point information of the target transportation equipment 302 , and the real-time position information of the target shovel loading equipment 301 .
[0127] The travel distance information of the target transport equipment 302 is the distance traveled by the target transport equipment 302 from the approach point, through the target parking point, to the departure point.
[0128] For example, Figure 8 FIG. 2 shows a schematic diagram of determining a target parking point provided by an embodiment of the present disclosure. Figure 8 As shown, in the coordinate system constructed with x as the horizontal coordinate and y as the vertical coordinate, the target shovel loading equipment is the point N, and its coordinates are (x0, y0); the coordinates of any entry point A are set to (x1, y1), and the coordinates of any exit point B are set to (x2, y2).
[0129] If the target parking point corresponding to the target transport equipment is set to M, its corresponding coordinates are (x, y); the distance between points N and M is Then we can get formula (1):
[0130] (x-x0) 2 +(y-y0) 2 =R 2 (1)
[0131] If formula (2) is set as:
[0132]
[0133] When f(x, y) takes the minimum value, it can be known that the corresponding route distance from the entry point A to the exit point B is the shortest and optimal; then, by substituting formula (1) into formula (2) and eliminating y, we can obtain the equation about x, and then solve the coordinates (x, y) of point M, that is, the coordinates (x, y) of point M are the points that minimize the value of f(x, y), thereby obtaining the coordinate value of the target parking point M.
[0134] Furthermore, based on the coordinates of the target parking point M, the coordinates of the approach point A, and the coordinates of the exit point B, the travel distance of the target transportation equipment 302 can be calculated.
[0135] For example, by using the straight-line distance calculation formula between two points to calculate the AM distance and the MB distance respectively, the travel distance of the target transport device 302 = AM + MB can be obtained.
[0136] In step S504 , the edge cloud server 304 determines the moving direction information of the target transportation equipment 302 according to the approach direction of the target transportation equipment 302 , the preset standard direction, and the departure point information of the target transportation equipment 302 .
[0137] by Figure 8 For example, when the target transport equipment moves from the entry point A to the preset parking point, and then drives to the exit point B via the preset parking point, the following method can be used to determine whether its travel direction is correct:
[0138] Determine that the direction of travel of the target transport equipment relative to a preset parking point is its approach direction (e.g., the direction of travel of the front of the target transport equipment relative to the preset parking point), and determine a first angle between the approach direction of the target transport equipment and a preset standard direction, the preset parking point being a parking point in the target parking area; determine a second angle between a line between the preset parking point and a departure point of the target transport equipment and the preset standard direction; and determine that the preset parking point is the target parking point when the difference between the first angle and the second angle is less than or equal to a preset angle threshold.
[0139] The moving direction of the target transport equipment relative to the preset parking point is the moving direction of the target transport equipment relative to the target parking point.
[0140] For example, if the preset standard direction is set to the north direction, the first angle is set to θ′, which means the angle between the line between point A and point M and the north direction; the second angle is μ, which means the angle between the line between point M and point B and the north direction;
[0141] Then the difference between the first angle and the second angle can be determined as θ = limθ′→μ |θ′-μ|, if θ≤ε, it is determined that the transportation direction of the target transportation equipment is correct.
[0142] Wherein, ε represents the preset angle threshold.
[0143] In step S505 , the edge cloud server 304 determines the location information of the target parking point according to the travel distance information and the travel direction information; and determines the target loading path according to the location information of the target parking point, the approach point and the departure point of the target transportation equipment 302 .
[0144] In some embodiments, the edge cloud server 304 is also used to train the neural network model according to the sample position information, sample material position information and sample obstacle information of multiple sample shoveling equipment to obtain the loading path model.
[0145] Among them, the loading path model is used to process the acquired real-time perception information to obtain the target loading path.
[0146] Step S506 : the edge cloud server 304 sends the target loading path to the target transportation equipment 302 .
[0147] Step S507 , the target transportation equipment 302 transports the loaded materials from the material loading area to the material unloading area according to the target loading path.
[0148] In some embodiments, the target transport device 302 may also monitor whether there are any intruding objects in its travel path while transporting materials.
[0149] For example, the preset sensing device will report the environmental sensing information it senses to the edge cloud server 304, and then the edge cloud server 304 will analyze the environmental sensing information to determine the presence of an intruding object; further, the edge cloud server 304 will generate alarm information based on the environmental sensing information; and send the alarm information to the operation and maintenance terminal.
[0150] The warning information includes at least one of the following: warning time, warning type, warning level, road condition information, vehicle information and traffic participant information.
[0151] For example, the intruding object may be a pedestrian, other working vehicles, etc. The following vehicle information indicates the transport equipment that needs to load and transport materials after the current target transport equipment 302 .
[0152] In some embodiments, the alarm information can also be determined by predicting and analyzing the motion trajectory of the target object. For example, if it is determined that the motion trajectory of a target object intersects with the target loading path of the target transportation equipment 302, an alarm message will also be generated to prompt the operation and maintenance personnel to adjust the travel path of the target transportation equipment 302 in a timely manner to reduce the proportion of accidents such as collisions.
[0153] In some embodiments, the edge cloud server 304 can be implemented using a mobile edge computing (MEC) server, wherein the edge cloud server 304 can be implemented using the fifth generation mobile communication technology (5G) when communicating with other devices.
[0154] In some embodiments, the edge cloud server 304 may also send relevant emergency handling strategies to the target transportation equipment 302, such as emergency parking, automatic departure and other decisions.
[0155] In this embodiment, a preset sensing device is used to sense the position of each object in the material loading area, and the real-time sensing information obtained is processed to quickly determine the target loading path, and the target loading path is sent to the target transportation equipment, so that the target transportation equipment can load and transport materials according to the target loading path, thereby achieving accurate loading and transportation of materials and improving the loading and transportation efficiency of materials.
[0156] In a fourth aspect, the embodiments of the present disclosure provide an electronic device, a computer-readable storage medium, and a computer program product, all of which can be used to implement the loading control method of any material in the embodiments of the present disclosure. The corresponding technical solutions and descriptions are referred to the corresponding records in the method part and will not be repeated here.
[0157] Fig. 9 A block diagram of a composition of an electronic device provided by an embodiment of the present disclosure is shown.
[0158] like Fig. 9 As shown, the electronic device includes: at least one processor 901; at least one memory 902, and one or more I / O interfaces 903, connected between the processor 901 and the memory 902; wherein the memory 902 stores one or more computer programs that can be executed by at least one processor 901, and the one or more computer programs are executed by at least one processor 901 so that at least one processor 901 can execute the above-mentioned material loading control method.
[0159] The present disclosure also provides a computer-readable storage medium on which a computer program is stored, wherein the computer program implements the above-mentioned material loading control method when executed by a processor / processing core. The computer-readable storage medium may be a volatile or non-volatile computer-readable storage medium.
[0160] The embodiments of the present disclosure also provide a computer program product, including a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above-mentioned material loading control method.
[0161] It will be appreciated by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In a hardware implementation, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or may be implemented as hardware, or may be implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable storage medium, which may include a computer storage medium (or a non-transitory medium) and a communication medium (or a temporary medium).
[0162] As is known to those of ordinary skill in the art, the term computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable program instructions, data structures, program modules or other data). Computer storage media include, but are not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), static random access memory (SRAM), flash memory or other memory technology, portable compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. In addition, it is known to those of ordinary skill in the art that communication media typically contain computer-readable program instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transmission mechanism, and may include any information delivery medium.
[0163] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.
[0164] The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be customized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.
[0165] The computer program product described herein may be implemented in hardware, software, or a combination thereof. In one optional embodiment, the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (SDK), etc.
[0166] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.
[0167] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.
[0168] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.
[0169] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of the module, program segment or instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of special hardware and computer instructions.
[0170] Example embodiments have been disclosed herein, and although specific terms are employed, they are used and should be interpreted only in a general illustrative sense and not for limiting purposes. In some instances, it will be apparent to those skilled in the art that, unless otherwise expressly noted, features, characteristics, and / or elements described in conjunction with a particular embodiment may be used alone or in combination with features, characteristics, and / or elements described in conjunction with other embodiments. Therefore, those skilled in the art will appreciate that various changes in form and detail may be made without departing from the scope of the present disclosure.
Claims
1. A material loading control method, characterized in that: The method comprises: Acquiring real-time sensing information, wherein the real-time sensing information is information obtained by a preset sensing device sensing the real-time position of at least one of a target transport device, a target shovel loading device, materials to be loaded, and obstacles in a material loading area; Inputting the real-time sensing information into a loading path model for processing to obtain a target loading path; The target loading path is sent to the target transport equipment, where the target loading path is used to guide the target transport equipment to load and transport the materials unloaded by the target shovel loading equipment.
2. The method according to claim 1, characterized in that Before inputting the real-time perception information into the loading path model for processing to obtain the target loading path, the method further includes: Acquire sample sensing information, wherein the sample sensing information includes sample position information of multiple sample shoveling devices, sample material position information, and sample obstacle information; According to the sample position information of the plurality of sample shoveling equipment, the sample material position information and the sample obstacle information, a neural network model is trained to obtain the loading path model.
3. The method according to claim 1, characterized in that The real-time location information of the target transport equipment includes the approach point information of the target transport equipment and the departure point information of the target transport equipment; Inputting the real-time perception information into a loading path model for processing to obtain a target loading path includes: Determine the travel distance information of the target transport equipment according to the approach point information of the target transport equipment, the departure point information of the target transport equipment and the real-time position information of the target shovel loading equipment; Determine the moving direction information of the target transportation equipment according to the approach direction of the target transportation equipment, the preset standard direction and the departure point information of the target transportation equipment; Determining the location information of the target parking point according to the travel distance information and the travel direction information; The target loading path is determined according to the location information of the target parking point, the approach point and the departure point of the target transportation equipment.
4. The method according to claim 3, characterized in that The travel direction information includes the movement direction of the target transport equipment relative to the target parking point; The determining the moving direction information of the target transport equipment according to the approach direction of the target transport equipment, the preset standard direction and the departure point information of the target transport equipment includes: Determine that the direction of travel of the target transport equipment relative to a preset parking point is its approach direction, and determine a first angle between the approach direction of the target transport equipment and the preset standard direction, wherein the preset parking point is a parking point in the target parking area; Determining a second angle between a line connecting the preset parking point and the departure point of the target transport equipment and the preset standard direction; In the case where it is determined that the difference between the first angle and the second angle is less than or equal to a preset angle threshold, determining the preset parking point as the target parking point; The moving direction of the target transport equipment relative to the preset parking point is the moving direction of the target transport equipment relative to the target parking point.
5. The method according to claim 3 or 4, characterized in that: Before determining the travel distance information of the target transport equipment according to the approach point information of the target transport equipment, the departure point information of the target transport equipment and the real-time position information of the target shovel loading equipment, the method further includes: Obtaining the length of the mechanical arm of the target shovel loading equipment and the length of the target transportation equipment; Determine an initial parking area according to the length of the target transport equipment, the real-time position information of the target shovel loading equipment and the length of its mechanical arm; The real-time position of the material to be loaded is matched with the initial parking area to determine the target parking area corresponding to the target transportation equipment, wherein the target parking area includes the position information of the target parking point.
6. The method according to claim 1, characterized in that After sending the target loading path to the target transport equipment, the method further includes: In response to the environmental perception information reported by the preset perception device, determining that there is an intruding object; generating warning information according to the environmental perception information; The alarm information is sent to the operation and maintenance terminal; wherein the alarm information includes at least one of the following: alarm time, alarm type, alarm level, road condition information, vehicle information and traffic participant information.
7. A material loading control device, characterized in that: include: An acquisition module, used to acquire real-time perception information, wherein the real-time perception information is information obtained by a preset perception device sensing the real-time position of at least one of a target transport device, a target shovel loading device, materials to be loaded, and obstacles in a material loading area; A processing module, used for inputting the real-time perception information into a loading path model for processing to obtain a target loading path; The sending module is used to send the target loading path to the target transportation equipment, and the target loading path is used to guide the target transportation equipment to load and transport the materials unloaded by the target shovel loading equipment.
8. An electronic device comprising a memory and a processor; the memory stores a computer program that can be executed by the processor, and when the computer program is executed by the processor, the material loading control method as described in any one of claims 1 to 6 is implemented.
9. A computer-readable medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the material loading control method according to any one of claims 1 to 6.
10. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the material loading control method according to any one of claims 1 to 6 is implemented.