Method, device and electronic equipment for calculating excavation volume of excavator

Abstract

The present application discloses a method, device, electronic device, and computer-readable storage medium for calculating the amount of earth excavated by an excavator. The method comprises: obtaining ground point cloud information of the excavator's operating scene and establishing an elevation map; obtaining position information of each point on the excavator's bucket tooth tip at a first excavation moment; calculating a first excavation depth of each point on the bucket tooth tip at the first excavation moment based on the position information and the elevation map; and calculating the amount of earth excavated by the excavator at the first excavation moment based on the first excavation depth of each point on the bucket tooth tip at the first excavation moment. This method calculates the amount of earth excavated by the excavator at each excavation moment, resolving the problem of existing methods of measuring earth excavation volume, which are unable to measure the amount of earth excavated by the excavator in real time during the excavation process, resulting in low accuracy in excavator operations.

Description

Technical Field

[0001] The present application relates to the technical field of excavators, and in particular to a method, device, electronic device, and computer-readable storage medium for calculating the amount of earth excavated by an excavator. Background Art

[0002] Excavators are frequently used in engineering and infrastructure projects. With the advancement of computer technology, improving the precision of excavator operations has become a pressing issue. Accurately measuring excavation volume is a crucial factor in improving excavator accuracy and efficiency.

[0003] Currently, there are various methods for measuring excavation volume (e.g., manual measurement, instrument measurement, and vision-based excavation volume estimation). However, these methods primarily measure the volume of soil excavated by the excavator bucket and cannot measure the actual volume of soil excavated by the bucket during the excavation process. This results in the excavator operator being unable to make real-time decisions on further excavation progress (e.g., determining whether to continue digging deeper or to end the excavation), which in turn affects the excavator's operating accuracy and may even lead to excavation failure.

[0004] Therefore, how to measure the amount of soil excavated by the excavator in real time during the excavation process, thereby improving the operating accuracy of the excavator, has become a technical problem that needs to be solved urgently. Summary of the Invention

[0005] The present application provides a method, device, electronic device and computer-readable storage medium for calculating the excavation volume of an excavator to solve the problem that the existing excavation volume measurement method cannot measure the excavation volume of the excavator in real time during the excavation process, thereby resulting in low excavator operation accuracy.

[0006] An embodiment of the present application provides a method for calculating the amount of earth removed by an excavator, the method comprising:

[0007] Acquire ground point cloud information of the excavator operation scene, and establish an elevation map based on the ground point cloud information, wherein the elevation map uses a horizontal plane of a first coordinate system as a reference plane, and the first coordinate system is a coordinate system with the base of the excavator's boom as the origin;

[0008] Acquire position information of each point on the bucket tooth tip of the excavator at a first excavation moment, where the position information is the coordinates of each point on the bucket tooth tip in the first coordinate system. The first excavation moment is any excavation moment within a process in which the excavator completes an excavation operation.

[0009] Calculating a first excavation depth of each point on the bucket tooth tip at the first excavation moment based on the position information and the elevation map;

[0010] The amount of earth excavated by the excavator at the first excavation moment is calculated according to the first excavation depth of each point of the bucket tooth tip at the first excavation moment.

[0011] The embodiment of the present application further provides a device for calculating the amount of earth excavated by an excavator, the device comprising: an elevation map construction unit, a position information acquisition unit, an excavation depth calculation unit, and an amount of earth excavated calculation unit;

[0012] The elevation map construction unit is configured to obtain ground point cloud information of the excavator operation scene and to establish an elevation map based on the ground point cloud information, wherein the elevation map uses a horizontal plane of a first coordinate system as a reference plane, wherein the first coordinate system is a coordinate system with the base of the excavator's boom as its origin;

[0013] The position information acquisition unit is configured to acquire position information of each point on the bucket tooth tip of the excavator at a first excavation moment, wherein the position information is the coordinates of each point on the bucket tooth tip in the first coordinate system, and the first excavation moment is any excavation moment within a process in which the excavator completes an excavation operation;

[0014] The excavation depth calculation unit is configured to calculate a first excavation depth of each point of the bucket tooth tip at the first excavation moment based on the position information and the elevation map;

[0015] The excavation amount calculation unit is used to calculate the excavation amount of the excavator at the first excavation moment according to the first excavation depth of each point of the bucket tooth tip at the first excavation moment.

[0016] The embodiment of the present application further provides an excavator, the excavator being configured with a vehicle-mounted device, a distance measuring device, and a plurality of sensors;

[0017] The vehicle-mounted device includes: a first acquisition component, an elevation map construction component, a second acquisition component, a first calculation component, and a second calculation component;

[0018] The first acquisition component is used to acquire ground point cloud information of the excavator operation scene;

[0019] The elevation map building component is used to build an elevation map based on the ground point cloud information, wherein the elevation map uses the horizontal plane of a first coordinate system as a reference plane, and the first coordinate system is a coordinate system with the base of the excavator's boom as the origin;

[0020] The second acquisition component is configured to acquire position information of each point on the bucket tooth tip of the excavator at a first excavation moment, wherein the position information is the coordinates of each point on the bucket tooth tip in the first coordinate system, and the first excavation moment is any excavation moment within a process in which the excavator completes an excavation operation;

[0021] The first calculation component is configured to calculate a first excavation depth of each point of the bucket tooth tip at the first excavation moment based on the position information and the elevation map;

[0022] The second calculation component is used to calculate the amount of soil excavated by the excavator at the first excavation moment based on the first excavation depth of each point of the bucket tooth tip at the first excavation moment;

[0023] The distance measuring device is used to collect ground point cloud information of the excavator operation scene and transmit the ground point cloud information to the first acquisition component;

[0024] The multiple sensors are used to collect the posture information of each joint of the excavator at the first excavation moment, and transmit the posture information to the second acquisition component, so that the second acquisition component obtains the position information of each point of the bucket tooth tip of the excavator at the first excavation moment based on the posture information.

[0025] An embodiment of the present application further provides an electronic device, comprising: a memory and a processor;

[0026] The memory is used to store one or more computer instructions;

[0027] The processor is configured to execute the one or more computer instructions to implement the above method.

[0028] An embodiment of the present application also provides a computer-readable storage medium on which one or more computer instructions are stored, and the instructions are executed by a processor to implement the above method.

[0029] Compared to the prior art, the excavation volume calculation method provided in this application establishes an elevation map based on the horizontal plane of a first coordinate system as a reference plane, and obtains the position information of each point on the excavator's bucket tooth tip in the first coordinate system at the first excavation moment, thereby calculating the first excavation depth of each point on the bucket tooth tip at the first excavation moment. Furthermore, the excavation volume of the excavator at the first excavation moment is calculated based on the first excavation depth of each point on the bucket tooth tip. This method is based on converting the elevation map and the coordinates of each point on the bucket tooth tip at a certain excavation moment into the same coordinate system, thereby obtaining the excavation depth of each point on the bucket tooth tip at that excavation moment, and further obtaining the amount of soil that the bucket can excavate at that excavation moment based on the excavation depth. This method realizes real-time calculation of the excavation volume of the excavator at each excavation moment, solving the problem of existing excavation volume measurement methods that cannot measure the excavation volume of the excavator in real time during the excavation process, resulting in low excavator operation accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1This is an application system diagram of a method for calculating the amount of earth excavated by an excavator provided in an embodiment of the present application;

[0031] Figure 2 This is a flow chart of a method for calculating the amount of earth excavated by an excavator provided in the first embodiment of the present application;

[0032] Figure 3 This is a schematic diagram of a method for discretizing points in an elevation map provided by the first embodiment of the present application;

[0033] Figure 4 This is a schematic diagram of a method for obtaining position information of each point on the bucket tooth tip provided in the first embodiment of the present application;

[0034] Figure 5 This is a schematic diagram of a method for calculating the digging depth of each point on the bucket tooth tip provided in the first embodiment of the present application;

[0035] Figure 6 This is a diagram showing the operation of an excavator provided in the first embodiment of the present application;

[0036] Figure 7 This is a flow chart for calculating the real-time excavation volume of an excavator provided by the first embodiment of the present application;

[0037] Figure 8 2 is a schematic structural diagram of an excavator excavation volume calculation device provided in a second embodiment of the present application;

[0038] Figure 9 is a structural schematic diagram of an excavator provided in a third embodiment of the present application;

[0039] Figure 10 It is a structural diagram of an electronic device provided in the fourth embodiment of the present application. DETAILED DESCRIPTION

[0040] The following description sets forth many specific details to facilitate a thorough understanding of the present application. However, the present application can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the scope of the present application. Therefore, the present application is not limited to the specific implementations disclosed below.

[0041] The professional terms involved in the embodiments of this application are explained below to facilitate understanding.

[0042] A point cloud is a collection of points, each of which is a three-dimensional point. It contains information such as each point's location (i.e., its three-dimensional X, Y, and Z coordinates), as well as color, light intensity, category label, normal vector, and grayscale value. Point clouds can atomize the real world, allowing us to restore it through high-precision point cloud data.

[0043] Point cloud information, also known as point cloud data, refers to a set of vectors in a three-dimensional coordinate system. These vectors are usually expressed in the form of (X, Y, Z) three-dimensional coordinates and are generally used to represent the surface shape of an object. In addition to the geometric position information represented by (X, Y, Z), point cloud data can also represent the color information, grayscale value information, etc. of a point. If P is used i ={X i , Y i , Z i , ...} represents a point in the point cloud, then Point Cloud = {P1, P2, P3, ..., P i} can represent a set of point cloud data.

[0044] Elevation refers to the distance from a point to a reference plane along the plumb line.

[0045] An elevation map refers to a drawing used to represent the elevation of a certain area. In this embodiment, it is a map used to represent the ground height of the excavator operation scene.

[0046] A transformation matrix is ​​a concept in linear algebra, where linear transformations can be represented by matrices. If T is a linear transformation that maps Rn to Rm, and x is a column vector with n elements, then the m×n matrix A is called the transformation matrix of T.

[0047] A coordinate system is a reference that describes the position and posture of an object. It is also called a reference frame or frame of reference. To facilitate dot and cross multiplications, an n-dimensional coordinate system is often constructed from n orthogonal unit vectors. The description of an object in a coordinate system is called a coordinate.

[0048] Excavators are essential machinery for engineering and infrastructure projects, playing an irreplaceable role in construction, transportation, and mining. Advances in computer technology and excavator technology have significantly improved the precision of excavator operations. However, further improving this precision remains a long-term challenge in the field of excavator technology.

[0049] The precise operation of an excavator involves many factors, such as the control accuracy of the excavator and the accuracy of the control equipment. Among them, the precise measurement of the excavation volume is an important factor in improving the operating accuracy and efficiency of the excavator.

[0050] Currently, there are various methods for measuring excavation volume, including manual measurement, instrumental measurement, and vision-based estimation. Manual measurement is time-consuming and labor-intensive, with low accuracy. Instrumental measurement is costly and still requires manual operation. Vision-based estimation suffers from poor robustness and cannot handle visual occlusion. Furthermore, these methods primarily measure the volume of soil removed by the excavator bucket and cannot measure the actual volume removed during the excavation process. Specifically, they can only measure the volume removed by the bucket after the excavator completes a single excavation, and cannot measure the volume removed at each excavation moment in real time. While measuring the volume removed can help the excavator operator determine whether further excavation is necessary, it cannot determine whether to continue digging or terminate the excavation after a certain excavation moment. Consequently, existing methods for measuring excavation volume prevent the operator from making real-time decisions about the excavation progress during the excavation process, impacting the accuracy of the excavation operation and even leading to excavation failure.

[0051] In light of this, the present application provides a method for calculating the amount of earth removed. This method establishes an elevation map based on the horizontal plane of a first coordinate system and obtains the position information of each point on the excavator's bucket tooth tip in the first coordinate system at a first excavation moment. This method then calculates the first excavation depth of each point at the bucket tooth tip at the first excavation moment. Furthermore, the sub-excavation amount of the excavator at the first excavation moment is calculated based on the first excavation depth of each point at the bucket tooth tip. This method enables real-time calculation of the excavator's excavation amount at each excavation moment, enabling the excavator operator to determine the next excavation task based on the excavation amount at each excavation moment, further improving the accuracy of excavator operations.

[0052] The following is a further detailed description of the excavation volume calculation method, device, electronic device, and computer-readable storage medium for an excavator described in this application in conjunction with specific embodiments and drawings.

[0053] Figure 1 This is an application system diagram of a method for calculating the amount of earth excavated by an excavator provided in an embodiment of the present application.

[0054] like Figure 1As shown, the application system includes: an excavator end 101 and a server end 102. The excavator end 101 and the server end 102 are electrically connected or network connected. The excavator end 101 is a terminal that controls the excavator to perform work content in the work scene. It can be a control module configured on the excavator, or it can be a remote control module of the excavator. The server end 102 can be a vehicle-mounted device configured on the excavator, or it can be a terminal device independent of the excavator. The server end 102 is deployed with the excavation volume calculation method provided in this application, which can calculate the excavation volume of the excavator at each excavation moment based on the working conditions of the excavator at each excavation moment. Of course, the server end 102 can also be a server, providing real-time excavation volume calculation for multiple excavators at the same time to assist the excavator operator in deciding the work content of the excavator.

[0055] The first embodiment of the present application provides a method for calculating the excavation volume of an excavator, by which the excavation volume of the excavator at each excavation moment can be calculated in real time.

[0056] Figure 2 This is a flow chart of the method for calculating the amount of earth excavated by the excavator provided in this embodiment. Figure 2 The method for calculating the amount of earth excavated provided by this embodiment is described in detail. The embodiments described below are used to explain the technical solution of this application and are not intended to be limiting in actual use.

[0057] like Figure 2 As shown, the method for calculating the amount of excavation provided by this embodiment includes the following steps S201 to S204.

[0058] Step S201: obtain ground point cloud information of the excavator operation scene, and establish an elevation map based on the ground point cloud information. The elevation map uses the horizontal plane of the first coordinate system as the reference plane. The first coordinate system is a coordinate system with the root of the excavator's boom as the origin.

[0059] The ground point cloud information refers to the point cloud information corresponding to the ground of the excavator operation scene. Since the point cloud information includes not only the position information of each point on the ground, but also the depth information of each point on the ground, the elevation map constructed by the point cloud information can more accurately display the real situation of the ground.

[0060] In an optional implementation provided in this embodiment, a ranging device is installed on the excavator, specifically on the top of the cabin of the excavator, and the ranging device is used to collect ground point cloud information of the excavator's operating scene.

[0061] The distance measuring device is a device that projects light onto the ground to collect the three-dimensional coordinates of each point on the ground. Therefore, the ground point cloud information collected by the distance measuring device is the coordinates of each point on the ground of the excavator operation scene in a second coordinate system with the distance measuring device as the origin.

[0062] This embodiment provides an optional ranging device, namely, a laser ranging radar. The laser ranging radar is an electronic measuring instrument used in the field of surveying and mapping science and technology, which can detect a large range of space and obtain the three-dimensional coordinates of any point in the space. The laser ranging radar has the advantages of long measurement distance (maximum measurement distance is 60m), large measurement angle (360° in the horizontal direction, ±145° in the vertical direction), fast data acquisition speed (maximum data acquisition speed 3000 points / second), etc., which meets the information acquisition needs of the excavator operation scene. Of course, other ranging devices such as depth cameras can be used as ranging devices installed on the excavator.

[0063] After the ranging device collects the ground point cloud information of the work scene, it will transmit the ground point cloud information to the server. The server will create an elevation map of the ground of the work scene based on the ground point cloud information. Steps S201-1 to S201-3 are an optional implementation method for creating an elevation map provided by this embodiment, specifically:

[0064] Step S201-1, transform the ground point cloud information based on the first transformation matrix to obtain second ground point cloud information, where the second ground point cloud information is the coordinates of each point on the ground in the first coordinate system, and the first transformation matrix represents the transformation relationship from the second coordinate system to the first coordinate system.

[0065] The ground point cloud information collected by the ranging equipment is the coordinates of each ground point in the second coordinate system, and the elevation map that needs to be established uses the horizontal plane of the first coordinate system as the reference plane. Therefore, the first step in establishing the elevation map is to convert the coordinate system of the ground point cloud information.

[0066] In this embodiment, the ground point cloud information can be transformed from the second coordinate system to the first coordinate system through the first transformation matrix, so that the ground point cloud information is transformed into the second ground point cloud information. This transformation only involves the transformation of the reference coordinate system and does not change the dimension of the point cloud information.

[0067] The first transformation matrix is ​​the transformation matrix between the second coordinate system and the first coordinate system. Since the origin of the first coordinate system is the base of the excavator's boom, that is, the connection between the excavator's boom and the cabin, the origin of the second coordinate system is the ranging device, and the positions of the base of the boom and the ranging device in the world coordinate system are fixed, the first transformation matrix can be obtained through the visual calibration method. It should be noted that as long as the installation position of the ranging device is not moved, the first transformation matrix between the first coordinate system and the second coordinate system is fixed. Therefore, it is only necessary to perform a visual calibration after the ranging device is installed.

[0068] Step S201-2: Create the elevation map based on the second ground point cloud information.

[0069] As shown above, the second ground point cloud information is the coordinates of each point on the ground in the first coordinate system, which can be expressed as Point Cloud = {P1, P2, P3, ..., P i} vector set, where P i ={X i , Y i , Z i} represents the three-dimensional coordinates of a point.

[0070] After the server obtains the second ground point cloud information, it can use point cloud rendering technology to create an elevation map. For example, the server can use WebGL to render the obtained second ground point cloud information into a corresponding elevation map.

[0071] Normally, the image or model rendered from the point cloud information is a point cloud image or point cloud model composed of a number of points. In this embodiment, in order to facilitate the subsequent calculation of the amount of excavation, it is necessary to grid the point elevation map initially rendered from the second ground point cloud information. An optional implementation method is to discretize the points in the point elevation map to obtain a mesh elevation map composed of a plurality of squares of the same size, wherein the side length of the square is the resolution of the elevation map. Based on the coordinate information of each point in the point elevation map, the coordinate information of each square in the mesh elevation map can be obtained. Since the second ground point cloud information is the three-dimensional coordinates of each ground point in the first coordinate system, the coordinate information of the square is also the three-dimensional coordinates in the first coordinate system.

[0072] Figure 3 This is a schematic diagram of a method for discretizing points in an elevation map provided by this embodiment.

[0073] like Figure 3As shown in (a), after the server obtains the second ground point cloud information, it can use point cloud rendering technology to obtain a point elevation map 301. The point elevation map 301 is composed of a number of points, point 302 is one of them, and each point in the point elevation map 301 has its three-dimensional coordinates in the first coordinate system. Figure 3 As shown in (b), a bounding box 303 is established on the point elevation map 301, and the bounding box 303 is discretized into a number of squares of the same size according to a preset side length (e.g., 0.1 meters). Each square contains multiple points, and square 304 is one of them. Figure 3 As shown in (c), the point closest to the center of each square is taken from the multiple points contained in the square, and the coordinate information of the point is used as the coordinate information of the corresponding square. For example, in square 304, point 305 is the point closest to the center of square 304, then the coordinate information of point 305 is used as the coordinate information of square 304.

[0074] In actual applications, since the ranging device is installed on the top of the excavator cabin, when the ranging device collects ground point cloud information, it is inevitable that light will be projected onto parts of the excavator arm, and information is also collected on parts of the excavator arm, and the point cloud information corresponding to the arm is also used as part of the ground point cloud information. In order to avoid mistaking the excavator arm for part of the ground, in an optional implementation method provided in this embodiment, before the step of establishing an elevation map based on the second ground point cloud information, it is necessary to delete the point cloud information corresponding to the excavator arm from the second ground point cloud information. Specifically, the position information of the excavator arm can be obtained through the sensors installed on each joint of the excavator arm, and according to the arm position information, it can be known which of the point cloud information in the second ground point cloud information is the point cloud information corresponding to the arm, so that the point cloud information corresponding to the arm is deleted from the second ground point cloud information.

[0075] To sum up, the specific methods for establishing elevation maps can be:

[0076] First, ground point cloud information of the excavator operation scene is collected by a ranging device, where the ground point cloud information is the coordinates of each point on the ground of the excavator operation scene in a second coordinate system with the ranging device as the origin.

[0077] Second, based on the first transformation matrix, the ground point cloud information is converted into second ground point cloud information, where the second ground point cloud information is the coordinates of each point on the ground in the first coordinate system.

[0078] Third, the point cloud information corresponding to the excavator arm is deleted from the second ground point cloud information.

[0079] Fourth, a point elevation map is established based on the second ground point cloud information after deleting the point cloud information corresponding to the excavator arm.

[0080] Fifth, the points in the point-like elevation map are discretized to obtain an elevation map consisting of a plurality of squares of the same size, where the side length of the square is the resolution of the elevation map.

[0081] It should be noted that an excavator usually needs to perform multiple excavations during the execution of an excavation task. In this embodiment, one excavation is corresponded to an excavation cycle. When the excavator performs an excavation task, it is necessary to re-establish the corresponding elevation map for each excavation cycle. This is mainly because the excavator has already excavated the ground in the previous excavation cycle, which has changed the ground conditions of the working scene.

[0082] Step S202 : acquiring position information of each point on the bucket tooth tip of the excavator at the first excavation moment, wherein the position information is the coordinates of each point on the bucket tooth tip in the first coordinate system.

[0083] As mentioned above, the excavation cycle is the time period corresponding to an excavation operation during the excavation task performed by the excavator. In this embodiment, in order to monitor the excavation process of the excavator in real time, an excavation cycle is further divided into multiple excavation moments, so as to accurately calculate the excavation amount corresponding to each excavation moment.

[0084] The first excavation moment is any excavation moment in the process of the excavator completing an excavation operation.

[0085] Since the ground excavated by the excavator cannot be an absolute plane, and the bucket tooth tip has a certain width, in the subsequent calculation of the excavation volume, no matter which position of the bucket tooth tip represents the entire bucket tooth tip, errors will occur due to the different contact conditions between each position on the bucket tooth tip and the ground. Therefore, in this embodiment, the bucket tooth tip is dotted, and a series of evenly distributed discrete digging points are used to represent the bucket tooth tip. The digging volume corresponding to each digging point is accumulated to obtain the bucket's digging volume, that is, the excavator's digging volume. Normally, the spacing between adjacent points on the bucket tooth tip should be less than the side length of the grid in the elevation map, that is, the spacing between adjacent points on the bucket tooth tip is less than the resolution of the elevation map. This ensures that each grid on the elevation map corresponds to a point on the bucket tooth tip, which can avoid missing calculations of the digging volume.

[0086] In an optional implementation provided in this embodiment, a plurality of sensors are installed on the excavator. Specifically, four sensors are installed at the connection between the excavator's cabin and the tracks, the connection between the boom and the cabin, the connection between the forearm and the boom, and the connection between the bucket and the forearm. The sensors are used to obtain the posture changes of each joint of the excavator at each excavation moment. The sensors will read the inclination data according to the posture changes of each joint of the excavator.

[0087] Steps S202-1 to S202-6 are an optional implementation method for obtaining the position information of each point of the bucket tooth tip provided in this embodiment. Figure 4 This is a schematic diagram of the method for obtaining the position information of each point of the bucket tooth tip provided by this embodiment. Figure 4 The method for obtaining the position information of each point of the bucket tooth tip provided in this embodiment is described in detail.

[0088] Step S202-1, based on the readings of the multiple sensors at the first excavation moment, calculate the boom joint angle of the excavator, the forearm joint angle of the excavator, and the bucket joint angle of the excavator at the first excavation moment.

[0089] like Figure 4 As shown in the figure, the downward movement of the boom, arm, and bucket is a counterclockwise rotation around the y-axis, and the joint angles of the boom, arm, and bucket gradually increase. The upward movement of the boom, arm, and bucket is a clockwise rotation around the y-axis, and the joint angles of the boom, arm, and bucket gradually decrease.

[0090] Sensor T1 is installed at the connection O between the boom and the cabin, sensor T2 is installed at the connection A between the forearm and the boom, sensor T3 is installed at the connection B between the bucket and the forearm, and sensor T4 is installed at K in the cabin. T1, T2, T3, and T4 represent the coordinate directions of sensor T1, sensor T2, sensor T3, and sensor T4 at the first excavation moment, respectively.

[0091] Assume that at the first excavation moment, the readings of sensor T1, sensor T2, sensor T3, and sensor T4 are θ1, θ2, θ3, and θ4 respectively, with the upward direction relative to the horizontal plane being positive and the downward direction being negative. The angle of the upper arm joint is is α, the angle of the forearm joint is β, bucket joint angle =γ. Then, we can establish the angle conversion equation between each joint angle and the sensor reading as follows:

[0092]

[0093]

[0094]

[0095] Since sensor T1 is rigidly installed at the connection between the boom and the cabin, sensor T2 is rigidly installed at the connection between the forearm and the boom, and sensor T3 is rigidly installed at the connection between the bucket and the forearm, the angle horn horn All are fixed values. Set the parameters k, m, and n of the angle conversion equation to: The above angle conversion equation can be expressed as:

[0096]

[0097] The above angle conversion equation can be used to calculate the arm joint angle α, the arm joint angle β, and the bucket joint angle γ based on the readings θ1, θ2, θ3, and θ4 of sensors T1, T2, T3, and T4 at any excavation time.

[0098] Step S202-2: determining a second transformation matrix based on the upper arm joint angle, wherein the second transformation matrix represents a transformation relationship from the first coordinate system to a third coordinate system with the root of the forearm as the origin.

[0099] The arm joint angle represents the arm's posture information at the first digging moment, such as Figure 4 As shown in the figure, line segment OA is the straight rod between the equivalent rotating joints of the upper arm. The DH (Denavit-Hartenberg) parameter model of the robot forward kinematics can be used to determine the second transformation matrix from the upper arm coordinate system (i.e., the first coordinate system with the upper arm root as the origin) to the lower arm coordinate system (i.e., the third coordinate system with the lower arm root as the origin) according to the upper arm joint angle, and R OA Represents the second transformation matrix, then the second transformation matrix can be expressed as:

[0100]

[0101] Step S202-3: determining a third transformation matrix according to the forearm joint angle, wherein the third transformation matrix represents a transformation relationship from the third coordinate system to a fourth coordinate system with the bucket root as the origin.

[0102] The forearm joint angle represents the posture information of the forearm at the first digging moment, such as Figure 4 As shown in the figure, line segment AB is the straight rod between the equivalent rotating joints of the forearm. The DH (Denavit-Hartenberg) parameter model of the robot forward kinematics can be used to determine the third transformation matrix from the forearm coordinate system (i.e., the third coordinate system with the forearm root as the origin) to the bucket coordinate system (i.e., the fourth coordinate system with the bucket root as the origin) according to the forearm joint angle, and RAB Represents the third transformation matrix, then the third transformation matrix can be expressed as:

[0103]

[0104] Step S202-4: determining a fourth transformation matrix according to the bucket joint angle, wherein the fourth transformation matrix represents a transformation relationship from the fourth coordinate system to a fifth coordinate system with the bucket tooth tip as the origin.

[0105] The bucket joint angle represents the bucket's posture information at the first digging moment, such as Figure 4 As shown, the line segment BH is the straight rod between the bucket's equivalent rotating joints. The DH (Denavit-Hartenberg) parameter model of the robot forward kinematics can be used to determine the fourth transformation matrix from the bucket coordinate system (i.e., the fourth coordinate system with the bucket root as the origin) to the bucket tooth tip coordinate system (i.e., the fifth coordinate system with the bucket tooth tip as the origin) according to the bucket joint angle, and R BH Represents the fourth transformation matrix, then the fourth transformation matrix can be expressed as:

[0106]

[0107] Step S202-5: Calculate a fifth transformation matrix based on the second transformation matrix, the third transformation matrix, and the fourth transformation matrix, where the fifth transformation matrix represents a transformation relationship from the first coordinate system to the fifth coordinate system.

[0108] like Figure 4 As shown, according to the above second transformation matrix, third transformation matrix, and fourth transformation matrix, the fifth transformation matrix from the boom coordinate system (i.e., the first coordinate system with the boom root as the origin) to the bucket tooth tip coordinate system (i.e., the fifth coordinate system with the bucket tooth tip as the origin) can be obtained, with R OH Represents the fifth transformation matrix, then the fifth transformation matrix can be expressed as:

[0109] R OH =R OA ·R AB ·R BH

[0110] Step S202-6, based on the fifth transformation matrix, convert the coordinates of each point of the bucket tooth tip of the excavator in the fifth coordinate system into coordinates in the first coordinate system, and use the coordinates of each point of the bucket tooth tip of the excavator in the first coordinate system as the position information of each point of the bucket tooth tip of the excavator.

[0111] The coordinates of each point on the excavator bucket's tooth tip are in the bucket's tooth tip coordinate system (i.e., the fifth coordinate system). Since the elevation map obtained in step S201 uses the horizontal plane of the first coordinate system as the reference plane, the coordinates of each point on the bucket's tooth tip must also be converted to the first coordinate system. Subsequent calculations can only be performed if the coordinates of each point on the bucket's tooth tip are consistent with the reference plane of the elevation map.

[0112] Based on the fifth transformation matrix, the coordinates of each point on the bucket tooth tip in the fifth coordinate system can be converted into coordinates in the first coordinate system, and the coordinates of each point on the bucket tooth tip in the first coordinate system are used as the position information of each point on the bucket tooth tip.

[0113] like Figure 4 As shown, H i (0,y i , BH) represents the coordinates of each point on the bucket tooth tip in the fifth coordinate system. According to the fifth transformation matrix, the coordinates of each point on the bucket tooth tip in the first coordinate system can be calculated, which can be expressed as follows:

[0114] H o =R OH ·[0,y i ,BH,1] T

[0115] Among them, y i Indicates the offset of each point of the bucket tooth tip from the middle point of the bucket.

[0116] Step S203 , calculating a first excavation depth of each point on the bucket tooth tip at the first excavation moment based on the position information and the elevation map.

[0117] The elevation map obtained through step S201 is a map in the first coordinate system, and the position information of each point on the bucket tooth tip obtained through step S202 is a three-dimensional coordinate in the first coordinate system. Therefore, through the above two steps, not only the elevation map and the position information of each point on the bucket tooth tip are obtained, but also both are converted into the same coordinate system to facilitate the calculation of the excavation depth of each point on the bucket tooth tip. In this embodiment, the excavation depth of each point on the bucket tooth tip at the first excavation moment is defined as the first excavation depth.

[0118] This embodiment provides an optional method for calculating the first excavation depth of each point of the bucket tooth tip, including the following steps S203-1 to S203-4: Figure 5 This is a schematic diagram of the method for calculating the digging depth of each point of the bucket tooth tip provided by this embodiment. Figure 5 The excavation depth calculation method provided in this embodiment is described in detail.

[0119] Step S203 - 1 , obtaining a first distance from each point on the bucket tooth tip to the horizontal plane of the first coordinate system according to the position information.

[0120] The position information of each point of the bucket tooth tip is the three-dimensional coordinate (x, y, z) of each point of the bucket tooth tip in the first coordinate system. Therefore, the first distance from each point of the bucket tooth tip to the horizontal plane of the first coordinate system is z in the three-dimensional coordinate.

[0121] like Figure 5 As shown, taking point M on the bucket tooth tip as an example, point M is an arbitrary excavation point on the bucket tooth tip. AB plane is the horizontal plane of the first coordinate system, and the distance d0 from point M to AB plane is the first distance.

[0122] Step S203 - 2 , projecting each point of the bucket tooth tip onto the elevation map to obtain projection points corresponding to each point of the bucket tooth tip.

[0123] Project each point on the bucket tooth tip onto the elevation map along the vertical direction of the horizontal plane of the first coordinate system, such as Figure 5 As shown, point M' is the projection point of point M on the elevation map.

[0124] Step S203-3: Obtain a second distance from the projection point corresponding to each point on the bucket tooth tip to the horizontal plane of the first coordinate system according to the elevation map.

[0125] Each point in the elevation map has its three-dimensional coordinates (x', y', z') in the first coordinate system. Therefore, the second distance from the projection point corresponding to each point on the bucket tooth tip to the horizontal plane of the first coordinate system is z' in the three-dimensional coordinates.

[0126] like Figure 5 As shown, the distance d from the projection point M' of point M on the elevation map to the AB surface is g is the second distance.

[0127] Step S203 - 4 , calculating the first excavation depth of each point on the bucket tooth tip according to the first distance and the second distance corresponding to each point on the bucket tooth tip.

[0128] By obtaining the first distance corresponding to each point on the bucket tooth tip and the second distance corresponding to the projection point corresponding to each point on the bucket tooth tip, the first digging depth of each point on the bucket tooth tip can be calculated based on the first distance and the second distance. Specifically, the difference between the second distance and the first distance can be used as the first digging depth of each point on the bucket tooth tip.

[0129] Still Figure 5For example, the difference between the second distance corresponding to the point M and the first distance corresponding to the point M is used as the first excavation depth of the point M. That is, the first excavation depth d of the point M at the first excavation moment can be expressed as:

[0130] d=d g -d o

[0131] Of course, if the bucket has not yet touched the ground, there will be no excavation depth. Therefore, in another optional implementation provided by this embodiment, before the step of calculating the first excavation depth of each point on the bucket tooth tip based on the first distance and the second distance corresponding to each point on the bucket tooth tip, it can be determined whether the point M on the bucket tooth tip has touched the ground based on the first distance and the second distance corresponding to the point M on the bucket tooth tip, that is, whether to start calculating the first excavation depth of the point M. Specifically,

[0132] If the first distance is less than or equal to the second distance, start calculating the first excavation depth of the point M; if the first distance is greater than the second distance, obtain the position information of the point M at the second excavation moment, and the second excavation moment is the next excavation moment after the first excavation moment; wherein, the point M is any point among the points of the bucket tooth tip, and M is a positive integer greater than or equal to 1.

[0133] like Figure 5 As shown, if at the first excavation moment, d0 is less than or equal to d g , point M is in contact with the ground or has been inserted into the ground, then it is necessary to calculate the first excavation depth d of point M at the first excavation moment. If at the first excavation moment, d0 is greater than d g , point M is not in contact with the ground, that is, there is no excavation depth of point M, then the position information of point M at the next excavation moment is continued to be obtained.

[0134] Step S204 , calculating the amount of earth excavated by the excavator at the first excavation moment according to the first excavation depth of each point of the bucket tooth tip at the first excavation moment.

[0135] The first excavation depth of each point on the bucket tooth tip at the first excavation moment is obtained in step S203. Then, the excavation amount of the excavator at the first excavation moment can be calculated based on the first excavation depth of each point. Specifically, the process may include the following steps S204-1 to S204-2.

[0136] Step S204 - 1 , calculating the sub-excavation amount corresponding to each point on the bucket tooth tip at the first excavation moment according to the first excavation depth at each point on the bucket tooth tip at the first excavation moment.

[0137] The contact moments between the bucket tooth tips and the ground are different. Therefore, at the first excavation moment, some excavation points may have already penetrated the ground, some excavation points may have just contacted the ground, and some excavation points may not have yet contacted the ground. Therefore, it is necessary to calculate the real-time excavation volume corresponding to each excavation point at the first excavation moment based on the real-time excavation depth of each excavation point at the first excavation moment. In this embodiment, the real-time excavation volume corresponding to each excavation point at the first excavation moment is defined as the sub-excavation volume.

[0138] In an optional implementation provided by this embodiment, calculating the sub-excavation volume corresponding to each point on the bucket tooth tip at the first excavation moment may include the following steps S204-11 to S204-12.

[0139] Step S204-11, calculates the excavation increment of each point of the bucket tooth tip at the first excavation moment based on the first excavation depth of each point of the bucket tooth tip at the first excavation moment, the first excavation length of the moving path of each point of the bucket tooth tip at the first excavation moment projected on the elevation map, and the first excavation width corresponding to each point of the bucket tooth tip on the elevation map.

[0140] The movement path of each point on the bucket tooth tip during the first excavation moment can be understood as the trajectory of the bucket tooth tip during the first excavation moment. Projecting this movement path onto the elevation map yields the horizontal excavation length of each point on the bucket tooth tip relative to the ground. In this embodiment, the horizontal excavation length of each point on the bucket tooth tip during the first excavation moment, projected onto the elevation map, is defined as the first excavation length.

[0141] The first excavation width corresponding to each point of the bucket tooth tip on the elevation map can be understood as the longitudinal width corresponding to each point of the bucket tooth tip on the elevation map. In an optional implementation provided in this embodiment, the side length of a grid on the elevation map is used as the first excavation width corresponding to each point of the bucket tooth tip on the elevation map. For illustration, a mesh elevation map is used as an example. A mesh elevation map has been described in detail in step S201. This elevation map is composed of multiple grids, and the resolution is the side length of the grid. The excavation increment of each point of the bucket tooth tip at the first excavation moment can be determined by multiplying the first excavation depth, the first excavation length, and the resolution at each point of the bucket tooth tip at the first excavation moment.

[0142] In an optional implementation provided by this embodiment, the first excavation length corresponding to each excavation moment is limited to the side length of a grid of the elevation map.

[0143] like Figure 5 As shown, the excavation increment Vi corresponding to point M at the first excavation moment can be expressed as:

[0144] V i =(d g -d o )×ε×ε

[0145] Where ε is the resolution of the elevation map.

[0146] If the distribution of excavation points on the bucket tooth tip is relatively dense, there may be a situation where one square in the elevation map corresponds to multiple excavation points. If the excavation increment corresponding to each excavation point of the square is calculated using the above formula, then the amount of soil in one square will be calculated repeatedly, and the calculated excavation increment will be more than the actual excavation increment. Therefore, if one square corresponds to multiple points on the bucket tooth tip, the excavation increment corresponding to any one of the points at the first excavation moment is calculated.

[0147] There will not be significant differences in the excavation depths of multiple excavation points corresponding to the same grid. Therefore, any excavation point can be randomly selected and the above formula can be used to calculate the excavation increment.

[0148] Step S204-12: Accumulate the excavation increments at the first excavation moment and each excavation moment before the first excavation moment to obtain the sub-excavation amount corresponding to each point of the bucket tooth tip at the first excavation moment.

[0149] After calculating the excavation increment of each point on the bucket tooth tip at the first excavation moment, the excavation increment corresponding to each point can be accumulated with the excavation increment of each point at each excavation moment before the first excavation moment to obtain the sub-excavation amount of each point at the first excavation moment.

[0150] Step S204 - 2 , accumulating the sub-excavation volumes corresponding to the respective points of the bucket tooth tip at the first excavation moment, to obtain the excavation volume of the excavator at the first excavation moment.

[0151] After calculating the sub-excavation volume of each point on the bucket tooth tip at the first excavation moment, the sub-excavation volume of each point at the first excavation moment is accumulated and added together to obtain the total excavation volume of the entire bucket at the first excavation moment, that is, the excavation volume of the excavator at the first excavation moment.

[0152] Take the mesh elevation map as an example. Each square in the elevation map has its three-dimensional coordinates. Each square in the elevation map is marked with j. At the same time, each excavation point on the bucket tooth tip is marked with i. Then, the elevation map grid j that each point i of the bucket tooth tip passes through during the excavation process is recorded, as well as the height of each point of the bucket tooth tip from the horizontal plane of the first coordinate system. The height of the projection points of each point on the bucket tooth tip on the elevation map from the horizontal plane of the first coordinate system The excavation volume V of the excavator at each excavation moment can be calculated in real time. The formula is as follows:

[0153]

[0154] After the server calculates the real-time excavation volume of the excavator, it can transmit the calculation result to the excavator. In another optional implementation method provided in this embodiment, the server can also transmit image information of the excavator during operation to the excavator, so that the excavator can not only provide the excavator operator with the real-time excavation volume results of the excavator, but also present a visual image of the excavation process of the excavator to the excavator operator, so that the excavator operator can intuitively see the status of the excavator and the excavator bucket at the current excavation moment, and thus make real-time decisions on the excavation content of the next excavation moment, such as: whether to continue digging deep or lift the bucket at the next excavation moment.

[0155] Visual images of the excavation process can be displayed on the excavator's display screen, such as the excavator's control module or the excavator's remote control system. This intuitive observation allows the excavator operator to make more accurate decisions.

[0156] Figure 6 This is a diagram showing the operation of the excavator provided in this embodiment.

[0157] Figure 6 (a) is a diagram showing the overall posture of the excavator at the first excavation moment. Figure 6 (b) Yes Figure 6 (a) The corresponding visualization image, Figure 6 (c) is a diagram showing the arm posture of the excavator at the first excavation moment. Figure 6 (d) Yes Figure 6 (c) Corresponding visualization image. Figure 6 (b) and Figure 6 (d) It is displayed on the display screen of the excavator, and the excavator operator can Figure 6 (b) and Figure 6 (d) Visually see the degree of contact between the bucket and the ground, the bucket's digging depth, and the current digging status of the excavator at the first digging moment.

[0158] Steps S201 to S205 provide a method for calculating the real-time excavation volume of the excavator. This method can calculate the current real-time excavation volume at each excavation moment in the excavation cycle. This method can enable the control of the excavator to reach a more refined level.

[0159] Figure 7This is a flow chart for calculating the real-time excavation volume of an excavator provided in this embodiment. During the operation of the excavator, the server can automatically monitor the excavation volume according to this flow.

[0160] like Figure 7 As shown in the figure, the real-time excavation volume calculation process of the excavator in one excavation cycle includes the following steps:

[0161] In step S701, in response to the excavator operator starting the excavation operation (for example, the excavator operator clicks the start button of the excavator, controls the movement of the excavator arm, etc.), the server controls the 3D laser ranging radar installed on the top of the excavator cabin to collect ground point cloud information of the excavator operation scene.

[0162] In step S702, the server obtains the ground point cloud information collected by the 3D laser ranging radar, converts the ground point cloud information into the boom coordinate system, and establishes an elevation map with the horizontal plane of the boom coordinate system as the reference plane.

[0163] In step S703, the server calculates the conversion matrix from the bucket tooth tip coordinate system to the boom coordinate system at the current excavation moment based on the inclination sensors installed at each joint of the excavator arm, converts the coordinates of each digging point at the bucket tooth tip from the bucket tooth tip coordinate system to the boom coordinate system, and calculates the first and second height information of each digging point and the projection point of each digging point on the elevation map to the horizontal plane of the first coordinate system.

[0164] In step S704, the server determines whether the first height information is less than or equal to the second height information.

[0165] In step S705, if the server determines that the first height information is less than or equal to the second height information, the excavation depth of the bucket tooth tip excavation point is calculated based on the first height information and the second height information; if the server determines that the first height information is greater than the second height information, the server returns to step S703 and recalculates the first height information and the second height information of each excavation point and the projection point of each excavation point on the elevation map to the horizontal plane of the first coordinate system at the next excavation moment.

[0166] In step S706, the server calculates the excavation increment of the excavator at the current excavation moment based on the excavation depth of the bucket tooth tip excavation point, accumulates the excavation volume at the current excavation moment, returns to step S703, recalculates the excavation increment at the next excavation moment, and accumulates the excavation volume at the next excavation moment.

[0167] The above-mentioned first embodiment provides an optional implementation method of the method for calculating the excavation volume of an excavator. It should be noted that the exemplary description in the first embodiment is only for the purpose of facilitating the understanding of the method described in this application, and is not intended to be limiting. The method for calculating the excavation volume described in this application includes but is not limited to the implementation method provided in the above-mentioned first embodiment.

[0168] The second embodiment of the present application provides a device for calculating the amount of earth removed by an excavator. Figure 8 Schematic diagram of the structure of the excavation volume calculation device of the excavator provided in this embodiment.

[0169] like Figure 8 As shown, the excavation amount calculation device provided by this embodiment includes: an elevation map construction unit 801, a position information acquisition unit 802, an excavation depth calculation unit 803, and an excavation amount calculation unit 804.

[0170] The elevation map construction unit 801 is used to obtain ground point cloud information of the excavator operation scene and establish an elevation map based on the ground point cloud information. The elevation map uses the horizontal plane of the first coordinate system as the reference plane. The first coordinate system is a coordinate system with the root of the excavator's boom as the origin.

[0171] Optionally, the excavator is equipped with a distance measuring device, and obtaining ground point cloud information of the excavator operation scene includes:

[0172] The ground point cloud information is collected by the ranging device, and the ground point cloud information is the coordinates of each point on the ground of the excavator operation scene in a second coordinate system with the ranging device as the origin.

[0173] Optionally, establishing an elevation map based on the ground point cloud information includes:

[0174] Converting the ground point cloud information based on a first conversion matrix to obtain second ground point cloud information, where the second ground point cloud information is the coordinates of each point on the ground in the first coordinate system, and the first conversion matrix represents a conversion relationship from the second coordinate system to the first coordinate system;

[0175] The elevation map is established based on the second ground point cloud information.

[0176] Optionally, the elevation map is composed of a plurality of squares of the same size, and the side length of the square is the resolution of the elevation map.

[0177] Optionally, before the step of establishing the elevation map according to the second ground point cloud information, the elevation map construction unit 801 is further configured to:

[0178] The point cloud information corresponding to the arm of the excavator is deleted from the second ground point cloud information.

[0179] The position information acquisition unit 802 is used to obtain the position information of each point of the bucket tooth tip of the excavator at the first excavation moment. The position information is the coordinates of each point of the bucket tooth tip in the first coordinate system. The first excavation moment is any excavation moment within the process of the excavator completing an excavation operation.

[0180] Optionally, the excavator is equipped with a plurality of sensors, and the acquiring of position information of each point of the bucket tooth tip of the excavator at the first excavation moment includes:

[0181] Calculating, based on readings of the multiple sensors at the first excavation moment, a boom joint angle of the excavator, a forearm joint angle of the excavator, and a bucket joint angle of the excavator at the first excavation moment;

[0182] Determining a second transformation matrix based on the upper arm joint angle, wherein the second transformation matrix represents a transformation relationship from the first coordinate system to a third coordinate system with the base of the forearm as the origin;

[0183] Determining a third transformation matrix according to the forearm joint angle, wherein the third transformation matrix represents a transformation relationship from the third coordinate system to a fourth coordinate system with the bucket root as the origin;

[0184] Determining a fourth transformation matrix according to the bucket joint angle, wherein the fourth transformation matrix represents a transformation relationship from the fourth coordinate system to a fifth coordinate system with the bucket tooth tip as the origin;

[0185] Calculate a fifth transformation matrix based on the second transformation matrix, the third transformation matrix, and the fourth transformation matrix, wherein the fifth transformation matrix represents a transformation relationship from the first coordinate system to the fifth coordinate system;

[0186] Based on the fifth transformation matrix, the coordinates of each point on the bucket tooth tip of the excavator in the fifth coordinate system are converted into coordinates in the first coordinate system, and the coordinates of each point on the bucket tooth tip of the excavator in the first coordinate system are used as the position information of each point on the bucket tooth tip of the excavator.

[0187] The excavation depth calculation unit 803 is configured to calculate a first excavation depth of each point on the bucket tooth tip at the first excavation moment based on the position information and the elevation map.

[0188] Optionally, calculating the first excavation depth of each point on the bucket tooth tip at the first excavation moment based on the position information and the elevation map includes:

[0189] Acquire a first distance from each point on the bucket tooth tip to a horizontal plane of the first coordinate system according to the position information;

[0190] Projecting each point of the bucket tooth tip onto the elevation map to obtain a projection point corresponding to each point of the bucket tooth tip;

[0191] Acquire, according to the elevation map, a second distance from a projection point corresponding to each point on the bucket tooth tip to a horizontal plane of the first coordinate system;

[0192] The first excavation depth of each point on the bucket tooth tip is calculated according to the first distance and the second distance corresponding to each point on the bucket tooth tip.

[0193] Optionally, before the step of calculating the first excavation depth of each point on the bucket tooth tip according to the first distance and the second distance corresponding to each point on the bucket tooth tip, the excavation depth calculation unit 803 is further configured to:

[0194] According to the first distance and the second distance corresponding to the point M on the bucket tooth tip, it is determined whether to start calculating the first excavation depth of the point M, specifically:

[0195] If the first distance is less than or equal to the second distance, start calculating the first excavation depth of the point M;

[0196] If the first distance is greater than the second distance, obtaining the position information of the point M at a second excavation time, where the second excavation time is the next excavation time after the first excavation time;

[0197] The point M is any point among the tooth tips of the bucket.

[0198] Optionally, calculating the first excavation depth of each point on the bucket tooth tip according to the first distance and the second distance corresponding to each point on the bucket tooth tip includes:

[0199] The first excavation depth of the point M is determined as the difference between the second distance corresponding to the point M and the first distance corresponding to the point M.

[0200] The excavation amount calculation unit 804 is used to calculate the excavation amount of the excavator at the first excavation moment according to the first excavation depth of each point of the bucket tooth tip at the first excavation moment.

[0201] Optionally, calculating the amount of earth excavated by the excavator at the first excavation moment according to the first excavation depth of each point of the bucket tooth tip at the first excavation moment includes:

[0202] Calculating the sub-excavation amount corresponding to each point on the bucket tooth tip at the first excavation moment according to the first excavation depth of each point on the bucket tooth tip at the first excavation moment;

[0203] The sub-excavation amounts corresponding to each point of the bucket tooth tip at the first excavation moment are accumulated to obtain the excavation amount of the excavator at the first excavation moment.

[0204] Optionally, the calculating, based on the first excavation depth of each point on the bucket tooth tip at the first excavation moment, the sub-excavation amount corresponding to each point on the bucket tooth tip at the first excavation moment includes:

[0205] Calculating an excavation increment of each point of the bucket tooth tip at the first excavation moment based on the first excavation depth of each point of the bucket tooth tip at the first excavation moment, a first excavation length of a movement path of each point of the bucket tooth tip at the first excavation moment projected on the elevation map, and a first excavation width corresponding to each point of the bucket tooth tip on the elevation map;

[0206] The excavation increments at the first excavation moment and each excavation moment before the first excavation moment are accumulated to obtain the sub-excavation amounts corresponding to each point of the bucket tooth tip at the first excavation moment.

[0207] Optionally, the elevation map is composed of a plurality of grids, the first excavation width corresponding to each point of the bucket tooth tip on the elevation map is the side length of the grid, and the side length of the grid is the resolution of the elevation map; the calculating the excavation increment of each point of the bucket tooth tip at the first excavation moment based on the first excavation depth of each point of the bucket tooth tip at the first excavation moment, the first excavation length projected on the elevation map of the movement path of each point of the bucket tooth tip at the first excavation moment, and the first excavation width corresponding to each point of the bucket tooth tip on the elevation map includes:

[0208] The product of the first excavation depth, the first excavation length, and the resolution at each point of the bucket tooth tip at the first excavation moment is used as the excavation increment of each point of the bucket tooth tip at the first excavation moment.

[0209] Optionally, if one of the squares corresponds to a plurality of points on the bucket tooth tip, the excavation increment corresponding to any one of the points at the first excavation moment is calculated.

[0210] The third embodiment of the present application provides an excavator, Figure 9 It is a structural schematic diagram of the excavator provided in this embodiment.

[0211] like Figure 9 As shown, the excavator provided in this embodiment is equipped with a vehicle-mounted device 901, a distance measuring device 902, and a plurality of sensors 903;

[0212] The vehicle-mounted device 901 includes: a first acquisition component 9011, an elevation map construction component 9012, a second acquisition component 9013, a first calculation component 9014, and a second calculation component 9015;

[0213] The first acquisition component 9011 is used to acquire ground point cloud information of the excavator operation scene;

[0214] The elevation map construction component 9012 is used to create an elevation map based on the ground point cloud information, wherein the elevation map uses the horizontal plane of a first coordinate system as a reference plane, wherein the first coordinate system is a coordinate system with the base of the excavator's boom as its origin;

[0215] The second acquisition component 9013 is used to obtain position information of each point of the bucket tooth tip of the excavator at a first excavation moment, where the position information is the coordinates of each point of the bucket tooth tip in the first coordinate system. The first excavation moment is any moment within the process of the excavator completing an excavation operation;

[0216] The first calculation component 9014 is configured to calculate a first excavation depth of each point on the bucket tooth tip at the first excavation moment based on the position information and the elevation map;

[0217] The second calculation component 9015 is used to calculate the amount of soil excavated by the excavator at the first excavation moment based on the first excavation depth of each point of the bucket tooth tip at the first excavation moment;

[0218] The distance measuring device 902 is used to collect ground point cloud information of the excavator operation scene and transmit the ground point cloud information to the first acquisition component;

[0219] The multiple sensors 903 are used to collect the posture information of each joint of the excavator at the first excavation moment, and transmit the posture information to the second acquisition component, so that the second acquisition component obtains the position information of each point of the bucket tooth tip of the excavator at the first excavation moment based on the posture information.

[0220] Optionally, the ranging device 902 is a 3D laser ranging radar, which is configured on the top of the cabin of the excavator.

[0221] Optionally, the sensor 903 is an inclination sensor, and the multiple inclination sensors are respectively arranged at the connection between the boom and the cabin, the connection between the arm and the boom, the connection between the bucket and the arm, and the connection between the cabin and the track of the excavator.

[0222] The specific operation method of the excavator can refer to the description of the first embodiment and will not be repeated here.

[0223] A fourth embodiment of the present application provides an electronic device, Figure 10 It is a structural diagram of the electronic device provided in this embodiment.

[0224] like Figure 10 As shown, the electronic device provided in this embodiment includes: a memory 1001, a processor 1002;

[0225] The memory 1001 is used to store computer instructions for executing a method for calculating the amount of earth removed by an excavator;

[0226] The processor 1002 is configured to execute computer instructions stored in the memory 1001 to perform the following operations:

[0227] Acquire ground point cloud information of the excavator operation scene, and establish an elevation map based on the ground point cloud information, wherein the elevation map uses a horizontal plane of a first coordinate system as a reference plane, and the first coordinate system is a coordinate system with the base of the excavator's boom as the origin;

[0228] Acquire position information of each point on the bucket tooth tip of the excavator at a first excavation moment, where the position information is the coordinates of each point on the bucket tooth tip in the first coordinate system. The first excavation moment is any excavation moment within a process in which the excavator completes an excavation operation.

[0229] Calculating a first excavation depth of each point on the bucket tooth tip at the first excavation moment based on the position information and the elevation map;

[0230] The amount of earth excavated by the excavator at the first excavation moment is calculated according to the first excavation depth of each point of the bucket tooth tip at the first excavation moment.

[0231] Optionally, the excavator is equipped with a distance measuring device, and obtaining ground point cloud information of the excavator operation scene includes:

[0232] The ground point cloud information is collected by the ranging device, and the ground point cloud information is the coordinates of each point on the ground of the excavator operation scene in a second coordinate system with the ranging device as the origin.

[0233] Optionally, establishing an elevation map based on the ground point cloud information includes:

[0234] Converting the ground point cloud information based on a first conversion matrix to obtain second ground point cloud information, where the second ground point cloud information is the coordinates of each point on the ground in the first coordinate system, and the first conversion matrix represents a conversion relationship from the second coordinate system to the first coordinate system;

[0235] The elevation map is established based on the second ground point cloud information.

[0236] Optionally, the elevation map is composed of a plurality of squares of the same size, and the side length of the square is the resolution of the elevation map.

[0237] Optionally, before the step of establishing the elevation map according to the second ground point cloud information, the method further includes:

[0238] The point cloud information corresponding to the arm of the excavator is deleted from the second ground point cloud information.

[0239] Optionally, the excavator is equipped with a plurality of sensors, and the acquiring of position information of each point of the bucket tooth tip of the excavator at the first excavation moment includes:

[0240] Calculating, based on readings of the multiple sensors at the first excavation moment, a boom joint angle of the excavator, a forearm joint angle of the excavator, and a bucket joint angle of the excavator at the first excavation moment;

[0241] Determining a second transformation matrix based on the upper arm joint angle, wherein the second transformation matrix represents a transformation relationship from the first coordinate system to a third coordinate system with the base of the forearm as the origin;

[0242] Determining a third transformation matrix according to the forearm joint angle, wherein the third transformation matrix represents a transformation relationship from the third coordinate system to a fourth coordinate system with the bucket root as the origin;

[0243] Determining a fourth transformation matrix according to the bucket joint angle, wherein the fourth transformation matrix represents a transformation relationship from the fourth coordinate system to a fifth coordinate system with the bucket tooth tip as the origin;

[0244] Calculate a fifth transformation matrix based on the second transformation matrix, the third transformation matrix, and the fourth transformation matrix, wherein the fifth transformation matrix represents a transformation relationship from the first coordinate system to the fifth coordinate system;

[0245] Based on the fifth transformation matrix, the coordinates of each point on the bucket tooth tip of the excavator in the fifth coordinate system are converted into coordinates in the first coordinate system, and the coordinates of each point on the bucket tooth tip of the excavator in the first coordinate system are used as the position information of each point on the bucket tooth tip of the excavator.

[0246] Optionally, calculating the first excavation depth of each point on the bucket tooth tip at the first excavation moment based on the position information and the elevation map includes:

[0247] Acquire a first distance from each point on the bucket tooth tip to a horizontal plane of the first coordinate system according to the position information;

[0248] Projecting each point of the bucket tooth tip onto the elevation map to obtain a projection point corresponding to each point of the bucket tooth tip;

[0249] Acquire, according to the elevation map, a second distance from a projection point corresponding to each point on the bucket tooth tip to a horizontal plane of the first coordinate system;

[0250] The first excavation depth of each point on the bucket tooth tip is calculated according to the first distance and the second distance corresponding to each point on the bucket tooth tip.

[0251] Optionally, before the step of calculating the first excavation depth of each point on the bucket tooth tip according to the first distance and the second distance corresponding to each point on the bucket tooth tip, the method further includes:

[0252] According to the first distance and the second distance corresponding to the point M on the bucket tooth tip, it is determined whether to start calculating the first excavation depth of the point M, specifically:

[0253] If the first distance is less than or equal to the second distance, start calculating the first excavation depth of the point M;

[0254] If the first distance is greater than the second distance, obtaining the position information of the point M at a second excavation time, where the second excavation time is the next excavation time after the first excavation time;

[0255] The point M is any point among the tooth tips of the bucket.

[0256] Optionally, calculating the first excavation depth of each point on the bucket tooth tip according to the first distance and the second distance corresponding to each point on the bucket tooth tip includes:

[0257] The first excavation depth of the point M is determined as the difference between the second distance corresponding to the point M and the first distance corresponding to the point M.

[0258] Optionally, calculating the amount of earth excavated by the excavator at the first excavation moment according to the first excavation depth of each point of the bucket tooth tip at the first excavation moment includes:

[0259] Calculating the sub-excavation amount corresponding to each point on the bucket tooth tip at the first excavation moment according to the first excavation depth of each point on the bucket tooth tip at the first excavation moment;

[0260] The sub-excavation amounts corresponding to each point of the bucket tooth tip at the first excavation moment are accumulated to obtain the excavation amount of the excavator at the first excavation moment.

[0261] Optionally, the calculating, based on the first excavation depth of each point on the bucket tooth tip at the first excavation moment, the sub-excavation amount corresponding to each point on the bucket tooth tip at the first excavation moment includes:

[0262] Calculating an excavation increment of each point of the bucket tooth tip at the first excavation moment based on the first excavation depth of each point of the bucket tooth tip at the first excavation moment, a first excavation length of a movement path of each point of the bucket tooth tip at the first excavation moment projected on the elevation map, and a first excavation width corresponding to each point of the bucket tooth tip on the elevation map;

[0263] The excavation increments at the first excavation moment and each excavation moment before the first excavation moment are accumulated to obtain the sub-excavation amounts corresponding to each point of the bucket tooth tip at the first excavation moment.

[0264] Optionally, the elevation map is composed of a plurality of grids, the first excavation width corresponding to each point of the bucket tooth tip on the elevation map is the side length of the grid, and the side length of the grid is the resolution of the elevation map; the calculating the excavation increment of each point of the bucket tooth tip at the first excavation moment based on the first excavation depth of each point of the bucket tooth tip at the first excavation moment, the first excavation length projected on the elevation map of the movement path of each point of the bucket tooth tip at the first excavation moment, and the first excavation width corresponding to each point of the bucket tooth tip on the elevation map includes:

[0265] The product of the first excavation depth, the first excavation length, and the resolution at each point of the bucket tooth tip at the first excavation moment is used as the excavation increment of each point of the bucket tooth tip at the first excavation moment.

[0266] Optionally, if one of the squares corresponds to a plurality of points on the bucket tooth tip, the excavation increment corresponding to any one of the points at the first excavation moment is calculated.

[0267] For the relevant parts of the above content, reference can be made to the description of the first embodiment and will not be repeated here.

[0268] A fifth embodiment of the present application provides a computer-readable storage medium, which includes computer instructions. When the computer instructions are executed by a processor, they are used to implement the methods described in each embodiment of the present application.

[0269] It should be noted that relational terms such as "first" and "second" in this document are used only to distinguish one entity or operation from another entity or operation, and do not require or imply any actual relationship or order between these entities or operations. In addition, the words "include," "have," "include," and "includes" and other similar forms are synonymous in meaning, and the ending of any one or more items following any of the above words is open-ended, and none of the above terms indicates that the one or more items are exhaustive or limited to the one or more items listed.

[0270] As used herein, unless expressly stated otherwise, the term "or" includes all possible combinations, except those that are infeasible. For example, if a statement states that a database may include A or B, then unless otherwise specified or infeasible, it may include databases A, B, or A and B. As a second example, if a statement states that a database may include A, B, or C, then unless otherwise specified or infeasible, it may include databases A, B, or C, or A and B, or A and C, or B and C, or A, B, and C.

[0271] It is worth noting that the above embodiments can be implemented by hardware or software (program code), or a combination of hardware and software. If implemented by software, it can be stored in the above-mentioned computer-readable medium. When the software is executed by a processor, it can execute the above-mentioned disclosed method. The computing unit and other functional units described in this disclosure can be implemented by hardware or software, or a combination of hardware and software. Those of ordinary skill in the art will also understand that the above-mentioned multiple modules / units can be combined into one module / unit, and each of the above-mentioned modules / units can be further divided into multiple sub-modules / sub-units.

[0272] In the above detailed description, the embodiments have been described with reference to many specific details, which may vary depending on the implementation. Certain adaptations and modifications may be made to the embodiments. For those skilled in the art, other embodiments will be readily apparent from the specific embodiments disclosed herein. This description and examples are for illustrative purposes only, and the true scope and nature of this application are described in the claims. The order of steps shown in the figures is also for illustrative purposes only and is not intended to be limiting to any particular steps or order. Therefore, those skilled in the art will appreciate that these steps may be performed in different orders when implementing the same method.

[0273] In the drawings and detailed description of this application, exemplary embodiments are disclosed. However, many variations and modifications may be made to these embodiments. Accordingly, although specific terms are used, these terms are used in a general and descriptive sense only and not for purposes of limitation.

Claims

1. A method for calculating the amount of earth excavated by an excavator, characterized in that: The method comprises: Acquire ground point cloud information of the excavator operation scene, and establish an elevation map based on the ground point cloud information, wherein the elevation map uses a horizontal plane of a first coordinate system as a reference plane, and the first coordinate system is a coordinate system with the base of the excavator's boom as the origin; Acquire position information of each point on the bucket tooth tip of the excavator at a first excavation moment, where the position information is the coordinates of each point on the bucket tooth tip in the first coordinate system. The first excavation moment is any excavation moment within a process in which the excavator completes an excavation operation. Calculating a first excavation depth of each point on the bucket tooth tip at the first excavation moment based on the position information and the elevation map; calculating the amount of earth excavated by the excavator at the first excavation moment according to the first excavation depth of each point of the bucket tooth tip at the first excavation moment; The step of calculating the first excavation depth of each point on the bucket tooth tip at the first excavation moment based on the position information and the elevation map includes: Acquire a first distance from each point on the bucket tooth tip to a horizontal plane of the first coordinate system according to the position information; Projecting each point of the bucket tooth tip onto the elevation map to obtain a projection point corresponding to each point of the bucket tooth tip; Acquire, according to the elevation map, a second distance from a projection point corresponding to each point on the bucket tooth tip to a horizontal plane of the first coordinate system; Calculating the first excavation depth of each point on the bucket tooth tip according to the first distance and the second distance corresponding to each point on the bucket tooth tip; Calculating the first excavation depth of each point on the bucket tooth tip according to the first distance and the second distance corresponding to each point on the bucket tooth tip includes: The difference between the second distance corresponding to point M and the first distance corresponding to point M is used as the first excavation depth of point M, and point M is any point among the points of the bucket tooth tip.

2. The method according to claim 1, characterized in that The excavator is equipped with a distance measuring device, and the acquisition of ground point cloud information of the excavator operation scene includes: The ground point cloud information is collected by the ranging device, and the ground point cloud information is the coordinates of each point on the ground of the excavator operation scene in a second coordinate system with the ranging device as the origin.

3. The method according to claim 2, characterized in that The step of establishing an elevation map based on the ground point cloud information includes: Converting the ground point cloud information based on a first conversion matrix to obtain second ground point cloud information, where the second ground point cloud information is the coordinates of each point on the ground in the first coordinate system, and the first conversion matrix represents a conversion relationship from the second coordinate system to the first coordinate system; The elevation map is established based on the second ground point cloud information.

4. The method according to claim 3, characterized in that The elevation map is composed of a plurality of grids of the same size, and the side length of the grid is the resolution of the elevation map.

5. The method according to claim 3, characterized in that Before the step of establishing the elevation map according to the second ground point cloud information, the method further includes: The point cloud information corresponding to the arm of the excavator is deleted from the second ground point cloud information.

6. The method according to claim 1, characterized in that The excavator is equipped with a plurality of sensors, and the acquisition of position information of each point of the bucket tooth tip of the excavator at the first excavation moment includes: Calculating, based on readings of the multiple sensors at the first excavation moment, a boom joint angle of the excavator, a forearm joint angle of the excavator, and a bucket joint angle of the excavator at the first excavation moment; Determining a second transformation matrix based on the upper arm joint angle, wherein the second transformation matrix represents a transformation relationship from the first coordinate system to a third coordinate system with the base of the forearm as the origin; Determining a third transformation matrix according to the forearm joint angle, wherein the third transformation matrix represents a transformation relationship from the third coordinate system to a fourth coordinate system with the bucket root as the origin; Determining a fourth transformation matrix according to the bucket joint angle, wherein the fourth transformation matrix represents a transformation relationship from the fourth coordinate system to a fifth coordinate system with the bucket tooth tip as the origin; Calculate a fifth transformation matrix based on the second transformation matrix, the third transformation matrix, and the fourth transformation matrix, wherein the fifth transformation matrix represents a transformation relationship from the first coordinate system to the fifth coordinate system; Based on the fifth transformation matrix, the coordinates of each point on the bucket tooth tip of the excavator in the fifth coordinate system are converted into coordinates in the first coordinate system, and the coordinates of each point on the bucket tooth tip of the excavator in the first coordinate system are used as the position information of each point on the bucket tooth tip of the excavator.

7. The method according to claim 1, characterized in that Before the step of calculating the first excavation depth of each point on the bucket tooth tip according to the first distance and the second distance corresponding to each point on the bucket tooth tip, the method further includes: According to the first distance and the second distance corresponding to the point M on the bucket tooth tip, it is determined whether to start calculating the first excavation depth of the point M, specifically: If the first distance is less than or equal to the second distance, start calculating the first excavation depth of the point M; If the first distance is greater than the second distance, the position information of the point M at a second excavation time is acquired, where the second excavation time is the next excavation time after the first excavation time.

8. The method according to claim 1, characterized in that Calculating the amount of earth excavated by the excavator at the first excavation moment according to the first excavation depth of each point on the bucket tooth tip at the first excavation moment includes: Calculating the sub-excavation amount corresponding to each point on the bucket tooth tip at the first excavation moment according to the first excavation depth of each point on the bucket tooth tip at the first excavation moment; The sub-excavation amounts corresponding to each point of the bucket tooth tip at the first excavation moment are accumulated to obtain the excavation amount of the excavator at the first excavation moment.

9. The method according to claim 8, characterized in that The calculating, based on the first excavation depth of each point on the bucket tooth tip at the first excavation moment, the sub-excavation amount corresponding to each point on the bucket tooth tip at the first excavation moment includes: Calculating an excavation increment of each point of the bucket tooth tip at the first excavation moment based on the first excavation depth of each point of the bucket tooth tip at the first excavation moment, a first excavation length of a movement path of each point of the bucket tooth tip at the first excavation moment projected on the elevation map, and a first excavation width corresponding to each point of the bucket tooth tip on the elevation map; The excavation increments at the first excavation moment and each excavation moment before the first excavation moment are accumulated to obtain the sub-excavation amounts corresponding to each point of the bucket tooth tip at the first excavation moment.

10. The method according to claim 9, characterized in that The elevation map is composed of a plurality of squares, the first excavation width corresponding to each point of the bucket tooth tip on the elevation map is the side length of the square, and the side length of the square is the resolution of the elevation map; Calculating the soil excavation increment of each point of the bucket tooth tip at the first excavation moment based on the first excavation depth of each point of the bucket tooth tip at the first excavation moment, a first excavation length of a movement path of each point of the bucket tooth tip at the first excavation moment projected on the elevation map, and a first excavation width corresponding to each point of the bucket tooth tip on the elevation map includes: The product of the first excavation depth, the first excavation length, and the resolution at each point of the bucket tooth tip at the first excavation moment is used as the excavation increment of each point of the bucket tooth tip at the first excavation moment.

11. The method according to claim 10, characterized in that: If one of the squares corresponds to a plurality of points on the bucket tooth tip, the excavation increment corresponding to any one of the points at the first excavation moment is calculated.

12. A device for calculating the amount of earth removed by an excavator, characterized in that: The device includes: an elevation map construction unit, a position information acquisition unit, an excavation depth calculation unit, and an excavation amount calculation unit; The elevation map construction unit is configured to obtain ground point cloud information of the excavator operation scene and to establish an elevation map based on the ground point cloud information, wherein the elevation map uses a horizontal plane of a first coordinate system as a reference plane, wherein the first coordinate system is a coordinate system with the base of the excavator's boom as its origin; The position information acquisition unit is configured to acquire position information of each point on the bucket tooth tip of the excavator at a first excavation moment, wherein the position information is the coordinates of each point on the bucket tooth tip in the first coordinate system, and the first excavation moment is any excavation moment within a process in which the excavator completes an excavation operation; The excavation depth calculation unit is configured to calculate a first excavation depth of each point of the bucket tooth tip at the first excavation moment based on the position information and the elevation map; The excavation amount calculation unit is configured to calculate the excavation amount of the excavator at the first excavation moment based on the first excavation depth of each point of the bucket tooth tip at the first excavation moment; The step of calculating the first excavation depth of each point on the bucket tooth tip at the first excavation moment based on the position information and the elevation map includes: Acquire a first distance from each point on the bucket tooth tip to a horizontal plane of the first coordinate system according to the position information; Projecting each point of the bucket tooth tip onto the elevation map to obtain a projection point corresponding to each point of the bucket tooth tip; Acquire, according to the elevation map, a second distance from a projection point corresponding to each point on the bucket tooth tip to a horizontal plane of the first coordinate system; Calculating the first excavation depth of each point on the bucket tooth tip according to the first distance and the second distance corresponding to each point on the bucket tooth tip; Calculating the first excavation depth of each point on the bucket tooth tip according to the first distance and the second distance corresponding to each point on the bucket tooth tip includes: The difference between the second distance corresponding to point M and the first distance corresponding to point M is used as the first excavation depth of point M, and point M is any point among the points of the bucket tooth tip.

13. An excavator, characterized in that: The excavator is equipped with a vehicle-mounted device, a distance measuring device, and a plurality of sensors; The vehicle-mounted device includes: a first acquisition component, an elevation map construction component, a second acquisition component, a first calculation component, and a second calculation component; The first acquisition component is used to acquire ground point cloud information of the excavator operation scene; The elevation map building component is used to build an elevation map based on the ground point cloud information, wherein the elevation map uses the horizontal plane of a first coordinate system as a reference plane, and the first coordinate system is a coordinate system with the base of the excavator's boom as the origin; The second acquisition component is configured to acquire position information of each point on the bucket tooth tip of the excavator at a first excavation moment, wherein the position information is the coordinates of each point on the bucket tooth tip in the first coordinate system, and the first excavation moment is any excavation moment within a process in which the excavator completes an excavation operation; The first calculation component is configured to calculate a first excavation depth of each point of the bucket tooth tip at the first excavation moment based on the position information and the elevation map; The second calculation component is used to calculate the amount of soil excavated by the excavator at the first excavation moment based on the first excavation depth of each point of the bucket tooth tip at the first excavation moment; The distance measuring device is used to collect ground point cloud information of the excavator operation scene and transmit the ground point cloud information to the first acquisition component; The multiple sensors are used to collect position information of each joint of the excavator at the first excavation moment, and transmit the position information to the second acquisition component, so that the second acquisition component obtains position information of each point of the bucket tooth tip of the excavator at the first excavation moment based on the position information; The step of calculating the first excavation depth of each point on the bucket tooth tip at the first excavation moment based on the position information and the elevation map includes: Acquire a first distance from each point on the bucket tooth tip to a horizontal plane of the first coordinate system according to the position information; Projecting each point of the bucket tooth tip onto the elevation map to obtain a projection point corresponding to each point of the bucket tooth tip; Acquire, according to the elevation map, a second distance from a projection point corresponding to each point on the bucket tooth tip to a horizontal plane of the first coordinate system; Calculating the first excavation depth of each point on the bucket tooth tip according to the first distance and the second distance corresponding to each point on the bucket tooth tip; Calculating the first excavation depth of each point on the bucket tooth tip according to the first distance and the second distance corresponding to each point on the bucket tooth tip includes: The difference between the second distance corresponding to point M and the first distance corresponding to point M is used as the first excavation depth of point M, and point M is any point among the points of the bucket tooth tip.

14. An electronic device, characterized in that: include: memory and processor; The memory is used to store one or more computer instructions; The processor is configured to execute the one or more computer instructions to implement the method according to any one of claims 1 to 11.

15. A computer-readable storage medium having one or more computer instructions stored thereon, characterized in that: The instruction is executed by a processor to implement the method according to any one of claims 1 to 11.

Images