Excavator monitoring system and method based on panoramic view, excavator and product
By using multi-camera and 360-degree ambient view module on the excavator for image stitching and fusion, the problem of visual blind spots in excavator operations is solved, panoramic monitoring is realized, and safety and efficiency are improved.
Patent Information
- Application Number
- CN202411133286.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-19
- Publication Date
- 2025-01-14
AI Technical Summary
When the excavator is operating, the movement of the excavator arm and bucket will block the camera, causing some areas to be unable to be captured, forming visual blind spots, increasing the risk of accidents and affecting the accuracy and efficiency of operations.
Using a plurality of first cameras and second cameras, 3D stereoscopic space modeling and image stitching are performed through the 360-degree ambient view module and the image fusion module, the images captured by the second camera are fused to display the area blocked by the digging arm in a perspective manner. Ensure the integrity of the 360-degree surrounding panorama.
It effectively solves the visual blind spot problem during excavator operation, improves the driver's work safety and efficiency, ensures that good visual effects are maintained when the excavator arm moves, reduces accident risk, and improves the efficiency of the loading and unloading process.
Smart Images

Figure CN119316728A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image acquisition and processing, and in particular to an excavator monitoring system, method, excavator and product based on panoramic view. Background Art
[0002] With the advancement of science and technology, 360-degree panoramic vision systems have become standard equipment for many construction machinery vehicles, especially on heavy machinery such as excavators. Such systems capture the environment around the vehicle by installing multiple cameras around the vehicle, and then stitch these video streams into a complete surround image through software algorithms, thereby helping the driver to obtain a more comprehensive field of view, reduce blind spots, and improve the safety and efficiency of operations. However, in actual operation, the special design of the excavator brings some challenges. The excavator's digging arm (digging arm) and bucket move frequently during operation, especially when they extend forward, which may block the camera installed in front of the vehicle, causing some areas to be unable to be captured, thus forming a visual blind spot. The existence of such a blind spot not only increases the risk of accidents, but may also affect the accuracy and efficiency of the excavator's operation. Summary of the invention
[0003] The embodiments of the present invention provide an excavator monitoring system, method, excavator and product based on panoramic surround vision to solve the blind spot problem existing in the traditional surround vision system during excavator operation, and further improve the safety and efficiency of the excavator operation.
[0004] The excavator includes an excavator body, a cockpit arranged on the body, and an excavating arm arranged at the front side of the body and capable of swinging up and down. In order to achieve the above-mentioned purpose, on the one hand, a panoramic view-based excavator monitoring system is provided, which includes:
[0005] A plurality of first cameras disposed on the cab, each of the plurality of first cameras being used to capture images of different areas around the excavator;
[0006] a second camera disposed at the front side of the body and below the digging arm, the second camera being used to capture an image below the digging arm;
[0007] Controller, including:
[0008] A camera data acquisition module, used for acquiring images captured by the plurality of first cameras and the second camera;
[0009] A 360-degree panoramic view module, used to calibrate the internal and external parameters of the multiple first cameras, perform 3D stereoscopic space modeling, and stitch multiple images taken by the multiple first cameras into a 360-degree panoramic view in the 3D stereoscopic space;
[0010] An image fusion module is used to calibrate the internal and external parameters of the second camera and fuse the image captured by the second camera into the 360-degree panoramic image, so that in the fused 360-degree panoramic image, the image captured by the second camera is used to display the area blocked by the excavator arm in the image captured by the first camera in a perspective manner.
[0011] Preferably, the excavator monitoring system, wherein the multiple first cameras include: a front camera installed on the front part of the top of the cab, a rear camera installed on the rear part of the top of the cab, a left camera installed on the left part of the top of the cab, and a right camera installed on the right part of the top of the cab.
[0012] Preferably, in the excavator monitoring system, the 360-degree panoramic module is further used for:
[0013] Determining first world coordinates of a plurality of images captured by each first camera in a predefined world coordinate system according to the internal and external parameters of the plurality of first cameras;
[0014] According to the first world coordinate of each first camera, the spatially overlapping areas of the multiple images taken by the multiple first cameras are aligned, the overlapping areas are determined as the fusion areas of the multiple first cameras, the fusion areas are regionally fused, and a first mapping table is created; wherein the first mapping table is used to describe the mapping relationship between the physical points in the world coordinate system and the pixel points in the multiple images taken by each first camera.
[0015] Preferably, in the excavator monitoring system, the step of regionally fusing the fusion area comprises: for each overlapping area in the overlapping areas, respectively calculating a plurality of grayscale values of a plurality of images forming the overlapping area, calculating a first mean value of the plurality of grayscale values, and adjusting the grayscale value of the overlapping area to the first mean value;
[0016] A corresponding stitching line is selected for each overlapping area, and images acquired by the multiple first cameras are stitched.
[0017] Preferably, in the excavator monitoring system, the image fusion module is further used for:
[0018] Determine the position of the second camera in the 3D stereoscopic model according to the internal and external parameters of the second camera, and obtain the second world coordinates of the image captured by the second camera in the predefined world coordinate system;
[0019] Creating a second mapping table according to the second world coordinate system, wherein the second mapping table is used to describe a mapping relationship between a physical point position in the world coordinate system and a pixel point position in the image data captured by the second camera;
[0020] For an area blocked by the digging arm in the image captured by the first camera, converting a corresponding first image pixel position in the 360-degree panoramic image into a first physical point position in the world coordinate system using the first mapping table;
[0021] The second mapping table is used to determine the second image pixel position corresponding to the first physical point in the image data captured by the second camera, and the image at the second image pixel position in the image captured by the second camera is fused into the 360-degree panoramic image.
[0022] Preferably, in the excavator monitoring system, the area blocked by the excavator arm in the image captured by the first camera is determined by the following steps:
[0023] Acquire the swing amplitude value of the digging arm through the CAN bus of the excavator body;
[0024] The first position of the digging arm in the image taken by the front camera is calculated by the swing amplitude value, and the first position is the area blocked by the digging arm in the image taken by the first camera.
[0025] Preferably, the excavator monitoring system further includes: calculating the first grayscale value and the second grayscale value of the image taken by the second camera and the image taken by the front camera in the overlapping area, performing mean calculation on the first grayscale value and the second grayscale value to obtain a third grayscale value, and adjusting the first grayscale value and the second grayscale value to be greater than the third grayscale value.
[0026] On the other hand, an excavator comprises any excavator monitoring system as described above.
[0027] On the other hand, a method for monitoring an excavator using any of the above-described excavator monitoring systems based on panoramic view, wherein the excavator comprises an excavator body, a cockpit disposed on the body, and an excavating arm disposed at the front side of the body and capable of swinging up and down, characterized in that it comprises:
[0028] Acquire images captured by the plurality of first cameras and the second camera;
[0029] Calibrate the internal and external parameters of the multiple first cameras, perform 3D stereoscopic space modeling, and stitch the multiple images captured by the multiple first cameras into a 360-degree panoramic view in the 3D stereoscopic space, so that in the fused 360-degree panoramic view, the image captured by the second camera is used to display the area blocked by the digging arm in the image captured by the first camera in a perspective manner;
[0030] The internal and external parameters of the second camera are calibrated, and the image captured by the second camera is fused into the 360-degree panoramic image.
[0031] In yet another aspect, a computer program product comprises a computer program, wherein when the computer program is executed by a processor, the steps in the method for monitoring an excavator as described above are implemented.
[0032] The above technical solution has the following technical effects:
[0033] The system of the embodiment of the present invention collects images taken by multiple first cameras and second cameras through the camera data acquisition module; then calibrates the internal and external parameters of multiple first cameras through the 360-degree panoramic module, performs 3D stereoscopic space modeling, and splices multiple images taken by multiple first cameras into a 360-degree panoramic view in the 3D stereoscopic space; then calibrates the internal and external parameters of the second camera through the image fusion module, and fuses the image taken by the second camera into the 360-degree panoramic view, so that in the fused 360-degree panoramic view, the image taken by the second camera is used to display the area blocked by the excavator arm in the image taken by the first camera in a perspective manner. It effectively solves the problem of visual blind spots when the excavator is operating, and improves the work safety and work efficiency of the excavator driver.
[0034] In a further embodiment, seamless stitching and fusion of images are achieved by adjusting the grayscale value of the overlapping area of the images, providing a high-quality 360-degree panoramic image; the swing amplitude value of the excavator arm is obtained through the CAN bus of the excavator body, and the fusion area of the image data acquired by the first camera and the second camera is determined according to the swing amplitude value, and the image fusion of the fusion area is achieved according to the conversion of the world coordinates in the same world coordinate system and the pixel positions of the images taken by the first camera and the second camera, ensuring that good visual effects can be maintained when the excavator arm moves, the driver can better monitor the surrounding environment, avoid accidents, and work more efficiently during loading and unloading. BRIEF DESCRIPTION OF THE DRAWINGS
[0035] Figure 1 This is a schematic structural diagram of an excavator monitoring system based on panoramic view according to an embodiment of the present invention;
[0036] Figure 2 A schematic diagram of the installation positions of a surround view camera and a perspective camera in an excavator monitoring system based on panoramic surround view according to an embodiment of the present invention;
[0037] Figure 3 The figure is a schematic diagram of the output interface of an excavator monitoring system based on panoramic view according to an embodiment of the present invention. DETAILED DESCRIPTION
[0038] To further illustrate the various embodiments, the present invention provides drawings. These drawings are part of the disclosure of the present invention, which are mainly used to illustrate the embodiments and can be used in conjunction with the relevant descriptions in the specification to explain the operating principles of the embodiments. With reference to these contents, a person of ordinary skill in the art should be able to understand other possible implementations and advantages of the present invention. The components in the figures are not drawn to scale, and similar component symbols are generally used to represent similar components.
[0039] The present invention will now be further described with reference to the accompanying drawings and specific implementation methods.
[0040] Embodiment 1:
[0041] In order to solve the problem of visual blind spots when the excavator is operating and improve the working safety and work efficiency of the excavator driver, this embodiment provides an excavator monitoring system based on panoramic view. The excavator of this embodiment includes an excavator body, a cockpit arranged on the body, and an excavating arm arranged on the front side of the body and capable of swinging up and down. Figure 1 FIG. 1 is a schematic diagram of the structure of an excavator monitoring system based on panoramic view according to an embodiment of the present invention. Figure 1 As shown, the system includes:
[0042] A plurality of first cameras are arranged on the cab, each of the plurality of first cameras being used to capture images of different areas around the excavator; in a specific implementation, the plurality of first cameras include: a front camera installed on the front portion of the cab top, a rear camera installed on the rear portion of the cab top, a left camera installed on the left portion of the cab top, and a right camera installed on the right portion of the cab top;
[0043] A second camera is arranged on the front side of the fuselage and below the digging arm, and the second camera is used to capture images below the digging arm; preferably, Figure 2 FIG. 1 is a schematic diagram of the installation positions of the surround view camera and the perspective camera in an excavator monitoring system based on panoramic view according to an embodiment of the present invention; Figure 2 As shown, the second camera is a perspective camera; preferably, the first camera is a fisheye camera, and the second camera is a non-fisheye camera;
[0044] Controller, including:
[0045] A camera data acquisition module, used for acquiring images taken by a plurality of first cameras and a second camera;
[0046] The 360-degree panoramic module is used to calibrate the internal and external parameters of multiple first cameras, perform 3D stereoscopic space modeling, and stitch multiple images taken by multiple first cameras into a 360-degree panoramic view in the 3D stereoscopic space.
[0047] Preferably, based on the internal and external parameters of the multiple first cameras, the first world coordinates of the multiple images taken by each first camera in a predefined world coordinate system are determined; based on the first world coordinates of each first camera, the spatially overlapping areas of the multiple images taken by the multiple first cameras are aligned, the overlapping areas are determined as fusion areas of the multiple first cameras, the fusion areas are regionally fused, and a first mapping table is created; wherein the first mapping table is used to describe the mapping relationship between the physical points in the world coordinate system and the pixel points in the multiple images taken by each first camera.
[0048] Furthermore, the step of regional fusion of the fusion area includes: for each overlapping area in the overlapping areas, respectively calculating multiple grayscale values of multiple images forming the overlapping area, calculating a first mean value of the multiple grayscale values, and adjusting the grayscale value of the overlapping area to the first mean value to achieve grayscale consistency of the overlapping area; for each overlapping area, selecting a corresponding stitching line, determining the position of the stitching seam, and using a weighted average method to transition the overlapping area, stitching the images acquired by the multiple first cameras to achieve a seamless stitching effect; and
[0049] The image fusion module is used to calibrate the internal and external parameters of the second camera and fuse the image taken by the second camera into the 360-degree panoramic view, so that in the fused 360-degree panoramic view, the image taken by the second camera is used to display the area blocked by the excavator arm in the image taken by the first camera in a perspective manner.
[0050] Preferably, the second camera and multiple first cameras are in the same world coordinate system, the front camera and the second camera are installed in front of the excavator, the front camera is on the front side of the excavator cab, and the second camera is below the excavator arm. The second camera and the front camera occupy the same spatial area in the world coordinate system and will not be blocked by the excavator arm at the same time.
[0051] Furthermore, based on the internal and external parameters of the second camera, the position of the second camera in the 3D stereoscopic space model is determined to obtain the second world coordinates of the image taken by the second camera in a predefined world coordinate system; based on the second world coordinates, a second mapping table is created, wherein the second mapping table is used to describe the mapping relationship between the physical points in the world coordinate system and the pixel points in the image data taken by the second camera.
[0052] Furthermore, since the front camera is blocked by the excavator arm, it is necessary to use the unblocked part of the second camera. This part needs to first calculate the image area blocked by the excavator arm in the front camera image, and then convert the image pixel position of the area into a physical position in the world coordinates, and then convert the world coordinates of the physical position to the image position in the perspective camera, and take the image of the position and merge it into the projection area of the original front camera to form a perspective effect. Because the swing amplitude value of the excavator arm is constantly changing during the swinging process, different swing amplitude values correspond to different areas blocked by the excavator arm, that is, there is a certain proportional relationship between the swing amplitude value and the area blocked by the excavator arm, so the area blocked by the excavator arm can be calculated according to the proportional relationship. The embodiment of the present invention obtains the swing amplitude value of the excavator arm through the CAN bus of the excavator body; calculates the first position of the excavator arm in the image taken by the front camera through the swing amplitude value, wherein the first position is the area blocked by the excavator arm in the image taken by the first camera. For the area blocked by the excavator arm in the image captured by the first camera, the first mapping table is used to convert the corresponding first image pixel position in the 360-degree panoramic image into the first physical point in the world coordinate system; the second mapping table is used to determine the second image pixel position corresponding to the first physical point in the image data captured by the second camera, and the image at the second image pixel position in the image captured by the second camera is merged into the 360-degree panoramic image. Preferably, the swing amplitude value of the excavator arm corresponds to the image position of the excavator arm in the images captured by the front camera and the second camera, and the image position of the excavator arm in the image captured by the front camera can be calculated by the swing amplitude value.
[0053] In a specific implementation, the first grayscale value and the second grayscale value of the overlapping area of the image taken by the second camera and the image taken by the front camera are calculated, the first grayscale value and the second grayscale value are averaged to obtain the third grayscale value, and the first grayscale value and the second grayscale value are adjusted to be greater than the third grayscale value, so that the fusion area of the second camera is displayed semi-transparently, and the semi-transparently displayed area is a part of the 360-degree panoramic view, thereby realizing the perspective display of the image of the area under the excavator arm in the fused 360-degree panoramic view; that is, for the area blocked by the excavator arm in the image taken by the first camera, the perspective display of the area is realized by displaying the image of the area taken by the second camera, thereby improving the security of monitoring.
[0054] In a specific implementation, the intrinsic parameter calibration of multiple first cameras and second cameras only needs to use the multiple first cameras to be calibrated and the second cameras to be calibrated to collect multiple pictures of the calibration template at different angles, and then use the LM algorithm to iteratively solve the multiple pictures after projecting the initial values to obtain the final intrinsic parameter values of the multiple first cameras and the second cameras.
[0055] In a specific implementation, the external parameters of the multiple first cameras and the second cameras are used to describe the positions and directions of the multiple first cameras and the second cameras relative to the world coordinate system. When the multiple first cameras and the second cameras take each image, the positions and directions of the images taken by the multiple first cameras and the second cameras in the world coordinate system are different. Therefore, each image has a set of external parameter values. Without considering the influence of distortion, a linear method is used to solve the initial values of the external parameters of the multiple first cameras and the second cameras. Then a nonlinear optimization method (LM, Levenberg-Marquard algorithm) is used to obtain relatively accurate external parameter values. Among them, in order to associate the images taken by the multiple first cameras and the second cameras, the world coordinates in the same world coordinate system must be used, and the images taken by all cameras must be converted to the same world coordinates.
[0056] In a specific implementation, the 360-degree panoramic view includes a 360-degree panoramic top view and a 3D rear view; with the car model as the center, the 360-degree panoramic top view and the 3D rear view are spliced. The 360-degree panoramic top view and the 3D rear view are model-rendered according to the first mapping table and the second mapping table to establish a three-dimensional projection surface, so that the images captured by the multiple first cameras and the second camera are mapped onto the projection surface to form a stereoscopic image. In a specific implementation, the perspective is switched to the 3D rear view by adjusting the position of the virtual perspective.
[0057] In a specific implementation, the images taken by the first camera and the second camera are perspectively projected in the 3D stereo space according to the internal and external parameter information of the first camera and the second camera in combination with a pre-selected image stitching technology, and the mapping relationship between the image pixels of the images taken by the first camera and the second camera corresponding to the world coordinates is calculated, so as to obtain the positions of the images taken by the first camera and the second camera in the world coordinate system, and further obtain the positions of the first camera and the second camera in the 3D stereo space.
[0058] Figure 3 FIG. 1 is a schematic diagram of an output interface of an excavator monitoring system based on panoramic view according to an embodiment of the present invention. Figure 3 As shown, the area on the left side of the figure is the 360-degree panoramic 2D car model display area; the right side of the figure is the excavator bucket perspective area and the 360-degree panoramic 3D car model display area, wherein the excavator bucket perspective area is the image of the perspective area of the excavator digging arm, that is, the image of the area blocked by the digging arm in the image taken by the front camera and which can be captured by the second camera.
[0059] Through the panoramic view-based excavator monitoring system of this embodiment, the excavator driver can easily check the visual blind spots in front of the bucket, prevent accidents at all times, reduce the dangers caused by visual blind spots, and further significantly improve the driver's work efficiency during loading and unloading.
[0060] Embodiment 2:
[0061] The present invention also provides an excavator comprising any of the excavator monitoring systems described above.
[0062] Embodiment three:
[0063] The present invention also provides a method for monitoring an excavator using any of the above-described excavator monitoring systems based on panoramic view, wherein the excavator comprises an excavator body, a cockpit disposed on the body, and an excavating arm disposed at the front side of the body and capable of swinging up and down. The method of this embodiment comprises:
[0064] Collecting images captured by a plurality of first cameras and a second camera;
[0065] Calibrate the internal and external parameters of the multiple first cameras, perform 3D stereoscopic space modeling, and stitch the multiple images taken by the multiple first cameras into a 360-degree panoramic image in the 3D stereoscopic space;
[0066] The internal and external parameters of the second camera are calibrated, and the image captured by the second camera is fused into the 360-degree panoramic image, so that in the fused 360-degree panoramic image, the image captured by the second camera is used to display the area blocked by the excavator arm in the image captured by the first camera in a perspective manner.
[0067] In a specific implementation, the specific steps of stitching into a 360-degree panoramic view refer to the description of the 360-degree panoramic module in the monitoring system in the previous text; the specific steps of fusing the image taken by the second camera into the 360-degree panoramic view refer to the description of the image fusion module in the monitoring system in the previous text.
[0068] Embodiment 4:
[0069] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the steps of the method for monitoring an excavator as described above are implemented.
[0070] Although the present invention has been specifically shown and described in conjunction with the preferred embodiments, it should be understood by those skilled in the art that various changes may be made to the present invention in form and details without departing from the spirit and scope of the present invention as defined by the appended claims, all of which are within the scope of protection of the present invention.
Claims
1. An excavator monitoring system based on panoramic view, the excavator comprising an excavator body, a cockpit arranged on the body, and an excavating arm arranged at the front side of the body and capable of swinging up and down, characterized in that: include: A plurality of first cameras disposed on the cab, each of the plurality of first cameras being used to capture images of different areas around the excavator; a second camera disposed at the front side of the body and below the digging arm, the second camera being used to capture an image below the digging arm; Controller, including: A camera data acquisition module, used for acquiring images captured by the plurality of first cameras and the second camera; A 360-degree panoramic view module, used to calibrate the internal and external parameters of the multiple first cameras, perform 3D stereoscopic space modeling, and stitch multiple images taken by the multiple first cameras into a 360-degree panoramic view in the 3D stereoscopic space; An image fusion module is used to calibrate the internal and external parameters of the second camera and fuse the image captured by the second camera into the 360-degree panoramic image, so that in the fused 360-degree panoramic image, the image captured by the second camera is used to display the area blocked by the excavator arm in the image captured by the first camera in a perspective manner.
2. The excavator monitoring system according to claim 1, characterized in that: The multiple first cameras include: a front camera installed on the front part of the cockpit top, a rear camera installed on the rear part of the cockpit top, a left camera installed on the left part of the cockpit top, and a right camera installed on the right part of the cockpit top.
3. The excavator monitoring system according to claim 2, characterized in that: The 360-degree panoramic view module is further used for: Determining first world coordinates of a plurality of images captured by each first camera in a predefined world coordinate system according to the internal and external parameters of the plurality of first cameras; According to the first world coordinate of each first camera, the spatially overlapping areas of the multiple images taken by the multiple first cameras are aligned, the overlapping areas are determined as the fusion areas of the multiple first cameras, the fusion areas are regionally fused, and a first mapping table is created; wherein the first mapping table is used to describe the mapping relationship between the physical points in the world coordinate system and the pixel points in the multiple images taken by each first camera.
4. The excavator monitoring system according to claim 3, characterized in that: The step of performing regional fusion on the fusion area comprises: for each overlapping area in the overlapping areas, respectively calculating a plurality of grayscale values of a plurality of images forming the overlapping area, calculating a first mean value of the plurality of grayscale values, and adjusting the grayscale value of the overlapping area to the first mean value; A corresponding stitching line is selected for each overlapping area, and images acquired by the multiple first cameras are stitched.
5. The excavator monitoring system according to claim 3, characterized in that: The image fusion module is further used for: Determine the position of the second camera in the 3D stereoscopic model according to the internal and external parameters of the second camera, and obtain the second world coordinates of the image captured by the second camera in the predefined world coordinate system; Creating a second mapping table according to the second world coordinate system, wherein the second mapping table is used to describe a mapping relationship between a physical point position in the world coordinate system and a pixel point position in the image data captured by the second camera; For an area blocked by the digging arm in the image captured by the first camera, converting a corresponding first image pixel position in the 360-degree panoramic image into a first physical point position in the world coordinate system using the first mapping table; The second mapping table is used to determine the second image pixel position corresponding to the first physical point in the image data captured by the second camera, and the image at the second image pixel position in the image captured by the second camera is fused into the 360-degree panoramic image.
6. The excavator monitoring system according to claim 5, characterized in that: The area blocked by the digging arm in the image captured by the first camera is determined by the following steps: Acquire the swing amplitude value of the digging arm through the CAN bus of the excavator body; The first position of the digging arm in the image taken by the front camera is calculated by the swing amplitude value, and the first position is the area blocked by the digging arm in the image taken by the first camera.
7. The excavator monitoring system according to claim 6, characterized in that: Also includes: Calculate the first grayscale value and the second grayscale value of the image captured by the second camera and the image captured by the front camera in the overlapping area, perform mean calculation on the first grayscale value and the second grayscale value to obtain a third grayscale value, and adjust the first grayscale value and the second grayscale value to be greater than the third grayscale value.
8. An excavator, characterized in that: It comprises an excavator monitoring system as described in any one of claims 1 to 7.
9. A method for monitoring an excavator using the excavator monitoring system based on panoramic view as claimed in any one of claims 1 to 7, wherein the excavator comprises an excavator body, a cockpit arranged on the body, and an excavating arm arranged at the front side of the body and capable of swinging up and down, characterized in that: include: Acquire images captured by the plurality of first cameras and the second camera; Calibrate the internal and external parameters of the multiple first cameras, perform 3D stereoscopic space modeling, and stitch the multiple images taken by the multiple first cameras into a 360-degree panoramic view in the 3D stereoscopic space; Calibrate the internal and external parameters of the second camera, and fuse the image captured by the second camera into the 360-degree panoramic image, so that in the fused 360-degree panoramic image, the image captured by the second camera is used to display the area blocked by the excavator arm in the image captured by the first camera in a perspective manner.
10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps in the method for monitoring an excavator according to claim 9 are implemented.