Surround view monitoring method for a work machine and the work machine

Abstract

The present disclosure relates to a surround view monitoring method for a work machine and the corresponding work machine. The work machine comprises a working mechanism extending outward from a vehicle body of the work machine. A first mechanism camera and a second mechanism camera that are capable of capturing at least a portion of the working mechanism, and at least one remaining camera except the first and second mechanism cameras are installed on and around the vehicle body. The method comprises the steps of: acquiring respective images by the cameras; generating a first panoramic image by using image acquired by the first mechanism camera and image acquired by the remaining camera, and generating a second panoramic image by using image acquired by the second mechanism camera and image acquired by the remaining camera; determining a first area occupied by the working mechanism in the first panoramic image and a second area occupied by the working mechanism in the second panoramic image; and comparing the first area with the second area and outputting the panoramic image corresponding to a smaller one of the first and second areas as a panoramic image of the work machine. The surround view monitoring method always ensures a smallest blind area resulting from blocking by the working mechanism to improve the driving and operating safety of the work machine.

Description

[0001] Description

[0002] SURROUND VIEW MONITORING METHOD FOR A WORK MACHINE AND THE WORK MACHINE

[0003] Technical Field

[0004] The present disclosure relates to the field of visual / vision control of a work machine, more particularly, to a surround view monitoring method and system for a work machine.

[0005] Background Art

[0006] The surround view (also called 360 panoramic view) monitoring technology is to shoot images through multiple cameras (generally wide-angle fisheye cameras) installed around a vehicle, and subject the images to stitching and composing process with special algorithms, thereby forming a complete set of seamlessly spliced real-time images with a panoramic view and an ultra-wide viewing angle, and displaying them on an on-board screen, so that the driver can clearly see whether there are pedestrians, moving objects, obstacles, etc. around the vehicle with a wider field of view and know their directions and distances relative to the vehicle. Being intuitive and not containing a blind area, the surround view monitoring technology can help the driver park the vehicle or pass through complex roads with ease, thus effectively reducing accidents such as scraping, collision and collapse.

[0007] The surround view monitoring technology is used widely in automobiles, and has also been applied to work machines or work vehicles in recent years, in particular for assisting drivers in construction operations to improve work safety. The existing surround view monitoring system used in work machines (such as excavators and loaders) generally includes four cameras installed at the front, back, left and right parts of the vehicle body of the work machine. However, unlike a common automobile, the work machine often includes a working mechanism or an implement extending outward from the vehicle body (for example, an excavator includes a boom, a stick and a bucket extending outward), and the working mechanism is generally disposed between the front camera and the right or left camera. Therefore, no matter how the installation positions of these cameras are adjusted, the working mechanism will always block the views of the cameras at least to some extent when it moves. Such blocking causes blind areas in the generated panoramic images of the work machine, which might adversely affect the driver's judgment on the surroundings of the work machine, resulting in danger.

[0008] Summary of the Invention

[0009] The present disclosure aims to solve the above problems and / or other defects of the existing surround view monitoring system for a work machine.

[0010] According to an aspect of the present disclosure, there is provided a surround view monitoring method for a work machine. The work machine comprises a working mechanism extending outward from a vehicle body of the work machine. A first mechanism camera, a second mechanism camera, and at least one remaining camera except the first and second mechanism cameras are installed on and around the vehicle body. The first mechanism camera and second mechanism camera are capable of capturing at least a portion of the working mechanism, and the at least one remaining camera cannot catch the working mechanism or can capture a less range of the working mechanism than the first and second mechanism cameras. The method comprises the steps of: acquiring respective images by the first and second mechanism cameras and the remaining camera; generating a first panoramic image by using image acquired by the first mechanism camera and image acquired by the remaining camera, and generating a second panoramic image by using image acquired by the second mechanism camera and image acquired by the remaining camera; determining a first area occupied by the working mechanism in the first panoramic image and a second area occupied by the working mechanism in the second panoramic image; and comparing the first area with the second area and outputting the panoramic image corresponding to a smaller one of the first and second areas as a panoramic image of the work machine.

[0011] According to an exemplary embodiment, the step of determining the first area and the second area comprises: selecting a predetermined region containing the working mechanism in the first or second panoramic image, conducting edge detection of the working mechanism in the predetermined region to delimit a contour of the working mechanism, and calculating an area closed by the contour as the first or second area.

[0012] According to an exemplary embodiment, the first panoramic image and the second panoramic image are images of two-dimensional bird's eye view.

[0013] Another aspect of the present disclosure provides a computer- readable storage medium having a computer program stored therein, the computer program including executable instructions which, when executed by a processor, implement the surround view monitoring method according to any of above configurations.

[0014] A further aspect of the present disclosure provides an electronic device. The electronic device comprises: a processor; and a memory for storing executable instructions of the processor, wherein the processor is configured to execute the executable instructions to implement the surround view monitoring method according to any of above configurations.

[0015] A still further aspect of the present disclosure provides a work machine. The work machine comprises: a working mechanism extending outward from a vehicle body of the work machine; a plurality of cameras installed on and around the vehicle body, the cameras comprising a first mechanism camera, a second mechanism camera, and at least one remaining camera except the first and second mechanism cameras, wherein the first and second mechanism cameras are capable of capturing at least a portion of the working mechanism, and the at least one remaining camera cannot catch the working mechanism or can capture a less range of the working mechanism than the first and second mechanism cameras; and an electronic device as stated above, which is in communicative connection with the plurality of cameras.

[0016] According to an exemplary embodiment, the first mechanism camera and the second mechanism camera are installed on the vehicle body at opposite sides of a vertical plane where the working mechanism is located.

[0017] According to an exemplary embodiment, the vehicle body comprises a cab and a cabin assembly, wherein the cab and a front compartment of the cabin assembly are disposed at opposite sides of the vertical plane where the working mechanism is located, and the first mechanism camera and the second mechanism camera are installed at a front top of the cab and a front top of the front compartment of the cabin assembly, respectively.

[0018] According to an exemplary embodiment, the remaining camera includes a left camera installed at a left top of the cab, and a right camera and a rear camera installed at a right top and rear top of the cabin assembly respectively.

[0019] According to an exemplary embodiment, the working mechanism comprises one or more of a boom, a stick and a working implement of the work machine.

[0020] Compared with the existing surround view monitoring system for a work machine, the surround view monitoring system according to the present disclosure, in terms of hardware, is provided on the vehicle body of the work machine with an additional camera capable of capturing images of the working mechanism of the work machine (referred to as "mechanism camera" in the present application), that is, a total of two mechanism cameras are provided. In terms of software, the additional mechanism camera (the "second mechanism camera") is complementary to the original one (the "first mechanism camera"), and the two mechanism cameras are used alternately; in other words, if it is determined that the blind area resulting from blocking by the working mechanism in a first panoramic image generated by using the first mechanism camera and the remaining cameras is smaller than that in a second panoramic image generated by using the second mechanism camera and the remaining cameras, the first panoramic image is used as the panoramic image of the work machine to be displayed on e.g. the on-board display device of the work machine. On the contrary, if it is determined that the blind area of the second mechanism camera is smaller than that of the first mechanism camera, the second panoramic image will be displayed as the panoramic image of the work machine. The selection or switching between the panoramic images respectively generated by the two mechanism cameras can be carried out continuously in real time during movement of the working mechanism, so that panoramic image with the least blind area resulting from blocking by the working mechanism can be obtained at every moment and every position of the working mechanism, thereby reducing the driver's negligence or misjudgment on the surroundings of the work machine due to the blind area in the images, and thus improving the driving and operating safety of the work machine.

[0021] Brief Description of the Drawings

[0022] Features and advantages of the present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with the non-limiting examples. The accompanying drawings are only schematic and not necessarily drawn to scale. In addition, only the portions necessary for clarifying the present disclosure are shown, while other portions may be omitted or simply mentioned. That is, the present disclosure may include other elements or parts in addition to those shown in the drawings. In the drawings:

[0023] Figs. 1 to 3 are perspective views of a work machine according to an embodiment of the present disclosure as seen from three different angles of view;

[0024] Fig. 4 is an exemplary panoramic image of the work machine shown in Figs. 1 to 3;

[0025] Fig. 5 is a flowchart of a surround view monitoring method according to an embodiment of the present disclosure; Fig. 6 is a schematic explanatory diagram of the step S130 for determining the area blocked by the working mechanism in the panoramic images in the method shown in Fig. 5; and

[0026] Fig. 7 is a schematic structural diagram of a surround view monitoring system according to an embodiment of the present disclosure.

[0027] Detailed Description of the Embodiments

[0028] Exemplary embodiments according to the present disclosure will be described in detail below with reference to the accompanying drawings. In the following description, numerous specific details are set forth in order to make those skilled in the art fully understand the present disclosure. However, it is apparent to those skilled in the art that the present disclosure may be implemented without some of those details. Moreover, it will be appreciated that the present disclosure is not limited to the specific embodiments described herein. On the contrary, any combination of features and elements described herein can be used for implementing the present disclosure, regardless of whether they relate to different embodiments or not. Therefore, the following aspects, features, embodiments and advantages are for illustration only and should not be regarded as elements or limitations of the claims, unless otherwise explicitly stated in the claims.

[0029] Figs. 1 to 3 show an exemplary work machine 100 to which a surround view monitoring system and method according to an embodiment of the present disclosure are applied. The work machine 100 is exemplified as a crawler excavator in this embodiment. The work machine 100 according to the present disclosure may include a vehicle body and a working mechanism (or called “working device”) 30 extending outward from the vehicle body. The working mechanism 30 can typically include a boom (or called “big arm”) 31, a stick (or called “small arm”) 32 and a working implement 33 (which is a bucket for an excavator, and can also be another implement for performing other tasks) that are connected with one another, and particularly includes the boom closest to the vehicle body. The vehicle body may include a main frame, and a cab 10 and a cabin assembly 20 supported on the main frame. The main frame of the excavator can be constructed, for example, as a revolving platform, which can carry the working mechanism 30, the cab 10 and the cabin assembly 20 to revolve as a whole, so that they can work in a sufficiently wide orientation range. The cabin assembly 20 can be formed by combining multiple compartments or cabins, such as an engine compartment for receiving the engine system of the work machine, a hydraulic compartment for accommodating a hydraulic system for actuating the working mechanism 30, and a fuel tank for containing fuel. A traveling device 40 (a crawler / track assembly in this embodiment) for supporting and moving the work machine 100 is arranged below the vehicle body.

[0030] The directional or positional terms such as "front", "rear", "left", "right", "up", "down", "top", "bottom", "vertical" and "horizontal" mentioned in the present application are described in relation to the vehicle body of the work machine 100, or in the view angle of the driver sitting normally in the cab. In the coordinate system shown in Figs. 1 to 3, F represents the forward direction, L represents the left direction and U represents the upward direction. Generally speaking, the working mechanism 30 extends forward from the vehicle body, is located in a vertical plane (FU plane), can swing up and down in the vertical plane, and can also pivot along with the revolving of the vehicle body in a horizontal plane (FL plane).

[0031] A plurality of cameras may be installed on and around the vehicle body of the work machine 100 to generate a panoramic image thereof. In the present application, the term "camera" should be understood broadly as any image acquisition device that can capture static or dynamic pictures / images within a certain viewing angle range, and generally, for example, a wide-angle fisheye camera can be adopted. Images captured by the respective cameras can be composed into a 360 panoramic image (hereinafter referred to as "panoramic image" for short) around the work machine 100 through stitching process. The technology of stitching / splicing images of multiple cameras into a panoramic image is known in the art, generally including camera calibration, distortion correction, panoramic model stitching, etc., and will not be described in detail herein. As an exemplary arrangement for the plurality of cameras, the work machine 100 may include a front camera 51, a left camera 52, a right camera 53, and a rear camera 54 that are schematically shown by blank triangles in Figs. 1 to 3. The front camera 51 is installed, for example, on the front top of the cab 10 (e.g., at a position on the front surface of the cab near the midpoint of the top edge thereof) to mainly photograph the front area of the vehicle body; the left camera 52 is installed, for example, on the left top of the cab 10 (e.g., at a position on the left surface of the cab near the midpoint of the top edge thereof) to mainly photograph the left area of the vehicle body; the right camera 53 is installed, for example, on the right top of the cabin assembly 20 (e.g., at a position on the right surface of the cabin assembly near the midpoint of the top edge thereof) to mainly photograph the right area of the vehicle body; and the rear camera 54 is installed, for example, on the rear top of the cabin assembly 20 (e.g. at a position on the rear surface of the cabin assembly near the midpoint of the top edge thereof) to mainly photograph the rear area of the vehicle body. Among the above four cameras, the front camera 51 captures the largest range of the working mechanism 30 as the working mechanism is located in the front area of the vehicle body, while the other three cameras either cannot catch the working mechanism 30 (for the rear camera, or for the left or right camera with a normal or small viewing angle) or only capture a small range of the working mechanism (for the left or right camera with a very large viewing angle).

[0032] The arrangement of the above four cameras is also a typical configuration of the existing surround view monitoring system of a work machine, and an exemplary panoramic image generated by the system is shown in Fig. 4 (it should be pointed out that Fig. 4 shows a gray image, but the panoramic image can actually be a color image reflecting the real world). The left part in Fig. 4 is an image of two-dimensional bird's-eye view (plan view), and the right part in Fig. 4 is an image of three-dimensional view. Because the working mechanism 30 of the work machine 100 is located between the front camera 51 and the right camera 53, no matter how the installation positions of these cameras are adjusted, the working mechanism 30 will always block the views of these cameras at least to some extent when it moves. Such blocking will appear as blind areas in a panoramic image of the work machine (see the areas containing image portions blocked by the working mechanism, as enclosed by rectangular solid lines in the left and right parts of Fig. 4), which may adversely affect the driver's judgment on the surroundings of the work machine and thus cause driving or operating hazards.

[0033] As stated above, the blocking or blind area caused by the working mechanism may put the work machine at risk. In order to reduce the risk, a switching camera 55 which is complementary to and used alternately with the front camera 51 (the use method will be described in detail below) is added into the surround view monitoring system used by the work machine according to the present disclosure in addition to the original four cameras described above. Like the front camera 51, the switching camera 55 is used to mainly photograph the front area of the vehicle body, thereby also capturing a larger range of the working mechanism as compared with the other three cameras 52, 53 and 54. To this end, the switching camera 55 may also be installed at the front side of the vehicle body. For example, in the illustrated embodiment, a compartment or box of the cabin assembly 20 which is close to the front side of the vehicle body (referred to as "front compartment" of the cabin assembly in the present application) and the cab 10 are located at opposite sides of a vertical plane (i.e., the FU plane) where the working mechanism 30 is located, and the switching camera 55 can be installed on the front top of the front compartment of the cabin assembly 20. More specifically, the front compartment of the cabin assembly may include, for example, a diesel tank 21 and a tool box 22, and the switching camera 55 may be installed at a position on the front surface of the diesel tank 21 near the midpoint of its top edge as shown in Figs. 1 to 3. Of course, the installation position of the switching camera 55 is not limited thereto, but can be appropriately adjusted according to specific configuration of the vehicle body of the work machine, as long as the switching camera 55 can capture the image of at least a portion of the working mechanism 30 and can capture a larger range of the working mechanism as compared with the remaining cameras except the front camera 51. The installation position of the front camera 51 follows the same principle as the switching camera 55. Therefore, the front camera 51 and the switching camera 55 work the same way, and can be referred to as first mechanism camera and second mechanism camera respectively in the present application. It should be noted that "first" and "second" are used herein to differentiate the two mechanism cameras only in terms of their names for the convenience of description, and do not imply that the two mechanism cameras are different in sequence or importance.

[0034] Next, a surround view monitoring method according to an embodiment of the present disclosure will be described with reference to Fig. 5, which is a method of generating panoramic images of a work machine by using the surround view monitoring system of the present disclosure. As shown in Fig. 5, in one possible embodiment, the surround view monitoring method according to the present disclosure may include: a step SI 10 of capturing respective images with the respective cameras; a step S120 of generating a first panoramic image and a second panoramic image corresponding to the first mechanism camera and the second mechanism camera respectively; a step S130 of determining a first area and a second area occupied by the working mechanism respectively in the first panoramic image and the second panoramic image; and a step S140 of comparing the first area with the second area and using the panoramic image corresponding to the smaller area as the panoramic image of the work machine.

[0035] In step SI 10, a plurality of cameras (such as the cameras 51, 52, 53, 54, 55 described above) installed around the vehicle body of the work machine 100 operate to capture their respective images. More specifically, each camera can take pictures in real time to obtain instantaneous static images, or each camera can continuously record a dynamic video in real time and acquire instantaneous images from the dynamic video.

[0036] In step S120, a first panoramic image is generated by using images captured by the first mechanism camera (the front camera 51) and images captured by the remaining cameras (the left camera 52, right camera 53, and rear camera 54) except the first mechanism camera and the second mechanism camera (the switching camera 55), and a second panoramic image is generated by using images captured by the second mechanism camera and images captured by the remaining cameras. As mentioned above, the panoramic images can be generated by using image stitching process known in the art, which generally includes camera calibration, distortion correction, panoramic model stitching, etc. The first panoramic image and the second panoramic image can be generated simultaneously or sequentially in any order. Similar to the panoramic image shown in Fig. 4, in each of the generated first and second panoramic images, there is a blind area resulting from blocking by the working mechanism 30.

[0037] In step S130, a first area which is occupied by the working mechanism 30 in the first panoramic image generated in step S120 and a second area which is occupied by the working mechanism 30 in the second panoramic image generated in step S120 are determined. This step can be implemented, for example, by computer image processing and recognition technology, typically by edge detection. The edge detection will be described in detail with reference to Fig. 6. When step S130 is implemented by using edge detection technology, a predetermined region containing the working mechanism 30 may be selected in the first or second panoramic image. Now an image of two-dimensional bird's- eye view in the left part of Fig. 4 is taken as an example of the first or second panoramic image. A region containing the portion blocked by the working mechanism and being enclosed by the rectangular solid line is used as the predetermined region, which can be selected by delimiting a range with an estimated position and size in the first or second panoramic image, wherein the range with an estimated position and size must cover the working mechanism in the image. Part (a) of Fig. 6 shows such a predetermined region as selected, which contains the portion blocked by the working mechanism. Next, edge detection is performed for the working mechanism within the predetermined region (the color image actually shown in part (a) of Fig. 6 can be converted into a gray image before edge detection) to determine the contour of the working mechanism, as shown in part (b) of Fig. 6. The edge detection can be implemented using various edge detection algorithms known in the art. Finally, the area enclosed by the determined contour is calculated as the first or second area, as shown in part (c) of Fig. 6.

[0038] In step SI 40, the first area and the second area determined in step SI 30 are compared with each other, and the panoramic image corresponding to the smaller one of the first and second areas is output as the panoramic image of the work machine 100. In other words, if the first area is smaller than the second area, it indicates that the blind area resulting from blocking by the working mechanism 30 in the first panoramic image generated from the image of the first mechanism camera (the front camera 51) and the images of the remaining cameras is smaller than that in the second panoramic image generated from the image of the second mechanism camera (the switching camera 55) and the images of the remaining cameras, and the panoramic image generated from the image of the first mechanism camera will be selected. On the contrary, if the first area is larger than the second area, the panoramic image generated from the image of the second mechanism camera will be selected. This ensures that the panoramic image of the work machine 100 always contains the smallest blind area resulting from blocking by the working mechanism 30. The panoramic image of the work machine can be output by, for example, being displayed on the on-board display device of the work machine 100. Of course, the panoramic image can also be transmitted and applied to any other scenarios where it is needed.

[0039] The above steps SI 10 to SI 40 can be carried out cyclically and continuously in real time during the movement (such as lifting or lowering) of the working mechanism 30, and accordingly, the imaging mode of the panoramic image may be switched back and forth between the first and second mechanism cameras, so that a panoramic image always with the smallest blind area resulting from blocking by the working mechanism can be obtained at every moment and every position of the working mechanism, thereby reducing the driver's negligence or misjudgment of the surroundings of the work machine caused by the blind area in the image, and thus improving the driving and operating safety of the work machine. Fig. 7 is a schematic structural diagram of an exemplary configuration of a surround view monitoring system 200 according to the present disclosure. The system includes the plurality of cameras 51, 52, 53, 54, 55 installed on and around the vehicle body of the work machine 100, and an electronic device 400. The electronic device 400 may include a processor 410 and a memory 420 for storing executable instructions of the processor 410. The processor 410 is configured to carry out the steps of the surround view monitoring method according to any of the above-mentioned embodiments by executing the executable instructions. The electronic device 400 may be a general-purpose computing device, such as an on-board high-performance computer or controller of the work machine 100. The electronic device 400 is in communicative connection with the plurality of cameras, so as to control the cameras to acquire respective images and to receive the acquired images from the cameras. The surround view monitoring system 200 may further include an onboard display device 15 in communicative connection with the electronic device 400, and the electronic device 400 may output the panoramic images obtained by the above-mentioned method to the on-board display device 15 for display. Optionally, the electronic device 400 may communicate with one or more other devices inside and outside the work machine (such as various on-board sensors, vehicle networking devices, cloud servers, etc.) through various communication ports 300 to obtain any required information.

[0040] The present disclosure also provides a work machine, which includes a working mechanism extending outward from its vehicle body, the above-mentioned plurality of cameras installed on and around the vehicle body, and the above-mentioned electronic device 400 in communicative connection with the cameras.

[0041] The present disclosure also provides a computer-readable storage medium, on which a computer program is stored. The program includes executable instructions, which, when executed by, for example, a processor, can implement the steps of the surround view monitoring method according to any of the above embodiments. In some possible embodiments, the present disclosure can also be embodied, in each of the aspects, as a program product, which includes program codes causing a terminal device to implement the steps of the surround view monitoring method according to various exemplary embodiments of the present disclosure when the program product is run on the terminal device.

[0042] The program product for implementing the surround view monitoring method according to the embodiments of the present disclosure can be a portable compact disc read-only memory (CD-ROM) that includes program codes and can be run on a terminal device such as an on-board computer. However, the program product of the present disclosure is not limited thereto. The readable storage medium herein can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, apparatus or device.

[0043] The program product can be any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or apparatus, or any combination thereof. Specific examples (a non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable readonly memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0044] The computer-readable storage medium may include a data signal that propagates in baseband or as part of a carrier wave, which carries readable program codes. Such a data signal can take various forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination thereof. The readable storage medium can also be any readable medium other than the readable storage medium, which can send, propagate or transmit a program used by or in combination with the instruction execution system, apparatus or device. The program code contained in the readable storage medium can be transmitted by any suitable medium, including but not limited to wireless or wired connection, optical cable, RF, etc., or any suitable combination thereof.

[0045] Program codes for performing the operations of the present application can be written in any combination of one or more programming languages, including object-oriented programming languages, such as Java and C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be completely executed on a user computing device, partially executed on a user device, executed as an independent software package, partially executed on a user computing device and partially executed on a remote computing device, or completely executed on a remote computing device or server. In a case involving a remote computing device, the remote computing device may be connected to a user computing device through any kind of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (for example, through the Internet provided by an Internet service provider).

[0046] Based on the above description, those skilled in the art can understand that the exemplary embodiments described herein can be implemented by software, or by software and hardware, when necessary. Therefore, the technical solution according to the embodiments of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as CD-ROM, USB disk, mobile hard disk, etc.) or on the network, and includes several instructions to make a computing device (which can be a personal computer, a server or a network device, etc.) implement the surround view monitoring method according to the embodiments of the present disclosure.

[0047] Industrial Applicability

[0048] When the work machine 100 is working and the working mechanism 30 is, for example, being lifted or lowered, the electronic device 400 implements cyclically and in real time the surround view monitoring method according to any of the above embodiments of the present disclosure in such a way as to control the first and second mechanism cameras and the remaining cameras to acquire their respective images and generate the first and second panoramic images, switch the imaging mode of the panoramic image back and forth between the first and second mechanism cameras depending on the sizes of blind areas resulting from blocking by the working mechanism in the first and second panoramic images, and acquire a panoramic image always with the smallest blind area at every moment and every position of the working mechanism, thereby reducing the driver's negligence or misjudgment on the surroundings of the work machine caused by the blind area in the image, and thus improving the driving and operation safety of the work machine.

[0049] The work machine 100 is not limited to the crawler excavator shown in the illustrated embodiment, but can be any other type of work machines (e.g. wheel loaders etc.) that perform operations (such as mining, construction, agriculture, and transportation) of specific industries and work or travel in various working environments (such as mines, construction sites, farms, and roads), as long as the work machine has a working mechanism that extends outward from its vehicle body and probably produces a blind area in the panoramic image of the work machine. There is not a limitation on the number of cameras installed on the work machine, as long as they can cooperatively generate a panoramic image of the work machine. Among the plurality of cameras, at least two cameras can capture at least a portion of the working mechanism, in which the two cameras that capture the largest range of the working mechanism are selected as the first and second mechanism cameras.

[0050] After considering the specification and practicing the disclosure herein, those skilled in the art would readily conceive of other embodiments of the present disclosure. The present disclosure is intended to cover any variation, use or adaptation, which follows the general principles of the present disclosure and includes common knowledge or conventional technical means in the art not disclosed herein. The specification and embodiments are to be regarded as exemplary only, with the true scope and spirit of the invention being determined by the appended claims

Claims

Claims1. A surround view monitoring method for a work machine (100), the work machine comprising a working mechanism (30) extending outward from a vehicle body of the work machine; a first mechanism camera (51), a second mechanism camera (55), and at least one remaining camera (52, 53, 54) except the first and second mechanism cameras are installed on and around the vehicle body, wherein the first mechanism camera (51) and second mechanism camera (55) are capable of capturing at least a portion of the working mechanism (30), and the at least one remaining camera (52, 53, 54) cannot catch the working mechanism (30) or can capture a less range of the working mechanism (30) than the first and second mechanism cameras, the method comprising the steps of: acquiring respective images by the first and second mechanism cameras (51, 55) and the remaining camera (52, 53, 54); generating a first panoramic image by using image acquired by the first mechanism camera (51) and image acquired by the remaining camera (52, 53, 54), and generating a second panoramic image by using image acquired by the second mechanism camera (55) and image acquired by the remaining camera (52, 53, 54); determining a first area occupied by the working mechanism (30) in the first panoramic image and a second area occupied by the working mechanism (30) in the second panoramic image; and comparing the first area with the second area and outputting the panoramic image corresponding to a smaller one of the first and second areas as a panoramic image of the work machine (100).

2. The surround view monitoring method according to claim 1, characterized in that the step of determining the first area and the second area comprises: selecting a predetermined region containing the working mechanism (30) in the first or second panoramic image, conducting edge detection of theworking mechanism in the predetermined region to delimit a contour of the working mechanism, and calculating an area closed by the contour as the first or second area.

3. The surround view monitoring method according to claim 1 or 2, characterized in that the first panoramic image and the second panoramic image are images of two-dimensional bird's eye view.

4. A computer-readable storage medium having a computer program stored therein, the computer program including executable instructions which, when executed by a processor, implement the surround view monitoring method according to any one of claims 1 to 3.

5. An electronic device (400), comprising: a processor (410); and a memory (420) for storing executable instructions of the processor; wherein the processor (410) is configured to execute the executable instructions to implement the surround view monitoring method according to any one of claims 1 to 3.

6. A work machine (100), comprising: a working mechanism (30) extending outward from a vehicle body of the work machine; a plurality of cameras installed on and around the vehicle body, the cameras comprising a first mechanism camera (51), a second mechanism camera (55), and at least one remaining camera (52, 53, 54) except the first and second mechanism cameras, wherein the first and second mechanism cameras are capable of capturing at least a portion of the working mechanism (30), and the at least one remaining camera cannot catch the working mechanism (30) or cancapture a less range of the working mechanism (30) than the first and second mechanism cameras; and an electronic device (400) according to claim 5, which is in communicative connection with the plurality of cameras.

7. The work machine (100) according to claim 6, characterized in that the first mechanism camera (51) and the second mechanism camera (55) are installed on the vehicle body at opposite sides of a vertical plane where the working mechanism (30) is located.

8. The work machine (100) according to claim 7, characterized in that the vehicle body comprises a cab (10) and a cabin assembly (20), wherein the cab and a front compartment (21, 22) of the cabin assembly are disposed at opposite sides of the vertical plane where the working mechanism (30) is located, and the first mechanism camera (51) and the second mechanism camera (55) are installed at a front top of the cab (10) and a front top of the front compartment (21, 22) of the cabin assembly (20), respectively.

9. The work machine (100) according to claim 8, characterized in that the remaining camera includes a left camera (52) installed at a left top of the cab (10), and a right camera (53) and a rear camera (54) installed at a right top and rear top of the cabin assembly (20) respectively.

10. The work machine (100) according to any one of claims 6 to 9, characterized in that the working mechanism (30) comprises one or more of a boom (31), a stick (32) and a working implement (33) of the work machine.

Images