Cockpit warning recognition system

Abstract

To provide a cockpit warning recognition system.SOLUTION: A plurality of cameras disposed on the construction vehicle and configured to capture images of surroundings of the construction vehicle, the plurality of cameras being disposed in a cockpit of the construction vehicle and signally connected to the cameras, the plurality of cameras being configured to receive the images of the surroundings and synthesize the images into a panoramic image of the construction vehicle; A cockpit warning recognition system, comprising: a processor configured to perform image recognition processing on a panoramic image to obtain a risk state of a construction vehicle; and a display device disposed in a cockpit and signal-connected to the processor, the display device being configured to receive and display the panoramic image, wherein the processor controls the display device to display a corresponding picture on the panoramic image according to the risk state of the construction vehicle.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present disclosure relates to vehicle warning technology, and more particularly to a cockpit warning recognition system. [Background technology]

[0002] Construction vehicles such as excavators and earthmoving machines should be equipped with warning devices such as reversing or rotating warning lights and buzzers to warn surrounding personnel and prevent them from colliding with them during work. However, such warning devices are only used to warn surrounding personnel and cannot provide workers with more complete information about surrounding risk objects, making it difficult for workers to effectively grasp changes in the work environment, such as personnel suddenly entering the operating radius of the construction vehicle.

[0003] Therefore, there is an urgent need for a warning system for construction vehicles to provide workers with more complete information about the working environment. Summary of the Invention [Means for solving the problem]

[0004] One of the objectives of the present disclosure is to provide a cockpit warning recognition system in which a processor receives surrounding images taken by multiple cameras, synthesizes them into a panoramic image of the construction vehicle, and performs image recognition processing on the panoramic image to determine whether there are any risk objects in the panoramic image and the movement speed and movement direction of the risk objects, thereby obtaining the risk status of the construction vehicle and displaying it on a display device in the cockpit.As a result, the cockpit warning recognition system can provide more complete information about surrounding risk objects to workers in the cockpit, allowing them to effectively grasp changes in the work environment, thereby significantly reducing work risks.

[0005] According to the above-mentioned object of the present disclosure, a cockpit warning recognition system is proposed which includes a plurality of cameras mounted on a construction vehicle and each arranged to capture images of the surroundings of the construction vehicle; a processor mounted in the cockpit of the construction vehicle so as to be signal-connected to the cameras and arranged to receive images of the surroundings and combine them into a panoramic image of the construction vehicle, and perform image recognition processing on the panoramic image to obtain the risk state of the construction vehicle; and a display device mounted in the cockpit so as to be signal-connected to the processor and arranged to receive and display the panoramic image, wherein the processor controls the display device to display a screen corresponding to the panoramic image depending on the risk state of the construction vehicle.

[0006] According to one embodiment of the present disclosure, there are four cameras, each with a 190-degree imaging range.

[0007] According to one embodiment of the present disclosure, the processor is provided within a display device having a touch screen.

[0008] According to one embodiment of the present disclosure, the panoramic image includes an image of a construction vehicle, and the processor is arranged to further divide the panoramic image into a plurality of regions surrounding the outside of the image of the construction vehicle, and the corresponding screen includes a plurality of sub-screens respectively corresponding to the plurality of regions, and the processor independently controls the plurality of sub-screens.

[0009] According to one embodiment of the present disclosure, each of the sub-screens includes a plurality of warning patterns arranged in a direction from beside the image of the construction vehicle to away from the image.

[0010] According to one embodiment of the present disclosure, in each of the sub-screens, the plurality of warning patterns have different colors.

[0011] According to one embodiment of the present disclosure, when at least one risk object appears in one of the areas of the panoramic image as a risk condition of the construction vehicle, the processor controls the display device to display a sub-screen corresponding to the area, but when the risk object no longer exists in the area of ​​the panoramic image as a risk condition of the construction vehicle, the processor controls the display device to close the sub-screen corresponding to the area.

[0012] According to one embodiment of the present disclosure, when the risk object enters the area and moves toward the image of the construction vehicle, the processor controls the display device to display a sub-screen corresponding to the area, and display the warning patterns on the sub-screen one by one at 100% brightness as the risk object approaches, but close the multiple warning patterns so that they gradually fade within a preset time after the risk object leaves, and display the warning patterns at the location where the risk object remains at 50% transparency.

[0013] According to one embodiment of the present disclosure, when the risk object enters the area and moves into the image at a speed greater than a preset speed, the processor controls the display device to display a sub-screen corresponding to the area, and display warning patterns on the sub-screen one by one at 100% brightness as the risk object approaches, but close the multiple warning patterns so that they gradually fade after the risk object leaves, repeating this process twice.

[0014] According to one embodiment of the present disclosure, the cockpit warning recognition system further comprises a buzzer provided in the construction vehicle so as to be signally connected to the processor, and the processor is further configured to control the buzzer to emit a warning sound when at least one risk object appears in the panoramic image as a risk condition of the construction vehicle.

[0015] According to one embodiment of the present disclosure, the cockpit warning recognition system further comprises a warning light provided on the construction vehicle so as to be signally connected to the processor, and the processor is further configured to control the warning light to emit a warning light when at least one risk object appears in the panoramic image as a risk condition of the construction vehicle. [Brief explanation of the drawings]

[0016] Aspects of the present disclosure will be better understood from the following detailed description taken in conjunction with the accompanying drawings. It should be noted that, in accordance with standard industry practice, features are not drawn to scale. In fact, the size of features may be arbitrarily increased or decreased for clarity of discussion. [Figure 1] FIG. 1 is a block diagram illustrating a cockpit warning recognition system according to one embodiment of the present disclosure. [Figure 2] 1 is a schematic diagram illustrating a construction vehicle according to an embodiment of the present disclosure. [Figure 3] FIG. 1 is a schematic diagram illustrating a panoramic image of a construction vehicle according to an embodiment of the present disclosure. [Figure 4] FIG. 10 is a display schematic diagram showing a warning pattern on a sub-screen of a response screen when a construction vehicle is in a risk state according to an embodiment of the present disclosure. [Figure 5] FIG. 10 is a display schematic diagram showing a warning pattern on a sub-screen of a response screen in a risk state of another construction vehicle according to an embodiment of the present disclosure. [Figure 6] FIG. 2 is a block diagram illustrating a cockpit warning recognition system according to another embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0017] The following detailed description of the embodiments of the present disclosure will be given. However, as will be understood, the embodiments provide many applicable concepts and are implemented in various specific contexts. The embodiments discussed and disclosed are for illustrative purposes only and are not intended to limit the scope of the present disclosure. Although all embodiments of the present disclosure disclose multiple different features, these features may be implemented individually or in combination as needed.

[0018] Furthermore, terms such as "first," "second," etc., as used in this specification do not refer to a particular order or sequence, but are merely used to distinguish between elements or operations described with the same technical terminology.

[0019] The spatial relationship between two elements described in this disclosure applies not only to the orientation shown in the drawings but also to orientations not shown in the drawings, such as the opposite orientation. Furthermore, the term "connection," "electrical connection," or similar terms between two members described in this disclosure is not limited to direct connection or electrical connection between the two, but can also include indirect connection or electrical connection as needed.

[0020] Please refer to FIG. 1 which is a block diagram of a cockpit warning recognition system 100 according to one embodiment of the present disclosure, and FIG. 2 which shows a schematic diagram of a construction vehicle 500. The cockpit warning recognition system 100 is used in the construction vehicle 500 to provide complete risk information to a worker in the cockpit 510 of the construction vehicle 500, thereby improving safety during operation of the construction vehicle 500. For example, the construction vehicle 500 may be an excavator, a bulldozer, a forklift, or the like.

[0021] The cockpit warning recognition system 100 may mainly include multiple cameras 200, a processor 300, and a display device 400. These cameras 200 are attached to the construction vehicle 500. For example, these cameras 200 may be attached to the front, rear, left, and right sides of the cockpit 510 of the construction vehicle 500 to capture images of the surroundings of the construction vehicle 500 from four sides of the construction vehicle 500. In some embodiments, there are four cameras 200, each with a 190-degree viewing range. In this way, the viewing ranges of these cameras 200 can completely cover the surroundings of the construction vehicle 500. The number and viewing ranges of the cameras 200 are not limited to those in the above embodiment, and may be any number as long as the combined viewing ranges of all the cameras 200 can completely cover the surroundings of the construction vehicle 500.

[0022] The processor 300 is installed in the cockpit 510 of the construction vehicle 500. The processor 300 can be signal-connected to the cameras 200 via wired or wireless transmission. This allows the processor 300 to receive images of the surroundings of the construction vehicle 500 captured by the cameras 200. See FIG. 3 , which illustrates a schematic diagram of a panoramic image 600 of the construction vehicle 500 according to an embodiment of the present disclosure. The processor 300 can stitch together the received images of the surroundings to construct the panoramic image 600 of the construction vehicle 500 in an image processing manner. In the example of FIG. 3 , the panoramic image 600 is an octagonal image. The panoramic image 600 may be an image of other shapes, such as a rectangle, a circle, or a polygon other than a rectangle, and the present disclosure is not limited thereto. The panoramic image 600 may include an image 500i of the construction vehicle 500. In some embodiments, the processor 300 can divide the panoramic image 600 into a plurality of regions 610a-610h, and these regions 610a-610h surround the outside of the image 500i of the construction vehicle 500. The panoramic image 600 is not limited to eight regions 610a-610h, and the panoramic image 600 can be divided according to usage needs, and the present disclosure is not limited thereto.

[0023] The processor 300 can further perform image recognition processing on the panoramic image 600 to recognize whether the risk target RO appears in the panoramic image 600, the movement speed of the risk target RO, and the distance between the risk target RO and the image 500i of the construction vehicle 500, thereby acquiring the risk state of the construction vehicle 500. For example, the processor 300 can calculate the movement speed of the risk target RO using the movement distance between two images of the risk target RO and the display time of each image. In addition, the processor 300 can acquire the distance between the risk target RO and the construction vehicle 500 by directly recognizing the distance between the risk target RO and the image 500i of the construction vehicle 500 in the panoramic image 600.

[0024] In some embodiments, the processor 300 is connected to the control system of the cockpit 510 of the construction vehicle 500 by wired or wireless signals, and can receive operation information of the construction vehicle 500. For example, the processor 300 can acquire rotation information, movement information, and machine tool work information of the construction vehicle 500 from the control system of the cockpit 510. The work information of the machine tool may be, for example, information on the digging work of an excavator, information on the earth-pushing and earth-shoveling work of a bulldozer, and information on the lifting and lowering of the forks of a forklift. Since the risk state when the construction vehicle 500 is rotating, moving, or operating is higher than the risk state when the construction vehicle 500 is stationary, the processor 300 can reduce the work risk of the construction vehicle 500 by more deeply determining the risk state of the construction vehicle 500 based on this operation information of the construction vehicle 500.

[0025] The processor 300 can predict whether the risk target RO will enter the working range of the construction vehicle 500 and the time when the risk target RO will enter the working range of the construction vehicle 500 based on the moving speed and direction of the risk target RO. When making the above prediction, the processor 300 can further combine the operation information of the construction vehicle 500.

[0026] The display device 400 may be provided in the cockpit 510 of the construction vehicle 500 and may be signal-connected to the processor 300 via a wired or wireless transmission format. In some embodiments, the processor 300 may be provided in the display device 400 and electrically connected to the display device 400 via a physical line. The display device 400 may receive a panoramic image 600 from the processor 300 and display the panoramic image 600. In some embodiments, as shown in FIG. 3 , the panoramic image 600 includes a display area 620, and the range R of the panoramic image 600, which is composed of images of the surroundings captured by the camera 200, is larger than the display area 620. That is, the detection range of the camera 200 is larger than the display range of the display device 400. This allows the processor 300 to recognize a risk object RO when the risk object RO is within the range R of the panoramic image 600 but not within the display area 620.

[0027] The processor 300 can further control the display device 400 to display a corresponding screen 700 on the panoramic image 600 in accordance with the risk state of the construction vehicle 500 acquired by the image recognition process. The corresponding screen 700 can include multiple sub-screens, such as sub-screens 710a-710h. In the embodiment shown in FIG. 3, the panoramic image 600 is divided into eight regions 610a-610h, and the corresponding screen 700 is divided into eight sub-screens 710a-710h, which are displayed corresponding to the regions 610a-610h, respectively. That is, the number of sub-screens 710a-710h is the same as the number of regions 610a-610h. The processor 300 can independently control these sub-screens 710a-710h. For example, if a risk target RO enters the region 610h of the panoramic image 600, the processor 300 controls the display device 400 to display the sub-screen 710h of the corresponding screen 700 in the region 610h.

[0028] In some embodiments, each of the sub-screens 710a-710h includes multiple warning patterns, such as warning patterns 712, 714, and 716. The warning patterns 712, 714, and 716 may be displayed as an on-screen display (OSD). The number of warning patterns on each of the sub-screens 710a-710h can be adjusted as needed and is not limited to three. As shown in FIG. 3 , the warning patterns 712, 714, and 716 are arranged next to the image 500i of the construction vehicle 500 in a direction away from the image 500i. Specifically, the warning pattern 712 is closest to the image 500i of the construction vehicle 500, the warning pattern 716 is farthest from the image 500i, and the warning pattern 714 is located between the warning patterns 712 and 716. In some embodiments, the warning patterns 712, 714, and 716 have different colors, which makes it easier for the worker to intuitively recognize the level of risk. For example, warning pattern 712 may be red, warning pattern 714 may be orange, and warning pattern 716 may be green.

[0029] 3 and 4 simultaneously, FIG. 4 shows a display schematic diagram of warning patterns 712, 714, and 716 on a sub-screen 710h of a response screen 700 in a risk state of the construction vehicle 500 according to one embodiment of the present disclosure. When at least one risk target RO appears in one of the regions 610a-610h of the panoramic image 600 as a risk state of the construction vehicle 500 acquired by the processor 300, the processor 300 controls the display device 400 to display a corresponding sub-screen among these sub-screens 710a-710h in this region of these regions 610a-610h. For example, when a risk target RO appears in the region 610h of the panoramic image 600 as a risk state of the construction vehicle 500, the processor 300 controls the display device 400 to display the sub-screen 710h in the region 610h. In some embodiments, when the risk target RO enters the area 610h and moves into the image 500i of the construction vehicle 500, the processor 300 controls the display device 400 to display the sub-screen 710h corresponding to the area 610h, and to display the warning patterns 716, 714, and 712 of the sub-screen 710h one by one at 100% brightness, i.e., directly from 0% to 100% brightness, as the risk target RO approaches. These warning patterns 716, 714, and 712 are faded to decrease from 100% brightness to 50% within a preset time, for example, within 5 seconds, after the risk target RO leaves.

[0030] Specifically, when the risk target RO enters the region 610h and is first close to the position corresponding to the warning pattern 716, the processor 300 controls the display device 400 to display the warning pattern 716 in the sub-screen 710h at 100% brightness in the region 610h. If the risk target RO continues to move toward the image 500i of the construction vehicle 500 to the position corresponding to the warning pattern 714, the processor 300 controls the display device 400 to display the warning pattern 714 in the sub-screen 710h at 100% brightness and gradually fade the warning pattern 716 from 100% brightness to 50% within a preset time. If the risk target RO continues to move to the position corresponding to the warning pattern 712, the processor 300 controls the display device 400 to display the warning pattern 712 in the sub-screen 710h at 100% brightness and gradually fade the warning pattern 714 from 100% brightness to 50% within the preset time, or gradually fade it to a closed state. If the risk target RO finally stays in the warning pattern 712, the warning pattern 712 is displayed with, for example, 50% transparency.

[0031] When the risk target RO disappears in one of the regions 610a to 610h of the panoramic image 600 as a risk state of the construction vehicle 500, the processor 300 controls the display device 400 to close the corresponding sub-screen among the sub-screens 710a to 710h of this region of the regions 610a to 610h. For example, when the risk target RO disappears in the region 610h, the processor 300 controls the display device 400 to close the corresponding sub-screen 710h.

[0032] 3 and 5, FIG. 5 shows a schematic diagram of the display of warning patterns 712, 714, and 716 on a sub-screen 710h of the response screen 700 in a risk state of another construction vehicle 500 according to an embodiment of the present disclosure. When a risk target RO enters one of the regions 610a to 610h, for example, region 610h, and moves rapidly toward the image 500i of the construction vehicle 500 at a speed greater than a preset speed, the processor 300 controls the display device 400 to display the sub-screen 710h corresponding to region 610h. The processor 300 controls the warning patterns 716, 714, and 712 on the sub-screen 710h to be displayed one by one at 100% brightness as the risk target RO approaches, and to gradually fade these warning patterns 716, 714, and 712 from 100% brightness to 50% brightness or to gradually fade to a fading state twice after the risk target RO leaves. The warning pattern 712, 714, or 716 corresponding to the last location where the RO at risk is staying is displayed with, for example, 50% transparency. Thus, the cockpit warning recognition system 100 can implement different warning forms depending on the movement conditions of the RO at risk.

[0033] The screen of the display device 400 may be a touch screen, and therefore may be used as a man-machine operation interface so that the driver can easily set the image recognition processing settings of the processor 300, the display settings of the display device 400, and the like.

[0034] 6, which illustrates a block diagram of a cockpit warning recognition system 100a according to another embodiment of the present disclosure. The cockpit warning recognition system 100a of this embodiment has almost the same structure as the above-described cockpit warning recognition system 100, except that the cockpit warning recognition system 100a further includes a buzzer 800 and a warning light 900.

[0035] Please refer to Figures 2 and 3 together. The buzzer 800 is provided on the construction vehicle 500 and is signal-connected to the processor 300 in the form of wired or wireless transmission. In some embodiments, the buzzer 800 is independent of the display device 400 and is electrically connected to the display device 400 using a line. The buzzer 800 may also be electrically connected to a circuit system in the cockpit 510 of the construction vehicle 500. When the processor 300 recognizes that a risk object RO has appeared in any of the areas 610a to 610h of the panoramic image 600 as a risk state of the construction vehicle 500, it controls the buzzer 800 to emit a warning sound to alert the worker.

[0036] The warning light 900 is provided on the construction vehicle 500 and is signal-connected to the processor 300 in the form of wired or wireless transmission. In some embodiments, the warning light 900 is externally attached to the display device 400 via a line. When the processor 300 recognizes that a risk object RO has appeared in any of the areas 610a to 610h of the panoramic image 600 as a risk state of the construction vehicle 500, it controls the warning light 900 to emit a warning light to alert the worker.

[0037] In some embodiments, when the construction vehicle 500 is not operating, the display device 400 can be used alone to display the panoramic image 600 and the corresponding screen 700. When the construction vehicle 500 is not operating, in addition to displaying the panoramic image 600 and the corresponding screen 700, a warning can also be provided by combining the buzzer 800 and the warning light 900. When the construction vehicle 500 is moving, rotating, and / or operating, it is preferable to use the display device 400 to display the panoramic image 600 and the corresponding screen 700, and in addition to displaying the buzzer 800 and the warning light 900, a warning can also be provided by combining the buzzer 800 and the warning light 900.

[0038] As can be seen from the above embodiment, one advantage of the present disclosure is as follows: In the cockpit warning recognition system of the present disclosure, a processor receives surrounding images taken by multiple cameras and synthesizes them into a panoramic image of the construction vehicle. Furthermore, by performing image recognition processing on the panoramic image, it is possible to recognize whether there are risk objects in the panoramic image and the movement speed and movement direction of the risk objects, thereby obtaining the risk status of the construction vehicle and displaying it on a display device in the cockpit. Therefore, the cockpit warning recognition system can provide more complete information about surrounding risk objects to the worker in the cockpit, allowing the worker to effectively grasp changes in the work environment, thereby significantly reducing work risks.

[0039] Although the embodiments of the present disclosure have been disclosed as above, the embodiments do not limit the present disclosure, and anyone skilled in the art may make various changes and modifications without departing from the spirit and scope of the present disclosure. Therefore, the scope of protection of the present disclosure shall be determined based on the content specified in the scope of the utility model registration claims. [Explanation of symbols]

[0040] 100: Cockpit Warning Recognition System 100a: Cockpit Warning Recognition System 200: Camera 300: Processor 400:Display device 500: Construction vehicles 500i:Image 510: Cockpit 600: Panorama image 610a:Area 610b:Area 610c: area 610d: Area 610e: Area 610f: area 610g: area 610h: area 620: Display area 700: Compatible screens 710a: Sub screen 710b: Subscreen 710c: Sub screen 710d: Sub screen 710e: Subscreen 710f: Sub screen 710g: Sub-screen 710h: Sub screen 712: Warning pattern 714: Warning pattern 716: Warning pattern 800: Buzzer 900: Warning light R:Range RO: Risk Object

Claims

1. A plurality of cameras provided on a construction vehicle and arranged to capture images of the surroundings of the construction vehicle; a processor provided in the cockpit of the construction vehicle so as to be signally connected to the plurality of cameras, the processor being configured to receive the plurality of surrounding images, synthesize the images into a panoramic image of the construction vehicle, and perform image recognition processing on the panoramic image to obtain a risk state of the construction vehicle; a display device provided within the cockpit so as to be signally connected to the processor and arranged to receive and display the panoramic image; Equipped with The processor controls the display device to display a corresponding screen to the panoramic image according to the risk state of the construction vehicle.

2. The cockpit warning recognition system according to claim 1, wherein the number of cameras is four, and each camera has a shooting range of 190 degrees.

3. The cockpit warning recognition system of claim 1 , wherein the processor is provided within the display device having a touch screen.

4. 2. The cockpit warning recognition system of claim 1, wherein the panoramic image includes an image of the construction vehicle, the processor is further configured to divide the panoramic image into a plurality of areas surrounding the outside of the image of the construction vehicle, the corresponding screen includes a plurality of sub-screens corresponding to the plurality of areas, and the processor independently controls the plurality of sub-screens.

5. The cockpit warning recognition system according to claim 4 , wherein each of the plurality of sub-screens includes a plurality of warning patterns arranged in a direction from beside the image of the construction vehicle to away from the image.

6. The cockpit warning recognition system according to claim 5, wherein in each of the plurality of sub-screens, the plurality of warning patterns have different colors.

7. When at least one risk object appears in one of the plurality of regions of the panoramic image as the risk state of the construction vehicle, the processor controls the display device to display the corresponding sub-screen in the one of the plurality of regions, The cockpit warning recognition system of claim 5, wherein when the at least one risk object is no longer present in one of the plurality of regions of the panoramic image as the risk state of the construction vehicle, the processor controls the display device to close the sub-screen corresponding to the one of the plurality of regions.

8. 8. The cockpit warning recognition system of claim 7, wherein when the at least one risk object enters one of the plurality of regions and moves into the image, the processor controls the display device to display the corresponding sub-screen in the one of the plurality of regions, display the plurality of warning patterns on the sub-screen one by one at 100% brightness as the at least one risk object approaches, but close the plurality of warning patterns to gradually fade within a preset time after the at least one risk object leaves, and display the corresponding one of the plurality of warning patterns at a location where the at least one risk object remains at 50% transparency.

9. 9. The cockpit warning recognition system of claim 8, wherein when the at least one risk object enters the one of the plurality of regions and moves into the image at a speed greater than a preset speed, the processor controls the display device to display the corresponding sub-screen in the one of the plurality of regions, and to display the plurality of warning patterns on the sub-screen one by one at 100% brightness as the at least one risk object approaches, but to close the plurality of warning patterns in the fading manner after the at least one risk object leaves, twice.

10. 2. The cockpit warning recognition system of claim 1, further comprising a buzzer provided in the construction vehicle so as to be signally connected to the processor, the processor being further configured to control the buzzer to emit a warning sound when at least one risk object appears in the panoramic image as the risk condition of the construction vehicle.

11. 2. The cockpit warning recognition system of claim 1, further comprising a warning light provided on the construction vehicle so as to be signally connected to the processor, the processor being further configured to control the warning light to emit a warning light when at least one risk object appears in the panoramic image as the risk condition of the construction vehicle.

Images