Detection system for a construction machine

Abstract

The present application provides a kind of detection system of engineering machinery.The remote control system (1) with the function of detection system has: multiple cameras (25,60) arranged in the arrangement environment of engineering machinery (10);Photographing image processing unit (50a) can display the photographing image of each multiple camera to the display (45);And, camera selection unit (50b) selects the camera for detection from multiple cameras (25,60) when detecting engineering machinery (10).Photographing image processing unit (50a) is configured to be able to receive the photographing image photographed by the selected camera for detection in the form of image containing detection target part, and display the photographing image on the display (45).

Description

Technical Field

[0001] This invention relates to a testing system for engineering machinery. Background Technology

[0002] When using hydraulic excavators and other construction machinery, it is necessary to properly inspect the exterior for damage such as cracks and oil leaks, as foreign objects may collide with various parts of the machine or the boom, stick, and other movable parts may be subjected to large loads.

[0003] In this situation, the operator who controls the construction machinery would typically conduct the inspection visually from close range.

[0004] In addition, as described in Patent Document 1, there is a known inspection device in which a support for an inspection camera that takes pictures of the engine compartment is mounted on a construction machine, and the inspection operator can visually confirm the images captured by the camera using a display in the driver's cab of the construction machine.

[0005] Existing technical documents

[0006] Patent documents

[0007] Patent Document 1: Japanese Patent Application Publication No. 2013-221303 Summary of the Invention

[0008] The problem that the invention aims to solve

[0009] However, it is desirable to be able to perform inspection work on construction machinery in locations far from the machinery itself. In particular, when using construction machinery that can be remotely operated, the operators of such machinery are located in locations far from the machinery, so it is desirable to be able to perform inspection work in remote locations as well.

[0010] In this case, in the technology described in Patent Document 1, since a camera needs to be installed on the construction machinery during inspection, even if the camera's images can be viewed on a monitor from a location away from the construction machinery, an operator is still required to install the camera. Furthermore, installing only one camera on the construction machinery makes it difficult to inspect multiple parts of the machinery. Therefore, it is necessary to install multiple cameras on the construction machinery and / or change the camera's mounting location.

[0011] On the other hand, in environments where construction machinery can be remotely operated, multiple cameras capable of capturing images of the machinery from the outside are typically installed to visually confirm its operational status. Therefore, utilizing these cameras for the inspection of construction machinery can be considered.

[0012] The present invention was made in view of the above background, and its object is to provide a system that can properly perform inspection operations on engineering machinery from a remote location.

[0013] Methods for solving problems

[0014] To achieve the above objectives, the inspection system for construction machinery of the present invention is characterized by comprising: a plurality of cameras configured in the configuration environment of the construction machinery in a manner capable of photographing the appearance of the construction machinery;

[0015] An image processing unit is disposed externally on the construction machinery, capable of receiving images captured by the plurality of cameras respectively; the image processing unit includes a display capable of displaying the received images; and...

[0016] The camera selection unit, when inspecting the construction machinery, selects from the plurality of cameras a camera specifically for photographing the inspection target area of ​​the construction machinery, i.e., the inspection camera.

[0017] The image processing unit is configured to receive an image captured by a selected detection camera in a manner that includes an image of the detection target area, and to display the image on the display screen.

[0018] Brief description of the attached diagram

[0019] Figure 1 This is a diagram showing the overall structure of a remote control system having the functions of a detection system as an embodiment of the present invention.

[0020] Figure 2 It means and Figure 1 The diagram shown is a block diagram of the control processing components of the remote control system.

[0021] Figure 3 This is an example Figure 1 The diagram shows the configuration of the control device.

[0022] Figure 4 This is an explanatory diagram of the selected processing for the detection camera.

[0023] Figure 5 This is an example of an image captured by a camera used for testing. Detailed Implementation

[0024] The following is for reference Figures 1 to 5 An embodiment of the present invention will be described. Figure 1This describes the overall configuration of a remote control system 1 for construction machinery, including the detection system as described in this embodiment. The remote control system 1 is configured such that an operator can remotely control the construction machinery 10 via an operating device 40. The construction machinery 10 is the type of construction machinery assigned to the operator or the operating device 40 as the object of control. It should be noted that... Figure 1 The text typically describes a scenario where there is one piece of engineering machinery 10 and one control device 40, but in the remote control system 1, there may be multiple pieces of engineering machinery 10 and multiple control devices 40.

[0025] In addition to the control device 40 and the construction machinery 10, the remote control system 1 also includes a server 50 that performs various operational and information processing related to the remote control system 1, and multiple cameras 60 (hereinafter referred to as field cameras 60) set up at the work site, wherein the work site serves as the configuration environment in which each construction machinery 10 is configured.

[0026] Furthermore, each piece of construction machinery 10, each operating device 40, and the server 50 can communicate with each other via a network NW, including a wireless communication network. Additionally, each operating device 40 and the server 50 can appropriately acquire images captured by the field-mounted cameras 60 via the network NW. Each field-mounted camera 60 is configured to appropriately change its shooting direction (orientation of the optical axis) and shooting magnification according to instructions from the operating device 40 or the server 50.

[0027] As for the various construction machines 10, a hydraulic excavator may be used, for example. In this case, the construction machine 10, like a known hydraulic excavator, has a tracked (or wheeled) running body 11, a slewing body 12 rotatably mounted on the running body 11, and a working device 13 installed at the front of the slewing body 12. The slewing body 12 has a cab 12a at the front and a machine room 12b at the rear that houses the engine 22 and hydraulic circuit device 21, which will be described later. In addition, the working device 13 has a boom 13a, a stick 13b, and attachments 13c (e.g., a bucket).

[0028] Additionally, although detailed illustrations have been omitted, as... Figure 2As shown, each piece of construction machinery 10 includes: multiple hydraulic actuators 20; a hydraulic circuit device 21, which includes a hydraulic pump (not shown) for supplying working oil to each hydraulic actuator 20, a direction switching valve, etc.; an engine 22 that drives the hydraulic pump of the hydraulic circuit device 21; an operating device 23 for operating each hydraulic actuator 20; an electric operating drive device 24 that drives the operating device 23; multiple cameras 25 (hereinafter referred to as mounted cameras 25), which are mounted on the rotating body 12, etc., to capture images of the surroundings of the construction machinery 10 (including the area in front of the cab 12a); and a control device 30 (hereinafter referred to as the machine-side control device 30), which has the function of performing operation control of the construction machinery 10. It should be noted that... Figure 2 The text describes, in a representative way, the presence of one hydraulic actuator 20 and a camera 25.

[0029] Hydraulic actuator 20 includes: hydraulic cylinders 20a, 20b, and 20c that respectively drive the boom 13a, stick 13b, and accessory 13c of the working device 13. Figure 1 (as shown); a hydraulic motor for driving the travel of the traveling body 11 (not shown); a hydraulic motor for driving the rotation of the rotating body 12 (not shown).

[0030] The operating device 23 includes a lever, a pedal, and a switch disposed in the cab 12a. Furthermore, the operating drive device 24 is configured to, for example, have multiple electric motors, which can drive the lever and pedal included in the operating device 23 via a suitable power transmission mechanism.

[0031] The mechanical control device 30 is composed of one or more electronic circuit units, such as a microcomputer, memory (RAM, ROM, etc.), interface circuit, and wireless communication device. Furthermore, the mechanical control device 30 has the following functions implemented by one or both of the installed hardware structure and program (software structure): the function of acquiring images captured by the camera 25; the function of communicating with the main control device 47 (described later) and the server 50 of the control device 40 via the network NW; and the function of controlling the operation of the construction machinery 10 by controlling the operation of the electric valves (electromagnetic proportional valves, etc.) of the hydraulic circuit device 21, the engine 22, and the operation drive device 24.

[0032] In this case, the mechanical side control device 30 can receive the working instructions of the construction machinery 10 or send information to the main side control device 47 or the server 50, indicating the captured images of the camera 25 and the working status of the construction machinery 10.

[0033] like Figure 3As shown, each control device 40 in the remote control room 41 includes: a seat 42 for the operator to sit on; an operating device 43 for the operator to operate for remote control of the construction machinery 10; a speaker 44 as an audio information (auditory information) output device; and a display 45 as a display information (visual information) output device.

[0034] The operating device 43 may, for example, be a device with the same or similar structure as the operating device 23 of the engineering machinery 10. Figure 3 The illustrated operating device 43 includes: an operating lever 43a with an operating pedal 43ap, which is arranged on the front side of the seat 42 for operation by an operator seated on the seat 42; and operating levers 43b, which are respectively arranged on the left and right sides of the seat 42. However, the operating device 43 may also have a different structure from the operating device 23 of the construction machinery 10. For example, the operating device 43 may also be a portable operating device with joysticks, operating buttons, etc.

[0035] Speakers 44 are located at multiple locations in the remote control room 41 (e.g., the front, rear, and left and right sides of the remote control room 41). A display 45, such as a liquid crystal display or a head-up display, is provided in the remote control room 41 in a manner that allows the operator seated in the seat 42 to visually confirm the information. It should be noted that in this embodiment, the display 45 functions as the display in this invention.

[0036] In addition, such as Figure 2 As shown, the control device 40 also includes: an operation status detector 46, which is used to detect the operation status of the control device 43; and a master control device 47, which has the function of performing control processing related to the control device 40. It should be noted that the master control device 47 can also be configured inside or outside the remote control room 41.

[0037] The operation status detector 46 includes, for example, a potentiometer, a contact switch, etc. assembled on the operation device 43, and is configured to output a detection signal indicating the operation status of each operation part (operation lever 43a, 43b, operation pedal 43ap, etc.) of the operation device 43.

[0038] The main control device 47 is composed of one or more electronic circuit units, such as a microcomputer, memory (RAM, ROM, etc.), interface circuit, and communication device. Furthermore, the main control device 47 has the following functions implemented by one or both of the installed hardware structure and program (software structure): the function of acquiring the detection signal of the operation status detector 46; the function of acquiring the captured images of the on-site camera 60 at the work site of the engineering machinery 10 equipped with the manipulated object via the network NW; the function of communicating with the mechanical control device 30 and the server 50 of each engineering machinery 10 of the manipulated object via the network NW; and the function of controlling the operation of the speaker 44 and the display of the monitor 45.

[0039] In this configuration, the master control device 47 can communicate with the machine-side control device 30 of the construction machinery 10 to obtain images captured by the camera 25 mounted on the construction machinery 10, information indicating the working status of the construction machinery 10, and can send working instructions for the construction machinery 10 generated based on the operating status of the operating device 43 to the machine-side control device 30. Furthermore, the master control device 47 can communicate with the server 50 to obtain various information and / or instructions related to the remote operation of the construction machinery 10 from the server 50.

[0040] Server 50 may be composed of one or more computers, or one or more electronic circuit units including microcomputers, or combinations thereof. Furthermore, server 50 has the following functions implemented by one or both of the installed hardware structure and program (software structure): the function of acquiring images captured by each on-site camera 60 via network NW; and the function of communicating via network NW with the mechanical side control device 30 of each construction machine 10 and the main side control device 47 of each operating device 40. Furthermore, server 50 can also communicate with the operating device 40 via a communication terminal (not shown) operated by an operator, etc.

[0041] Furthermore, the server 50 can obtain images captured by the camera 25 mounted on the construction machinery 10 and information indicating the working status of the construction machinery 10 from the mechanical control device 30 of the construction machinery 10 through communication with the mechanical control device 30 of the construction machinery 10.

[0042] In addition, through communication with the master control device 47 of the operating device 40, the server 50 can obtain information indicating the working status of the operating device 40 from the master control device 47, and can send various information and / or instructions related to the remote operation or detection of the construction machinery 10 to the master control device 47 of the operating device 40 that can operate the construction machinery 10.

[0043] In addition, server 50 stores information representing the operating schedules and operating histories of the construction machinery 10 and the control device 40, respectively. Server 50 also stores position information of each field-mounted camera 60 (indicating the position of each field-mounted camera 60 as observed in the global coordinate system) or position information of the camera's field-capable area (indicating the position of the field-capable range of each field-mounted camera 60 as observed in the global coordinate system). It should be noted that the position information of each field-mounted camera 60 may include information indicating the orientation of the optical axis of each field-mounted camera 60 or its variable range. Alternatively, the position information of each field-mounted camera 60 or its field-capable range may be stored in the memory of each field-mounted camera 60, and server 50 may appropriately retrieve this position information from each field-mounted camera 60.

[0044] Furthermore, the server 50 can appropriately acquire the position information (information representing the position of the construction machinery 10 as observed from the global coordinate system) of each construction machinery 10 using GNSS (Global Navigation Satellite System), images captured by on-site cameras 60 configured at the work site of each construction machinery 10, or images captured by cameras 25 mounted on other construction machinery 10s existing around each construction machinery 10. It should be noted that the position information of the construction machinery 10 may include information indicating the orientation of the traveling body 11 or rotating body 12 of the construction machinery 10.

[0045] In addition, the server 50 stores the following location information: location information of multiple inspection target parts (indicating the location observed in the local coordinate system set in the construction machinery 10), which is set in a manner that pre-determined location information for each model of construction machinery 10 as the parts for which appearance-based inspection should be appropriately performed for each model of construction machinery 10; location information of each camera 25 mounted on each model of construction machinery 10 (indicating the location observed in the local coordinate system set in the construction machinery 10) or location information of the shooting range of each camera 25 mounted on it (indicating the location observed in the local coordinate system set in the construction machinery 10). It should be noted that the location information of each camera 25 may also include information indicating the orientation of the optical axis of the camera 25.

[0046] The following supplementary explanation is provided. The position information of each detection object part of the construction machinery 10 and the position information of each camera 25 or its shooting range of the construction machinery 10 can also be saved to the mechanical side control device 30 of the construction machinery 10, and the server 50 can appropriately obtain this position information from the mechanical side control device 30.

[0047] Furthermore, as part of the processing related to the detection of multiple target parts of the construction machinery 10, the server 50 includes the following functions: as a function of the image processing unit 50a, it displays the captured images for detection on the display 45 of the control device 40 through cooperation between the server 50 and the main control device 47 of the control device 40; as a function of the camera selection unit 50b, it selects the detection camera for capturing images for detection from the mounted camera 25 of the construction machinery 10 and the field-set camera 60 at the work site where the construction machinery 10 is located. In this case, the image processing unit 50a includes the following functions: when the construction machinery 10 appears in the captured images of the field-set camera 60 or the mounted camera 25, it determines the image portion of each target part of the construction machinery 10 appearing in the captured image; it determines the state of the target part (whether it is in a normal state or there are some abnormalities) based on the captured images of the detection camera. As a supplementary explanation, in this embodiment, the main control device 47 of the control device 40 includes a part of the functions of the image processing unit of the present invention.

[0048] Next, the inspection operations for each piece of construction machinery 10 will be described. When an inspector intends to perform an inspection (appearance-based inspection) on each piece of construction machinery 10, they notify the server 50 of the inspection requirements by performing a prescribed operation on the control device 40 or by performing a prescribed operation on an operable communication terminal. Correspondingly, the server 50 begins processing related to the inspection of that piece of construction machinery 10.

[0049] Alternatively, the server 50 may also begin processing related to the inspection of the construction machinery 10 at the moment when the prescribed inspection start conditions based on the operation plan or operation history of each construction machinery 10 are met (e.g., the moment before the start of operation of the construction machinery 10 or the moment after the end of operation, or the moment when the cumulative operation time of the construction machinery 10 reaches a predetermined time, etc.), and when the inspection operator has started the operating device 40 of the construction machinery 10 that can operate the inspection object. It should be noted that the aforementioned inspection operator is not limited to the operator of the construction machinery 10, but may also be an operator other than the operator.

[0050] At this time, the server 50 sequentially selects multiple inspection target parts of the construction machinery 10 according to the operator's operation of the control device 40 (or operation of the communication terminal). Alternatively, the server 50 sequentially selects multiple inspection target parts of the construction machinery 10 in a predetermined order. Furthermore, the server 50 performs the following selection process through the camera selection unit 50b: selecting any one of the multiple field-mounted cameras 60 at the work site of the construction machinery 10 where the inspection target is located, the multiple mounted cameras 25 of the construction machinery 10 constituting the inspection target, and the mounted cameras 25 of other construction machinery 10s existing around the construction machinery 10 constituting the inspection target as an inspection camera. This inspection camera is used to photograph the selected inspection target parts of the construction machinery 10 constituting the inspection target (hereinafter simply referred to as selecting inspection target parts). It should be noted that in this embodiment, the multiple field-mounted cameras 60 at the work site (configuration environment) of the construction machinery 10 constituting the inspection target and the mounted cameras 25 of other construction machinery 10s existing around the construction machinery 10 are equivalent to the external cameras in this invention.

[0051] The camera selection unit 50b performs the processing in the following manner. That is, the camera selection unit 50b determines the positional relationship between each on-site camera 60 and the selected target part based on the positional information of each on-site camera 60 or its shooting range at the work site of the construction machinery 10 where the target is to be detected, the positional information of the construction machinery 10 constituting the target, and the positional information of the selected target part in the construction machinery 10, or based on this positional information and measurement information related to the posture of either or both of the rotating body 12 and the working device 13 constituting the target of the construction machinery 10.

[0052] In this case, the positional relationships to be determined include, for example, the distance between each on-site camera 60 and the selected detection object, the orientation of the selected detection object relative to each on-site camera 60, and whether the selected detection object exists within the shooting range of each on-site camera 60.

[0053] Additionally, for example, the measured values ​​of the rotation angle and tilt angle of the slewing body 12 are used as measurement information related to the posture of the slewing body 12 of the construction machinery 10; for example, the measured values ​​of the swing angles of the boom 13a, stick 13b, and attachment 13c are used as measurement information related to the posture of the working device 13. Furthermore, the server 50 obtains this measurement information via the mechanical control device 30 of the construction machinery 10 constituting the object of the measurement.

[0054] In addition, the camera selection unit 50b determines the positional relationship between each mounted camera 25 on the construction machinery 10 constituting the inspection object and the selected inspection object part based on the positional information of each mounted camera 25 on the construction machinery 10 constituting the inspection object or its shooting range, and the positional information of the selected inspection object part on the construction machinery 10, or based on the positional information and measurement information related to the posture of one or both of the rotating body 12 and the working device 13 of the construction machinery 10 constituting the inspection object.

[0055] In this case, the positional relationships to be determined include, for example, the distance between each camera 25 mounted on the construction machinery 10 constituting the object of inspection and the selected object of inspection, the orientation of the selected object of inspection relative to each camera 25, and whether the selected object of inspection exists within the shooting range of each camera 25. It should be noted that, similar to determining the positional relationships between each on-site camera 60 and the selected object of inspection, measurement information related to the respective postures of the rotating body 12 and the working device 13 of the construction machinery 10 constituting the object of inspection can also be obtained.

[0056] Furthermore, the camera selection unit 50b determines the positional relationship between each camera 25 mounted on the other construction machinery 10 and the selected detection part of the construction machinery 10 constituting the detection object, based on the positional information of other construction machinery 10 surrounding the construction machinery 10 constituting the detection object, the positional information of each camera 25 mounted on the other construction machinery 10 or its shooting range, the positional information of the construction machinery 10 constituting the detection object, and the positional information of the selected detection object part on the construction machinery 10 constituting the detection object, or based on these positional information and measurement information related to the posture of either or both of the rotating body 12 and the working device 13 of the other construction machinery 10 and measurement information related to the posture of either or both of the rotating body 12 and the working device 13 of the construction machinery 10 constituting the detection object.

[0057] In this case, the positional relationships to be determined include, for example, the distance between each camera 25 mounted on the other construction machinery 10 and the selected target area of ​​the construction machinery 10 being tested, the orientation of the selected target area of ​​the construction machinery 10 being tested relative to each camera 25 mounted on the other construction machinery 10, and information on whether the selected target area exists within the shooting range of each camera 25 mounted on the other construction machinery 10. It should be noted that, similar to determining the positional relationships between each on-site camera 60 and the selected target area, measurement information related to the respective postures of the rotating body 12 and the working device 13 of the construction machinery 10 constituting the target can be obtained. This is also true for measuring the relative postures of the rotating bodies 12 and the working devices 13 of the other construction machinery 10 existing around the construction machinery 10 constituting the target.

[0058] As described above, the camera selection unit 50b determines the positional relationship between the site-mounted cameras 60 at the work site of the construction machinery 10 where the object to be detected is located, the camera 25 mounted on the construction machinery 10, and the camera 25 mounted on other construction machinery 10s existing around the construction machinery 10, and the selected object to be detected part of the construction machinery 10 constituting the object to be detected.

[0059] Then, based on the determination result of the positional relationship, the camera selection unit 50b extracts cameras capable of capturing images of the selected detection target area (cameras whose capture range includes the selected detection target area) as candidates for detection cameras (hereinafter referred to as detection camera candidates). Furthermore, the camera selection unit 50b acquires each captured image (an image whose capture range includes the selected detection target area) of the candidate detection camera. In this case, if the candidate detection camera is a camera whose optical axis orientation can be changed, the camera selection unit 50b adjusts the orientation of the candidate detection camera within the variable range of the optical axis to be as close as possible to the selected detection target area, and then acquires the captured image of the candidate detection camera.

[0060] Then, the camera selection unit 50b performs image recognition processing on the captured images of each candidate detection camera to determine whether the image of the selected detection object can be identified from the captured image, and decides the candidate detection camera with a positive determination result as the detection camera for detecting the selected detection object. In this case, when there are multiple candidate detection cameras that can acquire captured images, the multiple candidate detection cameras are decided as the detection camera, wherein the server 50 is able to identify the image of the selected detection object from the captured images.

[0061] For example, in Figure 4In the illustrated scenario, it is assumed that the connection between the boom 13a and the stick 13b of the construction machinery 10 constituting the object of inspection is selected as the inspection target location 10x. Here, three field-mounted cameras 60 (60a, 60b, 60c) are installed at the work site of the construction machinery 10 constituting the object of inspection, and no other construction machinery 10 exists around the construction machinery 10 constituting the object of inspection. Figure 4 For example, the illustration only shows a camera 25a mounted on the top of the cab 12a in a way that can take pictures of the front, as a camera 25 mounted on the construction machinery 10.

[0062] In this situation, the field-mounted cameras 60a and 60b each have a field-capturing range that includes the selected detection object 10x, while the field-mounted camera 60c does not. Furthermore, the selected detection object 10x appears in the image captured by the field-mounted camera 60a. However, because there is an obstacle between the field-mounted camera 60b and the selected detection object 10x, the selected detection object 10x does not appear in the image captured by the field-mounted camera 60b.

[0063] Furthermore, the selected detection object part 10x is included within the shooting range of the mounted camera 25a of the mounted camera 25 in the construction machinery 10, while the selected detection object part 10x is not included within the shooting range of the other mounted cameras 25 (not shown). Moreover, an image of the selected detection object part 10x appears in the image captured by the mounted camera 25a.

[0064] Under these circumstances, on-site cameras 60a and 60b and mounted camera 25a were selected as candidates for inspection cameras. Furthermore, on-site camera 60a and mounted camera 25a were selected as the inspection cameras.

[0065] As described above, the camera selection unit 50b determines the inspection camera, thereby allowing one or more cameras—including the on-site camera 60 located at the work site of the construction machinery 10 constituting the inspection target, the camera 25 mounted on the construction machinery 10, and the camera 25 mounted on other construction machinery 10 located around the construction machinery 10—that can appropriately capture a suitable (clear) image of the selected inspection target area as the inspection camera. Therefore, cameras whose inspection target area is not within the shooting range are naturally not selected as inspection cameras, and cameras that cannot properly capture images of the selected inspection target area due to obstacles or lens damage are prevented from being selected as inspection cameras.

[0066] As a supplementary explanation, when multiple candidate detection cameras exist that can capture suitable (clear) images of the selected detection target area, the candidate camera that is closest to the selected detection target area or whose distance is within a specified distance can be selected as the detection camera. For example, in Figure 4 In the situation shown, the camera 25a can also be selected as the inspection camera.

[0067] Alternatively, the detection camera can be selected from among multiple candidate detection cameras capable of capturing suitable (clear) images of the selected detection object, for example, the candidate with the smallest deviation of its optical axis direction relative to the frontal direction of the selected detection object, or within a specified range. For example, it can also be in Figure 4 In the situation shown, only the on-site camera 60a is selected as the detection camera.

[0068] After the server 50 selects the inspection camera using the camera selection unit 50b in the manner described above, it executes the processing of the image processing unit 50a. The image processing unit 50a sends the image captured by the inspection camera to the main control device 47 of the construction machinery 10 constituting the inspection object, and instructs the main control device 47 to display the image on the display 45.

[0069] Accordingly, the main control device 47 displays the images captured by the inspection camera on the display 45. In this case, when multiple inspection cameras are present, the images captured by each inspection camera can be selectively displayed on the display 45 according to the prescribed operation performed by the inspection operator on the control device 40. For example, in Figure 4 In the situation shown, as Figure 5 As shown, the images captured by the on-site camera 60, which serves as the camera for testing, are displayed on the monitor 45.

[0070] Furthermore, if the inspection camera is one whose magnification and shooting direction (optical axis direction) can be adjusted, the inspection operator can instruct the server 50 to adjust the magnification and shooting direction of the inspection camera by performing pre-defined operations on the control device 40 or by operating the communication terminal (not shown). In this case, the server 50 controls the magnification and / or shooting direction of the inspection camera according to the instructions issued by the inspection operator.

[0071] The following supplementary information is provided. For example, if the server 50 is configured to automatically operate the construction machinery 10 constituting the inspection object, the image processing unit 50a of the server 50 can also adjust the orientation of the construction machinery 10 relative to the inspection camera by performing the driving action of the driving body 11 and / or the rotation action of the rotating body 12 of the construction machinery 10 according to the requirements of the inspection operator (based on the operation requirements of the operating device 40 or the communication terminal). Alternatively, it can be configured such that the main control device 47 of the operating device 40 performs the driving action of the driving body 11 and / or the rotation action of the rotating body 12 of the construction machinery 10 according to the requirements of the inspection operator (based on the operation requirements of the operating device 40 or the communication terminal).

[0072] Furthermore, the image processing unit 50a of the server 50 determines whether the selected inspection target area is in a normal state based on the images captured by the inspection camera and pre-learning. For example, a learned model is created by pre-learning images of each inspection target area of ​​the construction machinery 10 in a normal state, and the selected inspection target area is determined to be in a normal state based on this learned model. Then, information indicating the determination result is sent to the main control device 47. At this time, the main control device 47 displays the information indicating the determination result together with the images captured by the inspection camera on the display 45 or outputs the information as sound from the speaker 44.

[0073] Based on the images captured by the inspection camera and displayed on the monitor 45 as described above, the inspection operator confirms whether there are any abnormalities such as cracks or oil leaks at the selected inspection site. At this time, the inspection operator can identify the judgment result of the image processing unit 50a of the server 50 through the display on the monitor 45 or the sound output by the speaker 44, and compare the judgment result with their own judgment result.

[0074] Therefore, the inspection operator can carefully determine whether there are any abnormalities in the selected inspection area, thus enabling a highly reliable judgment. In particular, when the judgment result of the server 50 is displayed on the monitor 45 along with the image captured by the inspection camera, the inspection operator can visually recognize the judgment result of the server 50 and carefully visually identify the image of the selected inspection area. Therefore, the operator's reliability in determining whether there are any abnormalities can be effectively improved.

[0075] Furthermore, the determination result of whether there are any abnormalities in the selected inspection target parts by the inspection operator is sent to the server 50 through the operation of the control device 40 or the operation of the communication terminal by the inspection operator, and is stored in the server 50. In this embodiment, the appearance-based inspection described above is performed sequentially on each of the multiple inspection target parts constituting the inspection target of the construction machinery 10. It should be noted that the server 50 notifies the manager of the construction machinery 10 of the determination results regarding whether there are any abnormalities in each of the multiple inspection target parts of the construction machinery 10. Alternatively, the inspection operator notifies the manager of the construction machinery 10 of the determination results via a communication terminal or the like.

[0076] According to the embodiments described above, a field-mounted camera 60 at the work site of the construction machinery 10 constituting the inspection target, or a camera 25 mounted on another construction machinery 10 located around the construction machinery 10, or a camera 25 mounted on the construction machinery 10 itself, can be used as an inspection camera. Therefore, the inspection operator does not need to install the inspection camera on the construction machinery 10, and can observe appropriate images of each of the multiple inspection target parts of the construction machinery 10 from a location far from the construction machinery 10. Furthermore, inspections based on the appearance of each of the multiple inspection target parts of the construction machinery 10 can be efficiently performed from a location far from the construction machinery 10.

[0077] Furthermore, the inspection operator can adjust the magnification and / or shooting direction of the camera's image, thus obtaining images that easily confirm the presence or absence of abnormalities in the inspected area. This increases the reliability of the inspection operator's judgment regarding the presence or absence of abnormalities. In addition, the inspection operator can access the judgment results performed by the server 50, allowing for a more careful determination of whether the inspected area has abnormalities, further enhancing the reliability of the inspection operator's judgment results.

[0078] It should be noted that the present invention is not limited to the embodiments described above, and other embodiments may also be used. Several other embodiments will be described below. In the above embodiments, the server 50 is equipped with the functions of an image processing unit 50a and a camera selection unit 50b, but the main control device 47 of the operating device 40 or a communication terminal operable by the inspection operator may also possess all or some of the functions of the image processing unit 50a and the camera selection unit 50b.

[0079] In addition, in the above embodiment, the display 45 of the control device 40 is used as a display for displaying the images captured by the inspection camera, but the display may also be, for example, a display of a communication terminal that can be operated by the inspection operator.

[0080] In addition, in the above embodiment, the on-site camera 60 and the mounted camera 25 of other construction machinery 10 that exist around the construction machinery 10 constituting the object of detection are used as external cameras in the present invention, but the external camera may also include, for example, a camera mounted on a flying body such as a drone flying at the work site.

[0081] In addition, in the above embodiment, the camera that can capture images of the selected detection object part from the candidate detection cameras extracted in the following manner is selected as the detection camera. The extraction method is as follows: the positional relationship between each of the cameras 60 set up at the work site where the construction machinery 10 constituting the detection object is configured, each of the cameras 25 mounted on the construction machinery 10, and each of the cameras 25 mounted on other construction machinery 10s around the construction machinery 10 and the selected detection object part of the construction machinery 10 constituting the detection object is extracted.

[0082] However, for example, based on the aforementioned positional relationship, each camera capable of determining that the selected detection target area is included within the shooting range can be used as a detection camera. Alternatively, for example, a camera can be used as a detection camera that acquires images from each on-site camera 60 configured at the work site where the construction machinery 10 constituting the detection target is located, each mounted camera 25 of the construction machinery 10, and each mounted camera 25 of other construction machinery 10s existing around the construction machinery 10, and performs image recognition processing on each captured image to identify the selected detection target area, resulting in an image of the selected detection target area.

[0083] Furthermore, in the above embodiments, a hydraulic excavator is exemplified as construction machinery 10, but the construction machinery in the present invention is not limited to a hydraulic excavator, and may also be construction machinery such as a crane.

[0084] As described above, the inspection system for construction machinery of the present invention is characterized by comprising: a plurality of cameras configured in the configuration environment of the construction machinery in a manner that can photograph the appearance of the construction machinery;

[0085] An image processing unit is disposed externally on the construction machinery, capable of receiving images captured by the plurality of cameras respectively; the image processing unit includes a display capable of displaying the received images; and...

[0086] The camera selection unit, when inspecting the construction machinery, selects from the plurality of cameras a camera specifically for photographing the inspection target area of ​​the construction machinery, i.e., the inspection camera.

[0087] The image processing unit is configured to receive an image captured by a selected detection camera in a manner that includes an image of the detection target area, and to display the image on the display (first aspect of the invention).

[0088] According to a first aspect of the invention, a preferred inspection camera for photographing the inspection target area can be selected from a plurality of cameras configured in the environment of the construction machinery, and the image captured by the inspection camera (including an image of the inspection target area) can be displayed on a monitor external to the construction machinery. Therefore, the inspection operator can visually confirm the status of the inspection target area based on the information displayed on the monitor. Thus, the inspection work can be carried out appropriately from a location far from the construction machinery, eliminating the need to mount the inspection camera onto the construction machinery during inspection.

[0089] In the first aspect of the invention described above, it is preferred that the plurality of cameras include cameras disposed outside the construction machinery, i.e., external cameras, and cameras pre-mounted on the construction machinery, i.e., mounted cameras (second aspect of the invention).

[0090] According to this configuration, the plurality of cameras include not only external cameras but also mounted cameras. Therefore, it is a preferred inspection camera for capturing images containing images of various inspection objects of engineering machinery.

[0091] In the first or second aspect of the invention described above, it is preferred that the detection system for the construction machinery is configured such that, when the construction machinery has multiple detection target parts, the camera selection unit selects the detection camera for each of the multiple detection target parts (third aspect of the invention).

[0092] Therefore, the preferred detection camera is one that can capture images containing images of each of the multiple detection target areas.

[0093] In the first to third aspects of the invention described above, the camera selection unit can be configured to: obtain first information indicating the positional relationship between the construction machinery and each of the plurality of cameras, or indicating the positional relationship between the construction machinery and the cameraable areas of each of the plurality of cameras, and second information indicating the position of the detection target part on the construction machinery, and select the detection camera based at least on the first information and the second information (fourth aspect of the invention).

[0094] Therefore, by selecting a detection camera based on the first and second information mentioned above, it is possible to appropriately select a detection camera capable of capturing images containing images of the parts of the object being detected.

[0095] In the first to fourth aspects of the invention described above, the camera selection unit can be configured such that: the camera selection unit acquires images captured by the plurality of cameras respectively, performs image recognition processing to identify the image of the detection target region from the respective captured images, and selects the detection camera at least based on the result of the image recognition processing (fifth aspect of the invention).

[0096] Therefore, by selecting a detection camera based on the results of the image recognition processing described above, it is possible to select a camera from among multiple cameras that can actually capture an appropriate image of the object to be detected. For example, a camera that can prevent situations where a clear image of the object to be detected cannot be captured due to lens damage, the presence of obstacles, etc., can be selected as the detection camera.

[0097] In the first to fifth aspects of the invention described above, it is preferred that the image processing unit is configured to further have the function of changing at least one of the shooting direction of the testing camera, the shooting magnification of the testing camera, and the orientation of the construction machinery when acquiring the image captured by the testing camera (sixth aspect of the invention).

[0098] Based on the above configuration, the images captured by the detection camera can include images that are preferred for confirming the state of the detection object (e.g., making a frontal image of the detection object appear near the center of the captured image, or making the image of the detection object a size that is easy to confirm).

[0099] In the first to sixth aspects of the invention described above, the image processing unit further includes the following functions: the function of determining the state of the inspection target part based on the image captured by the inspection camera in a manner that includes an image of the inspection target part; and the function of reporting information indicating the result of the determination to the inspection operator who visually confirms the display when inspecting the construction machinery (seventh aspect of the invention).

[0100] Therefore, the inspection operator can confirm the state of the inspection object part based on the image of the inspection object part in the image captured by the inspection camera displayed on the monitor, and since the result of the determination can be obtained, it is possible to prevent the inspection operator from incorrectly determining the state of the inspection object part.

[0101] In the seventh aspect of the invention described above, the image processing unit may also be configured to display information indicating the result of the determination in parallel with displaying the image captured by the detection camera in a manner that includes an image of the detection target area (eighth aspect of the invention).

[0102] Therefore, the inspection operator can confirm the state of the inspection object part based on the image of the inspection object part in the image captured by the inspection camera displayed on the monitor, and at the same time identify the result of the above determination. Thus, it is possible to efficiently confirm the state of the inspection object part based on the image.

[0103] In the first to eighth aspects of the invention described above, the camera selection unit and the image processing unit can be configured to start each process according to the fulfillment of predetermined detection start conditions (ninth aspect of the invention). This allows for the detection of engineering machinery to be performed at an appropriate time.

Claims

1. A testing system for engineering machinery, characterized in that, have: Multiple cameras are configured in the configuration environment of the construction machinery in a manner that allows them to capture the appearance of the construction machinery; An image processing unit is disposed externally on the construction machinery, capable of receiving images captured by the plurality of cameras respectively; the image processing unit includes a display capable of displaying the received images; and... The camera selection unit, when inspecting the construction machinery, selects from the plurality of cameras a camera specifically for photographing the inspection target area of ​​the construction machinery, i.e., the inspection camera. The image processing unit is configured to receive an image captured by a selected detection camera in a manner that includes an image of the detection target area, and to display the image on the display screen. The camera selection unit is configured to: acquire first information indicating the positional relationship between the construction machinery and each of the plurality of cameras, or the positional relationship between the construction machinery and the cameraable areas of each of the plurality of cameras, and second information indicating the position of the detection target part observed in a local coordinate system set in the construction machinery, and select the detection camera based at least on the first information and the second information.

2. The testing system for engineering machinery according to claim 1, characterized in that, The plurality of cameras includes cameras configured on the exterior of the construction machinery, i.e., external cameras, and cameras pre-mounted on the construction machinery, i.e. mounted cameras.

3. The testing system for engineering machinery according to claim 1 or 2, characterized in that, The inspection system for the construction machinery is configured such that, when there are multiple inspection target parts for the construction machinery, the camera selection unit selects the inspection camera for each of the multiple inspection target parts.

4. The detection system for engineering machinery according to claim 1 or 2, characterized in that, The image processing unit is configured to further have the following function: when acquiring an image captured by the inspection camera, at least one of the shooting direction of the inspection camera, the shooting magnification of the inspection camera, and the orientation of the construction machinery is changed.

5. The testing system for engineering machinery according to claim 1 or 2, characterized in that, The image processing unit further includes the following functions: determining the state of the inspection object based on an image captured by the inspection camera in a manner that includes an image of the inspection object; and reporting information indicating the determination result to the inspection operator visually confirming the display during the inspection of the construction machinery.

6. The testing system for engineering machinery according to claim 5, characterized in that, The image processing unit is configured to display information indicating the result of the determination in parallel with displaying the image captured by the detection camera in a manner that includes an image of the detection target area.

7. The detection system for engineering machinery according to claim 1 or 2, characterized in that, The camera selection unit and the image processing unit are configured to begin each process upon the fulfillment of a predetermined detection start condition.

8. A testing system for engineering machinery, characterized in that, have: Multiple cameras are configured in the configuration environment of the construction machinery in a manner that allows them to capture the appearance of the construction machinery; An image processing unit is disposed externally on the construction machinery, capable of receiving images captured by the plurality of cameras respectively; the image processing unit includes a display capable of displaying the received images; and... The camera selection unit, when inspecting the construction machinery, selects from the plurality of cameras a camera specifically for photographing the inspection target area of ​​the construction machinery, i.e., the inspection camera. The image processing unit is configured to receive an image captured by a selected detection camera in a manner that includes an image of the detection target area, and to display the image on the display screen. The camera selection unit is configured in such a way that it acquires images captured by the plurality of cameras, performs image recognition processing to identify the image of the object to be detected from the images captured by the respective cameras by determining whether the image of the object to be detected in the construction machinery appears in the respective images, and selects the detection camera at least based on the result of the image recognition processing.

Images