Work machines, information management systems, programs
The integration of imaging and communication systems in excavators improves work efficiency by automating image capture and management, addressing the need for enhanced imaging capabilities in construction machinery.
Patent Information
- Application Number
- JP2019039569
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2019-03-05
- Publication Date
- 2024-11-11
- Estimated Expiration
- 2039-03-05
AI Technical Summary
Improving work efficiency at construction sites by enhancing imaging capabilities in working machines like excavators.
Equipping excavators with an imaging device that captures two-dimensional images of the surroundings, controlled by imaging control means to ensure the inclusion of a predetermined work target, and integrating a communication system for sharing imaging conditions and images among multiple machines.
Enhances work efficiency by automating image capture and management, reducing manual effort, and enabling efficient image storage and sharing across multiple excavators.
Smart Images

Figure 0007580184000001 
Figure 0007580184000002 
Figure 0007580184000003
Abstract
Description
[Technical field]
[0001] The present invention relates to a work machine and an information management system. 、 Regarding the program. [Background technology]
[0002] 2. Description of the Related Art Conventionally, there are known work machines, such as excavators, that are equipped with a camera to photograph the surroundings (see, for example, Patent Document 1). [Prior art documents] [Patent documents]
[0003] [Patent Document 1] JP 2019-015174 A Summary of the Invention [Problem to be solved by the invention]
[0004] It is desirable to improve work efficiency at a work site.
[0005] The present invention has an object to improve the efficiency of a work site by utilizing an imaging device provided on a work machine. [Means for solving the problem]
[0006] The present invention relates to a work machine, A running body, A rotating body mounted on the traveling body; An attachment mounted on the rotating body and including a working element; An imaging means mounted on the rotating body for capturing two-dimensional images of the surroundings and acquiring the images; and an imaging control means for determining whether or not an imaging range of the imaging means includes a predetermined work object designated in advance, and for capturing an image by the imaging means when it is determined that the imaging range includes the predetermined work object. 、 The imaging control means When the imaging range of the imaging means includes the predetermined work object, When the information relating to the position of the imaging means and the information relating to the direction of the imaging means match a predetermined position and a predetermined orientation that are designated in advance, an image is taken by the imaging means.
[0007] The present invention also relates to the above-mentioned working machine. Multiple machines An information management system comprising: Multiple The number of work machines is Each of the devices has a communication device capable of transmitting and receiving information to and from the other device, The imaging conditions for capturing an image and the captured image are transmitted and received by the communication device to be shared.
[0008] The present invention also provides an information management system including the above-described work machine and an information management device capable of transmitting and receiving information between the work machine, The information management device includes: Small A plurality of images taken by at least one work machine is received, and the plurality of images are arranged in chronological order.
[0010] The present invention also provides a method for producing a method for manufacturing a semiconductor device comprising the steps of: A work machine is controlled that includes a traveling body, a rotating body mounted on the traveling body, an attachment mounted on the rotating body and including a working element, and an imaging means mounted on the rotating body for two-dimensionally photographing the surroundings and acquiring the images. A program, Computer, an imaging control means for determining whether or not an imaging range of the imaging means includes a predetermined work object designated in advance, and, when it is determined that the imaging range includes the work object, for capturing an image by the imaging means; Function as a The imaging control means When the imaging range of the imaging means includes the predetermined work object, When the information relating to the position of the imaging means and the information relating to the direction of the imaging means match a predetermined position and a predetermined orientation that are designated in advance, an image is taken by the imaging means. Effect of the Invention
[0011] According to the present invention, the imaging device provided on the work machine can be used to improve the efficiency of the work site. [Brief description of the drawings]
[0012] [Figure 1] FIG. 1 is a side view of a shovel according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a block diagram showing a system configuration of the excavator shown in FIG. 1. [Diagram 3] 10 is a flowchart showing the flow of an image capturing process. [Figure 4] 11A and 11B are diagrams for explaining imaging conditions in an image capturing process. [Diagram 5] 11A and 11B are diagrams for explaining imaging conditions in an image capturing process. [Figure 6] 11A and 11B are diagrams illustrating examples of images captured in an image capturing process. [Figure 7] FIG. 13 is a block diagram showing a system configuration of an information management system according to a modified example of an embodiment of the present invention. [Figure 8] 8 is a diagram for explaining the operation of a shovel in the information management system of FIG. 7. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0013] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings.
[0014] [Excavator configuration] First, a configuration of a shovel 100 according to this embodiment will be described. The shovel 100 is an example of a work machine according to the present invention, and is configured to be able to suitably acquire images of a work target and record the progress of work by being equipped with an image acquisition device for a work machine according to the present invention.
[0015] FIG. 1 is a side view of a shovel 100 according to this embodiment. As shown in this figure, the excavator 100 includes a lower traveling body 1, an upper rotating body 3 mounted on the lower traveling body 1 so as to be rotatable via a rotating mechanism 2, a boom 4, an arm 5, and a bucket 6 as attachments, and a cabin 10 for an operator to ride in. The attachment is not limited to this as long as a working element (e.g., a bucket, a crusher, a crane device, etc.) is provided.
[0016] The lower traveling body 1 includes, for example, a pair of left and right crawlers, and each of the crawlers is hydraulically driven by a traveling hydraulic motor (not shown) to cause the excavator 100 to travel. The upper rotating body 3 rotates relative to the lower traveling body 1 by being driven by a rotating hydraulic motor or an electric motor (both not shown) or the like.
[0017] The boom 4 is pivotally attached to the front center of the upper rotating body 3 so as to be able to tilt up and down, an arm 5 is pivotally attached to the tip of the boom 4 so as to be able to rotate up and down, and a bucket 6 is pivotally attached to the tip of the arm 5 so as to be able to rotate up and down. The boom 4, arm 5, and bucket 6 are hydraulically driven by a boom cylinder 7, an arm cylinder 8, and a bucket cylinder 9, respectively. The cabin 10 is a cockpit where an operator sits, and is mounted, for example, on the front left side of the upper rotating body 3.
[0018] FIG. 2 is a block diagram showing the system configuration of the shovel 100. As shown in this figure, in addition to the above configuration, the shovel 100 is equipped with a controller 30, an imaging device 40, a movement / posture state sensor 42, a position sensor 43, an orientation sensor 44, an operation device 45, a display device 50, an audio output device 60, and a communication device 80. The image acquisition device according to the present invention is configured to include the controller 30, the imaging device 40, the movement / posture state sensor 42, the position sensor 43, the orientation sensor 44, and the communication device 80.
[0019] The imaging device 40 is attached to the upper part of the upper rotating body 3, and captures images of the periphery of the excavator 100 and outputs the images to the controller 30. The imaging device 40 includes a rear camera 40B, a left side camera 40L, and a right side camera 40R. The rear camera 40B is attached to the upper rear end of the upper rotating body 3 and captures an image of the area behind the upper rotating body 3. The left side camera 40L is attached to the upper left end of the upper rotating body 3 and captures an image of the left side of the upper rotating body 3. The right side camera 40R is attached to the upper right end of the upper rotating body 3 and captures images of the right side of the upper rotating body 3. Each of the rear camera 40B, left side camera 40L and right side camera 40R is mounted on the upper part of the upper rotating body 3 with its optical axis facing diagonally downward, and has an imaging range (angle of view) in the vertical direction that includes the ground near the shovel 100 to far away from the shovel 100. The rear camera 40B, the left side camera 40L, and the right side camera 40R may be attached to the upper surface of the upper rotating body 3. These cameras 40B, 40L, and 40R may be attached so as not to protrude from the side end of the upper rotating body 3, and so that their optical axes face diagonally downward so that a part of the upper rotating body 3 is captured. Since the target to be imaged and a part of the upper revolving body 3 are simultaneously captured as an image, it can be intuitively determined that the captured image is an image captured using a camera provided on the shovel 100. In addition, the sense of distance between the target to be imaged and the shovel 100 can also be intuitively understood.
[0020] The motion / posture state sensor 42 is a sensor that detects the motion state and posture state of the excavator 100, and outputs the detection result to the controller 30. The motion / posture state sensor 42 includes a boom angle sensor, an arm angle sensor, a bucket angle sensor, a three-axis inertial sensor (IMU: Inertial Measurement Unit), a swing angle sensor, and an acceleration sensor. These sensors may be composed of sensors that acquire rotational information such as stroke sensors of cylinders on the boom, etc., or rotary encoders, or may be replaced by acceleration (which may also include speed and position) acquired by an IMU. The arm angle sensor detects the rotation angle of the arm 5 relative to the boom 4 (hereinafter referred to as the "arm angle"). The bucket angle sensor detects the rotation angle of the bucket 6 with respect to the arm 5 (hereinafter referred to as the "bucket angle"). The IMUs are attached to each of the boom 4 and the arm 5, and detect the acceleration of the boom 4 and the arm 5 along three predetermined axes, and the angular acceleration of the boom 4 and the arm 5 around the three predetermined axes. The rotation angle sensor detects a rotation angle based on a predetermined angular direction of the upper rotating body 3. However, this is not limited to this, and the rotation angle may be detected based on a GPS or IMU sensor provided on the upper rotating body 3. The acceleration sensor is attached at a position away from the rotation axis of the upper rotating body 3, and detects the acceleration at that position of the upper rotating body 3. As a result, it can be determined whether the upper rotating body 3 is rotating or the lower traveling body 1 is traveling, etc., based on the detection result of the acceleration sensor.
[0021] The position sensor 43 is a sensor that acquires information on the position (current position) of the shovel 100, and is a GPS (Global Positioning System) receiver in this embodiment. The position sensor 43 receives a GPS signal including information on the position of the shovel 100 from a GPS satellite, and outputs the acquired position information of the shovel 100 to the controller 30. Note that the position sensor 43 does not have to be a GPS receiver as long as it can acquire information on the position of the shovel 100, and may be one that uses a satellite positioning system other than GPS, for example. The position sensor 43 may be provided on the lower traveling body 1 or on the upper rotating body 3.
[0022] The orientation sensor 44 is a sensor that acquires information on the orientation (direction) in which the shovel 100 is facing, and is, for example, a geomagnetic sensor. The orientation sensor 44 acquires the orientation information of the shovel 100 and outputs it to the controller 30. Note that the orientation sensor 44 is not particularly limited in terms of the type of sensor, etc., as long as it can acquire the orientation information of the shovel 100. For example, two GPSs may be provided, and the orientation information may be acquired from the position information of the GPS.
[0023] The operation device 45 is provided near the cockpit of the cabin 10, and is an operation means by which the operator operates each operation element (the lower traveling body 1, the upper rotating body 3, the boom 4, the arm 5, the bucket 6, etc.). In other words, the operation device 45 is an operation means for operating each hydraulic actuator that drives each operation element. The operation device 45 includes, for example, levers, pedals, various buttons, etc., and outputs operation signals according to the contents of these operations to the controller 30. In addition, the operation device 45 is an operation means including various setting means for operating the imaging device 40, the motion / posture state sensor 42, the position sensor 43, the orientation sensor 44, the display device 50, the audio output device 60, the communication device 80, etc., and outputs operation commands for each of these parts to the controller 30.
[0024] The display device 50 is provided near the cockpit in the cabin 10, and displays various types of image information to be notified to the operator under the control of the controller 30. The display device 50 is, for example, a liquid crystal display or an organic EL (Electroluminescence) display, and may be a touch panel type that also serves as at least a part of the operation device 45.
[0025] The audio output device 60 is provided near the cockpit in the cabin 10, and outputs various types of audio information to be notified to the operator under the control of the controller 30. The audio output device 60 is, for example, a speaker or a buzzer.
[0026] The communication device 80 is a communication device that transmits and receives various information with remote external devices, other shovels 100, etc. through a specified communication network (e.g., a mobile phone network or an Internet network ending in a base station) based on a specified wireless communication standard.
[0027] The controller 30 is a control device that controls the operation of each part of the shovel 100 to perform drive control of the shovel 100. The controller 30 is mounted inside the cabin 10. The functions of the controller 30 may be realized by any hardware, software, or a combination thereof, and is configured, for example, mainly with a microcomputer including a CPU, RAM, ROM, I / O, etc.
[0028] The controller 30 also includes, as functional units for executing various functions, an object detection unit 301, a state detection unit 302, an information acquisition unit 303, an image recording unit 304, and an information communication unit 305. The controller 30 also includes a storage unit 310 as a storage area defined in an internal memory such as an EEPROM (Electrically Erasable Programmable Read-Only Memory).
[0029] The target detection unit 301 detects a detection target around the shovel 100 based on an image captured by the imaging device 40. Specifically, the target detection unit 301 recognizes the detection target in the captured image and specifies the actual position and dimensions of the recognized detection target by applying various known image processing methods, a machine learning-based classifier, or the like. The detection target is a work target of the shovel 100. However, the detection target may also include people such as workers, other work machines or vehicles, construction materials temporarily placed, and the like.
[0030] The state detection unit 302 acquires information related to various states (operation state, control state, etc.) of the shovel 100 from various sensors (e.g., the imaging device 40, the operation / posture state sensor 42, the position sensor 43, the orientation sensor 44, the operation device 45, etc.), various actuators (e.g., a solenoid valve that performs hydraulic control), and various control devices (e.g., other functional units of the controller 30, etc.) mounted on the shovel 100. For example, the state detection unit 302 can acquire the position and orientation of the shovel 100 by the position sensor 43 and the orientation sensor 44, and can detect the excavation operation of the shovel 100 by detecting the operation and load of the bucket 6.
[0031] The information acquisition unit 303 acquires work-related information related to work performed by the shovel 100, and stores it in the storage unit 310. The work-related information includes work information, date and time information, weather information, position information, machine information, operator information, and the like. The work information may include information such as the name of the work (construction), the work location, the work content, the owner of the excavator, the subcontractor of the work, the intermediary, the end user related to the work, etc. The information acquisition unit 303 acquires the work information based on, for example, an input operation by the operator through the operation device 45. The date and time information includes a date, a day of the week, and a time. The information acquisition unit 303 acquires the date and time information from a clock means (for example, a real time clock (RTC)) in the controller 30. The weather information is information about the weather at the location and date and time where the shovel 100 is working, and includes information about weather classifications such as sunny, cloudy, rainy, and snowy. The information acquisition unit 303 acquires desired weather information from a weather-related server or website via the communication device 80. Alternatively, the information acquisition unit 303 may include an illuminance sensor, a raindrop detection sensor, etc., and acquire weather information based on the illuminance, the presence or absence of rain, etc. output by these sensors. The location information is information on the location of the shovel 100, and includes information on the longitude and latitude. The location information may further include altitude information, or may be geocode information such as GeoHash. The information acquisition unit 303 acquires the location information by the location sensor 43. The machine information is identification information of the shovel 100 for identifying the shovel 100, and is, for example, a predefined machine number, a shovel ID, etc. The information acquisition unit 303 acquires the machine information, for example, by reading out the machine number, etc., prerecorded in the storage unit 310, etc. The operator information is identification information of the operator operating the shovel 100, and is a predetermined operator ID or the like. The information acquisition unit 303 acquires the operator information based on, for example, an input operation performed by the operator through the operation device 45. Work-related information (work information, date and time information, weather information, location information, aircraft information, operator information, etc.) may be obtained by inputting information through the operation device 45 (either by directly inputting information or by selecting information from preset information) or automatically using communication technology or information processing technology. The work-related information may be stored in association with one another.
[0032] The image recording unit 304 records the image of the detection target (work target) acquired by the imaging device 40 in the storage unit 310 as target image information 3101. At this time, the image recording unit 304 may embed work-related information, figures showing the distance and dimensions of the work target, etc., in the image to be recorded.
[0033] When a predefined notification condition is met, the information communication unit 305 controls the communication device 80 to transmit and receive predetermined information to and from external devices and other excavators 100 .
[0034] [Shovel movement] Next, the operation of the shovel 100 when performing image capturing processing for capturing images and recording the work progress will be described. If the work progress can be acquired using the imaging device 40 provided in the shovel 100, it is not necessary to prepare a separate imaging device, and the process of manually capturing images of the work progress can be omitted, improving the work efficiency at the site. FIG. 3 is a flowchart showing the flow of this image capturing process, FIGS. 4 and 5 are diagrams for explaining the imaging conditions in the image capturing process, and FIG. 6 is a diagram showing an example of a captured image.
[0035] The image capturing process is carried out by the controller 30 executing a program stored in an internal storage device on a CPU. When the image capturing process is executed, as shown in FIG. 3, first, the controller 30 sets conditions related to the position of the shovel 100 and the direction of the imaging device 40 as imaging conditions for capturing an image in advance of the work (step S1). Specifically, as shown in FIG. 4(a), the controller 30 sets the imaging position P and imaging direction D of the imaging device 40 that captures an image based on the operation of the operator. The imaging position P is set based on information on longitude and latitude (or further altitude), and the imaging direction D is set based on information on azimuth angle (or further elevation angle). The imaging position P and imaging direction D may be relative positions and relative directions with respect to a reference absolute position and absolute direction. In addition, the imaging position P and imaging direction D can be set to any range (for example, the position is within a radius of 5 m, and the direction is within an azimuth of 5 degrees, etc.). The operator appropriately sets the imaging position P and imaging direction D so that the work target T of the shovel 100 is included in the image. The controller 30 records the imaging conditions including the set imaging position P and imaging direction D in the storage unit 310. The operator can set the position and direction on a specified map. In this case, the operator can specify the location where he / she wants to check the work progress while overlooking the entire map. The site supervisor or the manager of the work site may also specify the location. In this case, the location on the map can be specified outside the excavator 100 (on a touch panel, a management terminal, etc.), and the information can be transmitted to the excavator 100.
[0036] The imaging conditions set in step S1 are not limited to those including the imaging position P and imaging direction D of the imaging device 40. The imaging conditions may be such that an image is captured by the imaging device 40 when the imaging range of the imaging device 40 includes a predetermined subject area designated in advance around the shovel 100. The imaging range of the imaging device 40 includes the subject area when at least one of the position and direction of the imaging device 40 coincides with a predetermined position or a predetermined orientation designated in advance (including being included within a predetermined range). With these imaging conditions, an image of the desired subject area including the work target can be captured. The setting of the photographing conditions may be performed in the same manner as the setting of the imaging position P and the imaging direction D described above, or may be performed by specifying a desired subject area B on a map M displayed on the display device 50, for example, as shown in FIG. 5. In this case, the imaging position P of the imaging device 40 may or may not be specified. Note that, although a fan-shaped subject area B is illustrated in the example of FIG. 5, the shape of the subject area B is not particularly limited. The subject area B may be specified outside the shovel 100 (on a touch panel, a management terminal, etc.), and the information may be transmitted to the shovel 100.
[0037] Furthermore, there are no particular limitations on the position of the shovel 100 or the direction of the imaging device 40 when the conditions are set in step S1. For example, the imaging conditions may be set in a state where the shovel 100 is located away from the imaging position P. However, it is preferable that the image captured by the imaging device 40 is displayed on the display device 50, the operator actually checks the work target T to be photographed through the display device 50, and then the position of the shovel 100 and the direction of the imaging device 40 at that time are acquired by the position sensor 43 and the orientation sensor 44, and set as the imaging position P and the imaging direction D.
[0038] Further, the imaging conditions may include conditions other than the imaging position P and the imaging direction D of the imaging device 40 in addition to the imaging position P and the imaging direction D. For example, a time condition may be added, such as allowing an image of the same work target T (subject area) to be captured again on the condition that a predetermined time has passed since the previous image was captured. Alternatively, an action condition may be added, such as allowing an image of the same work target T (subject area) to be captured again on the condition that a predetermined action (e.g., a predetermined number of excavation actions) of the shovel 100 has been detected since the previous image was captured. Furthermore, these time conditions and action conditions may be combined.
[0039] Next, the controller 30 drives the shovel 100 to start work (step S2). In this embodiment, the shovel 100 performs excavation work on the work object T.
[0040] Next, the controller 30 acquires the position and direction of the imaging device 40 in the shovel 100 during work (step S3). Specifically, the state detection unit 302 of the controller 30 acquires the position of the imaging device 40 by the position sensor 43, and acquires the direction of the imaging device 40 by the orientation sensor 44. The position of the imaging device 40 refers to the respective positions of the rear camera 40B, the left side camera 40L, and the right side camera 40R, and the position of each of these cameras fixed in a known position relative to the shovel 100 (upper rotating body 3) is acquired by acquiring the position of the position sensor 43 in the shovel 100 by the position sensor 43. The direction of the imaging device 40 refers to the respective directions of the rear camera 40B, the left side camera 40L, and the right side camera 40R, and the direction of the shovel 100 (upper rotating body 3) is acquired by acquiring the direction of the shovel 100 (upper rotating body 3) by the orientation sensor 44.
[0041] Next, the controller 30 determines whether or not the position of the shovel 100 and the direction of the imaging device 40 acquired in step S3 satisfy the imaging condition set in step S1 (step S4). Specifically, the controller 30 judges whether or not the position and direction of the imaging device 40 match the imaging position P and imaging direction D, which are the imaging conditions. That is, it is judged whether or not the position of any imaging device 40 is at the imaging position P and the direction of the imaging device 40 (in this embodiment, the optical axis of the camera shown by the dashed line in FIG. 4) faces the imaging direction D. When a range is set for the imaging position P and the imaging direction D, if the position and direction of the imaging device 40 are within this range, it is judged that they match the imaging position P and the imaging direction D. Then, if it is determined that at least one of the acquired position and direction of the imaging device 40 does not satisfy the imaging conditions (step S4; No), the controller 30 transitions to the processing of the above-mentioned step S2 and continues the operation until the imaging conditions are satisfied.
[0042] In addition, in the case where conditions other than the imaging position P and the imaging direction D are set as the imaging conditions in the above-mentioned step S1, a determination is also made regarding those conditions in this step S4. For example, if a condition of the time elapsed since the previous image capture is set, the controller 30 counts the time elapsed since the previous image capture using, for example, an RTC, and judges whether or not the elapsed time has exceeded a predetermined time. However, it goes without saying that this judgment is not necessary when capturing an image for the first time. Alternatively, in a case where an operation condition is set such that an image is taken when a predetermined operation of the shovel 100 (for example, a predetermined number of excavation operations since the previous image was taken) is detected, the controller 30 determines whether or not the predetermined operation of the shovel 100 is detected by the operation / posture state sensor 42.
[0043] Moreover, it is preferable that the determination of the imaging conditions in step S4 (particularly the determination of the imaging direction D) is performed while the upper rotating body 3 is rotating after work is performed with the attachment. Furthermore, when recording the results after the work is completed, it is preferable to take pictures using a camera on the side opposite to the direction of rotation of the upper rotating body 3 after excavation. This allows the work target T after work to be reliably captured and photographed. However, this is not limited to the case where the attachment or the upper rotating body 3 is equipped with an imaging device that captures images of the front. Moreover, since the timing for taking images is preferably when the upper rotating structure 3 is stopped, this may be incorporated into the imaging conditions as an operating condition of the excavator 100. Furthermore, if the condition is that the lower traveling structure 1 or the attachment is stopped, no vibration due to operation is generated in the upper rotating structure 3, and clear images can be obtained.
[0044] In step S4, if it is determined that the position and direction of the imaging device 40 satisfy the imaging conditions (step S4; Yes), the controller 30 captures an image using any camera of the imaging device 40 that satisfies the imaging conditions (the left side camera 40L in FIG. 4(b)), as shown in FIG. 4(b) (step S5).
[0045] Then, the controller 30 records the image acquired in step S5 in the storage unit 310 as the target image information 3101 (step S6). At this time, as shown in Fig. 6(a), the image recording unit 304 of the controller 30 embeds work-related information related to the work performed by the shovel 100 in the image as necessary, and records the image in the storage unit 310. In this case, the image recording unit 304 of the controller 30 acquires the work-related information to be recorded together with the image by the information acquisition unit 303 based on, for example, the operation of the operator, and records the image in which the work-related information is embedded in the storage unit 310. In the example of Fig. 6(a), in addition to the construction name, work content, date, and work (shovel) ID, a figure R that indicates the distance to the work object T and the dimensions of the work object T is embedded in the image as the work-related information. When the figure R is used, the controller 30 detects the work object T from the image by the object detection unit 301, and acquires the distance to the work object T and its dimensions.
[0046] 6(b), the image recording unit 304 of the controller 30 may collectively record in the storage unit 310 an image (still image) of the work object T that has been photographed and an image (still image) of the work object T that has been photographed in the past in chronological order as target image information 3101. Alternatively, the images (still images) may be arranged in chronological order to generate a video, and this video may be recorded in the storage unit 310 as target image information 3101. This makes it possible to acquire a series of images or videos that show the progress of the work (changes in the work object T over time).
[0047] Next, the controller 30 judges whether or not the work by the shovel 100 has been completed (step S7), and if it is judged that the work has not been completed (step S7; No), the process proceeds to the above-mentioned step S2, and the work continues. As a result, until it is judged that the work has been completed, the acquisition of the position and direction of the imaging device 40, the judgment of whether or not these satisfy the imaging conditions, and the photographing and recording of images when the imaging conditions are satisfied are sequentially executed and repeated. Furthermore, when it is determined that the work is completed, for example, when the power source such as the engine of the shovel 100 is stopped by the operator and at least the actuators of the attachments or the like are not moved even when operated (step S7; Yes), the controller 30 ends the image capturing process. Note that, when a power storage means is provided, it is also possible to start up the imaging device 40 and the controller 30 and execute the above-mentioned process in a state in which the actuators are not moved even when operated.
[0048] [Technical effect of the present embodiment] As described above, according to this embodiment, when the imaging range of the imaging device 40 includes the subject area B designated in advance around the shovel 100, the imaging device 40 captures an image. This makes it possible to obtain the work progress by utilizing the imaging device 40 provided in the shovel 100. This eliminates the need to prepare a separate imaging device and also omits the process of manually capturing images of the work progress, thereby improving the work efficiency at the site.
[0049] Furthermore, by adding a time condition that an image is captured when a predetermined time has elapsed since the previous image was captured, the frequency of image capture over time can be regulated. This makes it possible to avoid a situation in which the same location is photographed frequently as the upper rotating body 3 rotates, for example, and to reduce the capacity of recorded images.
[0050] Furthermore, by adding an operation condition that an image is captured when a specified operation of the shovel 100 is detected, images can be acquired not only in accordance with the position and posture of the shovel 100, but also in accordance with the operation of the shovel 100. This makes it possible to, for example, capture images after each predetermined number of excavation operations, or capture images after the upper rotating body 3 has reliably stopped.
[0051] Furthermore, since work-related information relating to the work can be acquired and embedded in the captured image, the amount of information in the image can be increased. Furthermore, by arranging the captured images in chronological order to create a video, it is possible to obtain a video showing the progress of the work (changes in the work object T over time).
[0052] [Variations] Next, a modification of the above embodiment will be described. This modified example differs from the above embodiment in that images taken by a plurality of shovels 100 are stored in a management server. Note that the same components as those in the above embodiment are given the same reference numerals, and the description thereof will be omitted.
[0053] FIG. 7 is a block diagram showing a system configuration of an information management system 200 in this modified example. As shown in this figure, the information management system 200 is configured to include a plurality of shovels 100 (in this modified example, 100A, 100B, 100C, 100D, ...) configured similarly to one another, a management server 400, and an information terminal 500. Note that in Fig. 7, the configurations of the shovels 100 (100B, 100C, 100D) other than the shovel 100A are omitted from the illustration.
[0054] The multiple shovels 100 are capable of sending and receiving various information to each other via the communication network 150 using the communication devices 80 each equipped with, and are also capable of sending and receiving various information to and from the management server 400 and the information terminal 500 via the communication network 150.
[0055] The management server 400 is an example of an information management device according to the present invention, and is, for example, a server device installed in a management center or the like that is provided outside a work site where a plurality of excavators 100 are working. This management server 400 may be an in-house server operated by the business operator that operates the information management system 200 or an associated business operator thereof, or may be a so-called cloud server.
[0056] Specifically, the management server 400 includes a communication device 410 and a control device 420 . The communication device 410 is capable of transmitting and receiving various types of information to and from each of the multiple shovels 100 via the communication network 150. The control device 420 controls various operations in the management server 400. The control device 420 includes a storage unit 421 as a storage area defined in an internal memory such as an EEPROM, and stores and manages various information in the storage unit 421. For example, the storage unit 421 stores (accumulates) images (target image information 3101) captured by each shovel 100 and transmitted as target image information 4210.
[0057] The information terminal 500 is, for example, a mobile terminal such as a tablet or a smartphone owned by a user. The user can access and view various work records in the information management system 200 through the information terminal 500. The information terminal 500 may be a stationary or mobile computer terminal. Specifically, information terminal 500 includes a communication device 510, a display device 520, and a control device 530. The communication device 510 is capable of transmitting and receiving various information to and from each of the multiple shovels 100 and the management server 400 through the communication network 150. The display device 520 is, for example, a liquid crystal display or an organic EL (Electroluminescence) display, and may be a touch panel type that also serves as an operation means. The control device 530 controls various operations in the information terminal 500.
[0058] FIG. 8 is a diagram for explaining the operation of the shovel 100 in the information management system 200. In the information management system 200, a plurality of shovels 100 share the same imaging conditions as in the above embodiment through direct (or joint) communication via the management server 400. Then, while each of the plurality of shovels 100 operates in a manner substantially similar to the above embodiment, the shovels 100 transmit images (target image information 3101) captured by each of the shovels 100 to the management server 400, and all images are stored in the management server 400.
[0059] For example, as shown in Fig. 8, the shovel 100A photographs the work target T with soil piled up before work at time t1 with the rear camera 40B (Fig. 8(a)), and then photographs the work target T with the right side camera 40R after excavation has progressed at time t2 (Fig. 8(b)). The photographed images are sequentially transmitted from the shovel 100A to the management server 400 by the communication device 410. Thereafter, another shovel 100B photographs the work target T in a state where excavation has been completed at time t3 with the left side camera 40L (FIG. 8(c)), and then photographs the work target T in a state where it has been leveled at time t4 with the right side camera 40R (FIG. 8(d)). These images are also sequentially transmitted from the shovel 100B to the management server 400 by the communication device 410. In this way, images taken by a plurality of different shovels 100 are stored and managed in the management server 400. The stored plurality of images may be organized by the management server 400 into a time-series series of images or a video.
[0060] A user who owns the information terminal 500 can view images stored in the management server 400 through the communication network 150. In this case, the control device 530 of the information terminal 500 receives a plurality of images from the management server 400 through the communication network 150, stores (or temporarily stores) them in, for example, a storage unit (not shown), and causes the display device 520 to display the plurality of images in a list or continuously.
[0061] The timing for transmitting images from each shovel 100 to the management server 400 is not particularly limited, and may be after a series of operations is completed, for example. Furthermore, embedding of work-related information into an image may be performed by each shovel 100 where the image was captured, or may be performed by the management server 400. When embedded by the management server 400, the work-related information may be transmitted from the shovel 100 to the management server 400, or a configuration capable of acquiring the work-related information (information acquisition unit 303 in the shovel 100) may be provided in the management server 400.
[0062] [others] Although the embodiment of the present invention has been described above, the present invention is not limited to the above embodiment or its modifications. For example, in the above embodiment, the imaging conditions are set before the work is performed, but the conditions may be set during a break in the work, for example by temporarily interrupting the work. In addition, there may be multiple imaging conditions, for example, different work objects may be photographed, or the same work object may be photographed from multiple different angles (directions), etc. When the same work object is photographed from multiple different angles, a 3D image may be generated using the acquired images.
[0063] Furthermore, in the modified example of the above embodiment, images are stored in the management server 400, but a plurality of shovels 100 may transmit and receive imaging conditions and images to each other and share them without using the management server 400. Alternatively, a plurality of shovels 100 may share only imaging conditions, and each may hold acquired images.
[0064] Furthermore, the present invention is not limited to capturing an image using an imaging device, but may also be adapted to read out a recorded image or extract specific information from an image. For example, the past image may be viewable by moving to the image capturing position where the image was captured in the past (or by taking a position and posture corresponding to the past image, such as at least matching the direction to that of the image in question). Viewing may be done from the monitor of the work machine, or from a management server or information terminal. Alternatively, a specific subject or event may be detected from collected images (or generated video), and the image, location, time, etc. may be searched for. A specific subject is, for example, an embankment made during embankment work, or a hole made during excavation. An event may include, for example, the operation of the work machine itself, such as "digging was performed," "turned," or "discharged," as well as information on the work machine or work site, such as the approach of a truck. To enable such a search, it is sufficient to store the acquired image in association with other information (for example, the name of the work machine owner, the renter, the name of the construction site, the name of the contractor, the date and time of shooting, etc.). In this way, images in which a specific subject or event is captured can be identified and sorted by the image analysis means, and used for various analytical information.
[0065] Furthermore, the work machine according to the present invention may be a work machine other than a shovel, for example, a wheel loader, an asphalt finisher, a forklift, a crane, or the like. Moreover, the information management system according to the present invention may include these different work machines. In addition, the details shown in the embodiment can be modified as appropriate without departing from the spirit of the invention. [Explanation of symbols]
[0066] 1 Undercarriage 2 Swivel mechanism 3. Upper rotating body 30 Controller 40 Imaging device 40B Rear camera 40L Left side camera 40R Right side camera 42 Motion and Posture Sensor 43 Position Sensor 44 Orientation Sensor 80 Communication Equipment 100, 100A, 100B Shovel 150 Communication Network 200 Information Management System 310 Storage section 400 Management Server 500 Information terminal 3101 Target image information 4210 Target image information B Subject area D Imaging direction P Imaging position T Work target
Claims
1. A running body, A rotating body mounted on the traveling body; An attachment mounted on the rotating body and including a working element; An imaging means mounted on the rotating body for capturing two-dimensional images of the surroundings and acquiring the images; an imaging control means for determining whether or not an imaging range of the imaging means includes a predetermined work object designated in advance, and for capturing an image by the imaging means when it is determined that the imaging range includes the predetermined work object; The imaging control means When the imaging range of the imaging means includes the predetermined work object, When information related to the position of the imaging means and information related to the direction of the imaging means match a predetermined position and a predetermined orientation designated in advance, an image is captured by the imaging means. Working machinery.
2. The imaging control means causes the imaging means to again image the predetermined work object on the condition that a predetermined time has elapsed since the imaging means previously imaged the predetermined work object.
2. The work machine of claim 1.
3. a motion detection means for detecting a motion of the work machine, the imaging control means causes the imaging means to again image the predetermined work target on condition that a predetermined operation of the work machine has been detected by the operation detection means after the imaging means previously imaged the predetermined work target. A work machine according to claim 1 or 2.
4. an information acquisition means for acquiring work-related information related to work performed by the work machine; and an image processing means for embedding the work-related information in the image captured by the imaging means. A work machine according to any one of claims 1 to 3.
5. An image processing means for arranging the multiple images captured by the imaging means in chronological order, A work machine according to any one of claims 1 to 4.
6. An information management system including a plurality of work machines according to any one of claims 1 to 5, The plurality of work machines include Each of the devices has a communication device capable of transmitting and receiving information to and from the other device, The imaging conditions for capturing an image and the captured image are transmitted and received by the communication device to each other, thereby sharing the image. Information management system.
7. 6. An information management system comprising the work machine according to claim 1 and an information management device capable of transmitting and receiving information between the work machine, the information management device receives a plurality of images taken by at least one work machine and arranges the plurality of images in chronological order; Information management system.
8. A program for controlling a work machine having a running body, a rotating body mounted on the running body, an attachment mounted on the rotating body and including a working element, and an imaging means mounted on the rotating body for taking two-dimensional photographs of the surroundings and acquiring images thereof, comprising: Computer, an imaging control means for determining whether or not an imaging range of the imaging means includes a predetermined work object designated in advance, and, when it is determined that the imaging range includes the work object, for capturing an image by the imaging means; Function as a The imaging control means When the imaging range of the imaging means includes the predetermined work object, When information related to the position of the imaging means and information related to the direction of the imaging means match a predetermined position and a predetermined orientation designated in advance, an image is captured by the imaging means. program.
Citation Information
Patent Citations
Three-dimensional image creation device, method, and program
JP2013021538A
Shape measurement system
JP2018146594A
Shovel
JP2019015174A
Work machine
JP2020033704A
Construction machine and construction management system
WO2017061515A1
