Information processing device and control program for information processing device

Abstract

To suitably perform an operation control of a target using map information.SOLUTION: A shovel 100 comprises an imaging device 40 that acquires peripheral information, a range sensor 41, and a controller 30. Based on the acquired information, the controller 30 forms a local OGM 73 in which the acquired information is displayed on a map; based on the local OGM 73, the controller forms a global OGM 74 in which information in an area wider than an area of the local OGM 73 is displayed on a map; and based on the local OGM 73, the controller controls an operation of the shovel 100.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to an information processing device and a control program therefor. [Background technology]

[0002] Conventionally, there is known a technique for acquiring information about the surroundings of a target using a camera or a physical sensor. For example, the technology described in Patent Document 1 creates map information such as Occupancy Grid Maps (OGM) that show the probability of objects existing around a target (vehicle), and uses these maps to control the target's movement. The OGM is divided into grids, and the surrounding situation is determined by showing the probability of objects existing in each grid. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2011-123551 Summary of the Invention [Problem to be solved by the invention]

[0004] However, if a map is simply created that covers an area including the periphery of the target, it may not be possible to appropriately control the operation using this map. For example, when the target is a work machine, a finer grid size is desirable so that even delicate work can be detected, and omnidirectional information is required, including information from the front, back, left, right, and above. This increases the calculation load and takes time, which can lead to the risk of not being able to control the operation properly. This risk is particularly pronounced in small computers used in vehicles or in edge computing.

[0005] The present invention has been made in view of the above circumstances, and has as its object to suitably control the operation of an object using map information such as OGM. [Means for solving the problem]

[0006] The present invention provides an information processing device, information acquisition means for acquiring information about the surroundings of a target; a first map creation means for creating first map information that displays the information in a first area as a map based on the information acquired by the information acquisition means; a second map creation means for creating second map information based on the first map information, the second map information being a map display of the information in a second area larger than the first area; an action control means for controlling the action of the object based on the first map information; Equipped with 、 the first map creation means creates a plurality of pieces of first map information having different areas based on information acquired by a plurality of pieces of information acquisition means having different information acquisition ranges; The second map creation means creates the second map information based on the plurality of first map information. The composition was as follows.

[0007] The present invention also provides a control program for an information processing device including an information acquisition unit that acquires information about the surroundings of a target, the control program comprising: Computer, a first map creation means for creating first map information that displays the information in a first area as a map based on the information acquired by the information acquisition means; a second map creation means for creating second map information based on the first map information, the second map information being a map display of the information in a second area larger than the first area; an action control means for controlling the action of the object based on the first map information; Function as 、 the first map creation means creates a plurality of pieces of first map information having different areas based on information acquired by a plurality of pieces of information acquisition means having different information acquisition ranges; The second map creation means creates the second map information based on the plurality of first map information. I did so. [Effects of the Invention]

[0008] According to the present invention, the movement of an object can be suitably controlled using map information. [Brief explanation of the drawings]

[0009] [Figure 1]FIG. 1 is a side view of a shovel according to an embodiment of the present invention. [Figure 2] FIG. 2 is a block diagram showing a system configuration of the excavator in FIG. 1. [Figure 3] FIG. 3 is a data flow diagram showing the flow of data in the object detection process of the present embodiment. [Figure 4] 2 is a diagram showing an example of a first OGM, a second OGM, and a local OGM of the present embodiment. FIG. [Figure 5] 2A and 2B are diagrams illustrating an example of a local OGM and a global OGM according to the present embodiment. [Figure 6] 10A and 10B are diagrams for explaining a modified example of the object detection process of the present embodiment. [Figure 7] 10A and 10B are diagrams for explaining the relationship between the tilt angles of the imaging device and distance sensor of the present embodiment and the position and range of the local OGM. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[0011] [Excavator configuration] First, a description will be given of the configuration of the shovel 100 according to this embodiment. The shovel 100 is configured to be equipped with the information processing device according to the present invention, and thereby to be able to suitably control its operation using map information.

[0012] FIG. 1 is a side view of a shovel 100 according to this embodiment. As shown in this figure, the excavator 100 includes a lower traveling body 1, an upper rotating body 3 mounted on the lower traveling body 1 so as to be rotatable via a rotating mechanism 2, a boom 4, an arm 5, and a bucket 6 as attachments, and a cabin 10 in which an operator rides. The attachment is not limited to this, as long as it is provided with a working element (for example, a bucket, a crusher, a crane, etc.).

[0013] The lower traveling body 1 includes, for example, a pair of left and right crawlers, and each crawler is hydraulically driven by a traveling hydraulic motor (not shown), thereby causing the excavator 100 to travel. The upper rotating body 3 rotates relative to the lower traveling body 1 by being driven by a hydraulic swing motor or an electric motor (neither of which are shown).

[0014] The boom 4 is pivotally attached to the front center of the upper rotating body 3 so as to be able to tilt up and down, and an arm 5 is pivotally attached to the tip of the boom 4 so as to be able to rotate up and down, and a bucket 6 is pivotally attached to the tip of the arm 5 so as to be able to rotate up and down. The boom 4, arm 5, and bucket 6 are hydraulically driven by a boom cylinder 7, an arm cylinder 8, and a bucket cylinder 9, respectively. The cabin 10 is a control room in which an operator sits, and is mounted, for example, on the front left side of the upper rotating body 3. In response to operations by an operator sitting in the cabin 10, the excavator 100 operates actuators to drive driven elements such as the lower traveling body 1, the upper rotating body 3, the boom 4, the arm 5, and the bucket 6.

[0015] FIG. 2 is a block diagram showing the system configuration of the shovel 100. As shown in this figure, in addition to the above configuration, the shovel 100 is equipped with a controller 30, an imaging device 40, a distance sensor 41, a motion / posture state sensor 42, a position sensor 43, an operation device 45, a display device 50, an audio output device 60, and a communication device 80. An information processing device according to the present invention includes at least the controller 30.

[0016] The imaging device 40 captures an image of the periphery of the shovel 100 and outputs the image to the controller 30. The imaging device 40 includes a rear camera 40B, a left camera 40L, and a right camera 40R. The "periphery" of the shovel 100 only needs to include at least a predetermined range within a predetermined distance from the shovel 100. The rear camera 40B is attached to the rear of the upper rotating body 3 and captures an image of the area behind the upper rotating body 3. The left camera 40L is attached to the left side of the upper rotating body 3, and captures an image of the left side of the upper rotating body 3. The right camera 40R is attached to the right side of the upper rotating body 3 and captures an image of the right side of the upper rotating body 3. Each of these rear camera 40B, left camera 40L, and right camera 40R is attached to the upper rotating body 3 so that its optical axis faces diagonally downward, and has an imaging range (angle of view) in the vertical direction that includes the ground near the shovel 100 to the far side of the shovel 100. In addition, the horizontal imaging range (angle of view) of the rear camera 40B, left camera 40L, and right camera 40R is, for example, a range that includes approximately all directions around the shovel 100.

[0017] The distance sensor 41 is a distance measuring means that measures the distance to an object in the vicinity of the shovel 100 and acquires that information (two-dimensional or three-dimensional distance information), and outputs the acquired information to the controller 30. The distance sensor 41 includes a rear distance sensor 41B, a left distance sensor 41L, and a right distance sensor 41R. The rear distance sensor 41B is attached to the rear of the upper rotating body 3, and measures the area behind the upper rotating body 3. The measurement range of the rear distance sensor 41B corresponds to the imaging range of the rear camera 40B. The left distance sensor 41L is attached to the left side of the upper rotating body 3, and performs measurements on the left side of the upper rotating body 3. The measurement range of the left distance sensor 41L corresponds to the imaging range of the left camera 40L. The right distance sensor 41R is attached to the right side of the upper rotating body 3, and performs measurements to the right of the upper rotating body 3. The measurement range of the right distance sensor 41R corresponds to the imaging range of the right camera 40R. In this embodiment, a LIDAR (Light Detection and Ranging) sensor that uses light is used as each distance sensor 41. However, the type of distance sensor 41 is not particularly limited, and may be, for example, a millimeter wave radar or a distance measuring device using a stereo camera.

[0018] Furthermore, imaging devices 40 and distance sensors 41 that measure and capture images in the same direction correspond to each other and form pairs. Specifically, rear camera 40B and rear distance sensor 41B form rear sensor unit 46B, left camera 40L and left distance sensor 41L form left sensor unit 46L, and right camera 40R and right distance sensor 41R form right sensor unit 46R.

[0019] The operation / posture state sensor 42 is a sensor that detects the operation state and posture state of the excavator 100, and outputs the detection results to the controller 30. The operation / posture state sensor 42 includes a boom angle sensor, an arm angle sensor, a bucket angle sensor, a three-axis inertial sensor (IMU: Inertial Measurement Unit), a swing angle sensor, and an acceleration sensor. These sensors may be composed of sensors that acquire rotational information such as stroke sensors of cylinders on the boom, etc., or rotary encoders, or may be replaced by acceleration (which may also include speed and position) acquired by an IMU. The arm angle sensor detects the rotation angle of the arm 5 relative to the boom 4 (hereinafter referred to as the "arm angle"). The bucket angle sensor detects the rotation angle of the bucket 6 with respect to the arm 5 (hereinafter referred to as the "bucket angle"). The IMUs are attached to the boom 4 and the arm 5, respectively, and detect the acceleration of the boom 4 and the arm 5 along three predetermined axes and the angular acceleration of the boom 4 and the arm 5 around the three predetermined axes. The rotation angle sensor detects the rotation angle based on a predetermined angular direction of the upper rotating body 3. However, this is not limited to this, and the rotation angle may be detected based on a GPS or IMU sensor provided on the upper rotating body 3. The acceleration sensor is attached at a position away from the rotation axis of the upper rotating body 3, and detects the acceleration at that position of the upper rotating body 3. As a result, it can be determined whether the upper rotating body 3 is rotating or the lower traveling body 1 is traveling, etc., based on the detection result of the acceleration sensor.

[0020] The position sensor 43 is a sensor that acquires information about the position (current position) of the shovel 100, and is a GPS (Global Positioning System) receiver in this embodiment. The position sensor 43 receives a GPS signal containing information about the position of the shovel 100 from a GPS satellite, and outputs the acquired position information of the shovel 100 to the controller 30. Note that the position sensor 43 does not have to be a GPS receiver as long as it can acquire information about the position of the shovel 100, and may be one that uses a satellite positioning system other than GPS, for example. The position sensor 43 may be provided on the lower traveling body 1 or on the upper rotating body 3.

[0021] The operation device 45 is provided near the cockpit of the cabin 10 and is an operation means by which the operator operates each operating element (undercarriage 1, upper rotating body 3, boom 4, arm 5, bucket 6, etc.). In other words, the operation device 45 is an operation means for operating each hydraulic actuator that drives each operating element. The operation device 45 includes, for example, levers, pedals, various buttons, etc., and outputs operation signals to the controller 30 according to the operation content of these. The operation device 45 is also an operation means for operating the imaging device 40, distance sensor 41, motion / posture state sensor 42, position sensor 43, display device 50, audio output device 60, communication device 80, etc., and outputs operation commands for each of these parts to the controller 30.

[0022] The display device 50 is provided near the cockpit in the cabin 10, and displays various types of image information to be notified to the operator under the control of the controller 30. The display device 50 is, for example, a liquid crystal display or an organic EL (Electroluminescence) display, and may be a touch panel type that also serves as at least a part of the operation device 45.

[0023] The audio output device 60 is provided near the cockpit in the cabin 10, and outputs various types of audio information to notify the operator under the control of the controller 30. The audio output device 60 is, for example, a speaker or a buzzer.

[0024] The communication device 80 is a communication device that transmits and receives various information to and from remote external devices, other excavators 100, etc., through a predetermined communication network NW (for example, a mobile phone network or an Internet network with a base station as its terminal) based on a predetermined wireless communication standard.

[0025] The controller 30 is a control device that controls the operation of each part of the shovel 100 to control the drive of the shovel 100. The controller 30 is mounted inside the cabin 10. The functions of the controller 30 may be realized by any hardware, software, or a combination thereof, and for example, the controller 30 is configured mainly by a microcomputer including a CPU, RAM, ROM, I / O, etc. In addition to these, the controller 30 may also be configured to include, for example, an FPGA, an ASIC, etc.

[0026] The controller 30 also includes, as functional units that execute various functions, an OGM calculation unit 31 and an object detection determination unit 32. The controller 30 also includes a storage unit 35 as a storage area defined in an internal memory such as an EEPROM (Electrically Erasable Programmable Read-Only Memory).

[0027] The OGM calculation unit 31 creates a two-dimensional occupancy grid map or OGM that quantizes and expresses distance information based on the outputs from the imaging device 40 and distance sensor 41. The OGM is a map display of the existence probability of the position, velocity, etc. of a detected specific object (detection target), and more specifically, probability information regarding the existence probability of the detection target is placed in the corresponding cell on the grid based on the distance information of the detection target (see FIG. 4). Although the present embodiment takes an example of a two-dimensional OGM, the same applies to a three-dimensional OGM to which height information is added. More specifically, the OGM calculation unit 31 includes a camera OGM creation unit 311 that detects objects based on the output of the imaging device 40, analyzes their positions, and creates an OGM (a first OGM 71 described below), a distance sensor OGM creation unit 312 that detects objects based on the output of the distance sensor 41 and creates an OGM (a second OGM 72 described below), a local OGM creation unit 313 that combines these OGMs to create a local OGM 73, and a global OGM creation unit 314 that creates a wide-ranging global OGM 74 based on the local OGM 73 (see Figure 3).

[0028] The object detection determination unit 32 performs detection determination of a specific object (detection target) around the shovel 100 based on each local OGM 73 created by the OGM calculation unit 31. Then, the object detection determination unit 32 outputs the detection result (determination result) to the display device 50.

[0029] The storage unit 35 stores various programs and various data for operating each part of the shovel 100, and also functions as a work area for the controller 30. In this embodiment, the storage unit 35 stores various programs as well as various data acquired by the imaging device 40, the distance sensor 41, etc., calculation results, etc. The storage unit 35 also stores in advance feature amounts (image feature amounts) of detection targets in the object detection process described below. In the storage unit 35, at least one feature amount is associated with each of various types of objects and people that can be selected by the operator as detection targets.

[0030] The shovel 100 can communicate with the management device 200 via a predetermined communication network NW. The communication network NW may include, for example, a mobile communication network terminated at a base station. The communication network NW may also include a satellite communication network that uses communication satellites in the sky. The communication network NW may also include the Internet network or the like. The communication network NW may also include a short-range communication network that complies with protocols such as WiFi or Bluetooth (registered trademark). This allows the shovel 100 to transmit (upload) various types of information to the management device 200. The shovel 100 may also be configured to be able to communicate with the support device 300 via a communication network NW.

[0031] The management device 200 (an example of an external device or information processing device) is located in a location geographically separated from users who own the shovel 100 and the support device 300. The management device 200 is, for example, installed in a management center or the like located outside the work site where the shovel 100 works, and is a server device mainly composed of one or more server computers or the like. In this case, the server device may be an in-house server operated by the business operator that operates the system or an associated business operator related to that business operator, or it may be a rental server. Furthermore, this server device may be a so-called cloud server. Furthermore, the management device 200 may be a server device (a so-called edge server) located in a management office or the like within the work site of the shovel 100, or it may be a fixed or portable general-purpose computer terminal. As described above, the management device 200 can communicate with each of the shovel 100 and the support device 300 via the communication network NW. This allows the management device 200 to receive and store (accumulate) various information uploaded from the shovel 100. Furthermore, the management device 200 can transmit various information to the support device 300 in response to a request from the support device 300.

[0032] The assistance device 300 (an example of a user terminal or terminal device) is a user terminal used by a user. The user may include, for example, a work site supervisor, a manager, an operator of the shovel 100, a manager of the shovel 100, a serviceman for the shovel 100, a developer of the shovel 100, etc. The assistance device 300 is, for example, a general-purpose mobile terminal such as a laptop computer terminal, a tablet terminal, or a smartphone owned by a user. The assistance device 300 may also be a fixed general-purpose terminal such as a desktop computer. The assistance device 300 may also be a dedicated terminal (a mobile terminal or a fixed terminal) for receiving information. The support device 300 can communicate with the management device 200 via the communication network NW. This allows the support device 300 to receive information transmitted from the management device 200 and provide the information to the user via a display device mounted on the support device 300. The support device 300 may also be configured to be able to communicate with the excavator 100 via the communication network NW.

[0033] [Shovel movement] Next, the operation of the shovel 100 when executing an object detection process for detecting a specific object in the vicinity will be described. Figure 3 is a data flow diagram showing the flow of data in this object detection process, Figures 4(a) to (c) are diagrams showing examples of the first OGM71, second OGM72, and local OGM73 described below, and Figures 5(a) and (b) are diagrams showing examples of the local OGM73 and global OGM74 described below.

[0034] The object detection process is performed by the CPU of the controller 30 executing a predetermined program stored in an internal storage device. This process may be started and ended based on an operation by an operator, or may be performed continuously while the excavator 100 is in operation. In this embodiment, a human body (person) is assumed to be detected as the detection target. Therefore, when the process is executed, a human body is selected as the detection target by, for example, an operation by an operator, and a feature amount corresponding to this detection target (for example, a human face) is read out from the storage unit 35 and set.

[0035] 3, when the object detection process is executed, the controller 30 first acquires image information of the periphery of the shovel 100 using the imaging device 40, and acquires distance information to objects around the shovel 100 using the distance sensor 41. The controller 30 records the acquired image information and distance information in the storage unit 35. Note that the imaging device 40 and the distance sensor 41 here correspond to each other as the rear sensor unit 46B, the left sensor unit 46L, or the right sensor unit 46R. Furthermore, the controller 30 may take photographs and measure the entire periphery while rotating the upper rotating body 3 .

[0036] Next, the controller 30 generates an OGM that represents the presence of a person (human body) around the excavator 100 by the OGM calculation unit 31. Here, the OGM calculation unit 31 first creates a first OGM 71 based on an image acquired by the imaging device 40 and a second OGM 72 based on distance information acquired by the distance sensor 41, separately.

[0037] The first OGM 71 is created by the camera OGM creating unit 311 as an OGM for a detected human body based on image information acquired by the imaging device 40. Here, for example, a first OGM 71 as shown in Fig. 4(a) is created. In the first OGM 71, the position of an object region R1 detected as a human body (or an object presumed to be a human body) from an image captured by the imaging device 40 is displayed in a planar form on a two-dimensional map partitioned into a grid pattern, for example.

[0038] The second OGM 72 is generated by the distance sensor OGM generation unit 312 based on the distance information acquired by the distance sensor 41, detecting a human body and generating the OGM. Here, for example, a second OGM72 as shown in Fig. 4(b) is created. In this second OGM72, the position of an object region R2 detected as a human body (or an object presumed to be a human body) based on the distance information is displayed in a planar form on a two-dimensional map partitioned into a grid, for example, in the same way as the first OGM71.

[0039] Next, as shown in FIG. 3, the OGM calculation unit 31 causes the local OGM creation unit 313 to synthesize the first OGM 71 and the second OGM 72 to create a local OGM 73. In the local OGM73 of this embodiment, as shown in Fig. 4(c), for example, a grid in which object regions R1 and R2 exist in both the first OGM71 shown in Fig. 4(a) and the second OGM72 shown in Fig. 4(b) is set to an object region R3 in which a human body has been detected. That is, in each of the first OGM71 and the second OGM72, a "1" is assigned to a grid in which an object region exists and a "0" is assigned to a grid in which no object region exists, and when the sum of the numerical values of corresponding grids is "1" during synthesis, the grid is set to the object region R3. However, this synthesis method is not limited to that of this embodiment. For example, when creating each of the first OGM71 and the second OGM72, instead of just "0" and "1," a numerical value weighted according to the likelihood of the existence of each object region may be assigned to the grid of the object region. Then, when synthesizing these, if the numerical value obtained by taking the sum (or product) of the object regions of the corresponding grids is equal to or greater than a predetermined threshold, the grid may be set as the object region R3.

[0040] The local OGM 73 is an example of first map information according to the present invention, and as shown in Fig. 5(a), is an OGM having a local range (area) within the periphery of the shovel 100. In this embodiment, multiple (six) local OGMs 73 are created so as to surround the entire periphery of the shovel 100. There are no particular limitations on the range or number of the local OGMs 73. In addition, it is preferable that the cell size of the local OGMs 73 is small within a range that does not cause an excessive calculation load, in terms of improving the accuracy of object detection.

[0041] 3, the controller 30 causes the object detection determination unit 32 to determine whether a person (human body) has been detected around the excavator 100 based on each local OGM 73 created by the OGM calculation unit 31. In this embodiment, it is determined that a person exists in a grid that is set as an object region R3 among the local OGMs 73. Alternatively, it may be determined that a person exists only in a grid (object region R3) that has an assigned numerical value equal to or greater than a predetermined value. In this way, by performing object detection using the local OGM 73 with a relatively narrow range, the detection determination calculation can be speeded up. Note that the control that directly uses the local OGM 73 is not limited to object detection, but can be widely applied to the operation control of the shovel 100, and can be suitably applied in particular to operation control that requires quick response (for example, collision determination, etc.). Then, the object detection determination unit 32 outputs the detection result (determination result) to the display device 50. The display mode is not particularly limited as long as it allows the position of the detection target to be identified.

[0042] Next, the OGM calculation unit 31 generates the global OGM 74 based on the local OGM 73 and the like using the global OGM generation unit 314. The global OGM 74 is an example of second map information according to the present invention, and is an OGM that covers a wide range (area) around the excavator 100 and has a range that is farther than the local OGM 73, as shown in FIG. 5(b). The global OGM creation unit 314 creates the global OGM 74 by acquiring surrounding information from multiple local OGMs 73. However, ranges that are not included in any of the local OGMs 73 may be created separately. The creation time (update period) of the global OGM 74 is longer than that of the local OGM 73. The global OGM74 may have a larger range than the local OGM73. The cell size of the global OGM74 may be larger than that of the local OGM73.

[0043] Then, the global OGM creating unit 314 outputs (information about) the global OGM 74 to the display device 50. The operator can recognize the situation in a wide range around the excavator 100 by visually checking the global OGM 74. The use of the global OGM 74 is not limited to the transmission (display) of information to the operator, but can be suitably applied to, for example, situations where quick response is not required.

[0044] [Technical effect of this embodiment] As described above, according to this embodiment, a local OGM 73 that displays information about a local area around the shovel 100 as a map is created based on information about the area around the shovel 100 acquired by the imaging device 40 and the distance sensor 41, and a global OGM 74 that displays information about an area wider than the area (range) of the local OGM 73 as a map is created based on this local OGM 73. Then, the operation of the shovel 100 is controlled based on the local OGM 73. In this way, by controlling the movement using the local OGM 73 with a narrow range, the movement control can be performed faster than when simply creating a map that includes a wide area around the target and performing the movement control. Therefore, the movement control of the target can be performed preferably using map information.

[0045] Furthermore, according to this embodiment, the global OGM 74, which has a wider range than the local OGM 73, is output to the display device 50. This allows even the global OGM 74, which takes a relatively long time to create (update), to be suitably used for transmitting (displaying) information to the operator. That is, the local OGM 73 with a shorter update cycle can be used to suitably perform operational control that requires quick response, while the global OGM 74 with a longer update cycle can be used suitably for display purposes that do not require quick response.

[0046] [Variation 1] FIG. 6 is a diagram for explaining a modified example of the object detection process of this embodiment. When performing object detection, the object detection determination unit 32 may select at least one local OGM73 corresponding to the operating state of the shovel 100 from the multiple local OGM73s based on the operating state of the shovel 100, and perform object detection using only this selected local OGM73. Here, the "operation state" of the shovel 100 includes each state of the shovel 100, such as movement, rotation, attachment operation, etc. Also, a local OGM 73 that is not used does not need to be created in the first place.

[0047] For example, as shown in FIG. 6(a), when the excavator 100 moves, object detection (operation control) may be performed using only the local OGMs 73 (73RF, 73LF) located in the moving direction. Alternatively, as shown in Figure 6(b), when the excavator 100 (upper rotating body 3) rotates, object detection (operation control) may be performed using only the local OGM73 (73RF, 73R, 73RB, 73L) in the area where the upper rotating body 3 and attachments are (expected to be) included in the rotation.

[0048] In this way, by selecting at least one local OGM 73 based on the operating state of the shovel 100 and controlling the operation of the shovel 100 based on the selected local OGM 73, the speed of the operation control can be further increased.

[0049] [Variation 2] FIG. 7 is a diagram for explaining the relationship between the tilt angles of the imaging device 40 and the distance sensor 41 and the position and range (size) of the local OGM 73. In FIG. The detection position and detection range of the imaging device 40 and the distance sensor 41 change depending on the PTZ state (operation state of pan, tilt, or zoom). Therefore, as an example (tilt operation example) shown in Fig. 7, the position and range of the local OGM 73 change depending on the PTZ state of the imaging device 40 and the distance sensor 41. Therefore, the imaging device 40 and the distance sensor 41 may be appropriately subjected to PTZ operation based on the position and range of the desired local OGM 73. In this case, each of the imaging device 40 and the distance sensor 41 is configured to be capable of PTZ operation (pan, tilt, and zoom operations), and the controller 30 can of course control this.

[0050] [others] Although the embodiments of the present invention have been described above, the present invention is not limited to the above-described embodiments or modifications thereof. For example, in the above embodiment, the imaging device 40 and the distance sensor 41 are mounted on the shovel 100, but the imaging device 40 and the distance sensor 41 do not have to be mounted on the shovel 100, and may be installed at a high place or mounted on an unmanned aerial vehicle such as a drone, for example. The acquired data may then be transmitted to the shovel 100, or the data may be transmitted to the management device 200 or the support device 300 to execute a detection process, and the results may then be transmitted to the shovel 100.

[0051] In the above embodiment, the detection results (based on the outputs) of the imaging device and the distance sensor are synthesized (fused). However, the information acquisition means according to the present invention is not limited to the imaging device and the distance sensor, and may include, for example, a millimeter wave sensor or an audio (sound sensor) as long as it can acquire information about the periphery of the target (shovel) (the combination is not limited).

[0052] Furthermore, the information processing device according to the present invention may be one that processes information around an object, and is not limited to one that is mounted on the object. Furthermore, the "subject" in this case is not limited to shovels, but is of course applicable to construction machinery in general, and is also suitably applicable to work machinery including construction machinery. In addition, the details shown in the embodiments can be modified as appropriate without departing from the spirit of the invention. [Explanation of symbols]

[0053] 100 Shovel 30 Controllers 31 OGM calculation section 32 Object detection and judgment unit 35 Storage section 40 Imaging device 41 Distance Sensor 42 Motion and Posture Sensor 50 Display device 73 Local OGM (first map information) 74 Global OGM (Second Map Information) 200 Management device 300 Support Devices

Claims

1. information acquisition means for acquiring information about the surroundings of a target; a first map creation means for creating first map information that displays the information in a first area as a map based on the information acquired by the information acquisition means; a second map creation means for creating second map information based on the first map information, the second map information being a map display of the information in a second area larger than the first area; an action control means for controlling the action of the object based on the first map information; Equipped with the first map creation means creates a plurality of pieces of first map information having different areas based on information acquired by a plurality of pieces of information acquisition means having different information acquisition ranges; the second map creation means creates the second map information based on the plurality of pieces of first map information; Information processing device.

2. a display control means for displaying the second map information on a display means; The information processing device according to claim 1 .

3. a detection means for detecting a motion state of the object; The operation control means selecting at least one piece of first map information from the plurality of pieces of first map information based on the motion state of the target; controlling the movement of the object based on the at least one piece of first map information; 3. The information processing device according to claim 1.

4. The information acquisition means The imaging device includes an imaging unit that acquires image information and a distance measuring unit that acquires distance information, As the information, positions and existence probabilities of objects around the target are acquired. The information processing device according to claim 1 .

5. the imaging means and the distance measuring means are configured to be capable of PTZ operation, the first map creation means causes the imaging means and the distance measuring means to perform PTZ operations based on the position and range of the first map information to be created; The information processing device according to claim 4 .

6. The object is a construction machine. The information processing device according to claim 1 .

7. A control program for an information processing device including an information acquisition unit that acquires information about the surroundings of a target, Computer, a first map creation means for creating first map information that displays the information in a first area as a map based on the information acquired by the information acquisition means; a second map creation means for creating second map information based on the first map information, the second map information being a map display of the information in a second area larger than the first area; an action control means for controlling the action of the object based on the first map information; It functions as the first map creation means creates a plurality of pieces of first map information having different areas based on information acquired by a plurality of pieces of information acquisition means having different information acquisition ranges; the second map creation means creates the second map information based on the plurality of pieces of first map information; A control program for an information processing device.

Images