Underwater work device

Abstract

To provide an underwater work device capable of performing the work even when water becomes turbid during the underwater work.SOLUTION: An underwater work device includes a work device 10 which has an acoustic measuring instrument 11 and an optical measuring instrument 12 and performs the work on a work object 50 present underwater and a control device 40 which controls the work device 10. The acoustic measuring instrument 11 uses sound to acquire point cloud data representing the surface of an object as point cloud data 51 of the sound. The optical measuring instrument 12 uses light to acquire point cloud data representing the surface of the object and including information about a color as point cloud data 52 of the light. The control device 40 includes a point cloud map 61 of the sound being the map created from the point cloud data 51 of the sound and a point cloud map 62 of the light being the map created from the point cloud data 52 of the light, determines the relationship between the point cloud map 62 of the light and the point cloud map 61 of the sound and the relationship between the point cloud map 61 of the sound and the point cloud data 51 of the sound, and updates the point cloud map 62 of the light by using the point cloud data 51 of the sound and the point cloud map 61 of the sound.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to an underwater work device for performing work underwater. [Background technology]

[0002] When working with a work object (object to be worked on) in water such as a river or ocean, or in water filled inside a structure such as a tank or pool, using an underwater work device, the water may become cloudy. Such cloudy water occurs when the work involves active interaction with the work object, such as cutting, relocation, and recovery. Even in work that does not involve active interaction with the work object, such as surveying, the water may become cloudy due to water currents, etc. Such cloudy water may hinder the recognition of the work object and its surrounding environment by optical measuring instruments such as optical cameras.

[0003] When the water is very turbid, it is difficult to recognize the work object using optical measuring instruments due to scattering caused by the turbidity, and work must be suspended until the turbidity is cleared. Even in such cases, it is possible to grasp the work object and its surrounding environment by using measuring devices such as sonar that use ultrasonic waves, which have higher transparency than light. However, due to the difference in the physical properties of ultrasonic waves and light, acoustic measurement using ultrasonic waves has the disadvantage that it has lower spatial resolution than optical measurement using light, and cannot obtain color information of the work object. For this reason, it is difficult for operators of underwater work equipment to recognize the work object using only the information obtained by acoustic measurement, making it difficult to work on the work object.

[0004] Patent Document 1 describes an example of a device that can recognize a work object even in muddy water. The underwater laser visualization device described in Patent Document 1 includes an oscillation head that irradiates pulses of laser light into muddy water, and an imaging camera unit with an image intensifier that has a shutter mechanism, and uses a pulsed light source and shutter control to block reflected light from scattering objects in the muddy water and capture only the reflected light from the work object to be photographed, making it possible to recognize the work object even in muddy water. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Publication No. 7-72250 Summary of the Invention [Problem to be solved by the invention]

[0006] For example, in a device that recognizes a work object in turbid water using a pulsed light source and shutter control, such as the underwater laser visualization device described in Patent Document 1, the difference in flight time between the light reflected from a scatterer in the turbid water and the light reflected from the work object must be sufficiently large relative to the opening and closing speed of the shutter. Therefore, when the water becomes turbid around an underwater work object, the difference in distance between the scatterer in the turbid water and the work object is small, making it difficult to selectively block the light reflected from the scatterer and capture only the light reflected from the work object.

[0007] As such, with conventional underwater work equipment, there is a problem that when working underwater, especially when the water becomes cloudy around the work object, it is difficult for workers to recognize the work object, making it difficult to work on the work object using the underwater work equipment.

[0008] An object of the present invention is to provide an underwater work device that can perform work even if the water becomes turbid during underwater work. [Means for solving the problem]

[0009] The underwater work device according to the present invention includes a work device equipped with an acoustic measuring device and an optical measuring device for performing work on a work object existing underwater, a control device for controlling the work device, and an operation device used to operate the work device. The acoustic measuring device uses sound to acquire point cloud data representing the surfaces of objects existing around the work device as sound point cloud data. The optical measuring device uses light to acquire point cloud data representing the surfaces of objects existing around the work device and having information about colors as light point cloud data. The control device includes a sound point cloud map which is a map of objects existing around the work device created from the sound point cloud data, and a light point cloud map which is a map of objects existing around the work device created from the light point cloud data. The control device determines the relationship between the light point cloud map and the sound point cloud map, and the relationship between the sound point cloud map and the sound point cloud data, and updates the light point cloud map using the sound point cloud data and the sound point cloud map. Effect of the Invention

[0010] According to the present invention, it is possible to provide an underwater work device that is capable of carrying out work even if the water becomes turbid during underwater work. [Brief description of the drawings]

[0011] [Figure 1A] 1A and 1B are diagrams showing examples of how an underwater work device performs work underwater. [Figure 1B] FIG. 13 is a diagram showing an example of the underwater work device performing work underwater when the water becomes turbid. [Diagram 2] 1 is a block diagram showing an example of the configuration of an underwater work device according to a first embodiment of the present invention. [Diagram 3] FIG. 4 is a block diagram showing an example of a data flow in a measurement data processing unit included in the control device. [Figure 4] 5 is a flowchart showing the flow of processing executed by a measurement data processing unit in the first embodiment. [Diagram 5] FIG. 5 is a block diagram showing an example of the configuration of an underwater work device according to a second embodiment of the present invention. [Figure 6] 10 is a flowchart showing the flow of processing executed by a measurement data processing unit in the second embodiment. [Figure 7A] 1A and 1B are schematic diagrams showing the top surface and cross section of a work object to be cut in a case where the work involves cutting the work object. [Figure 7B] FIG. 7B is a schematic diagram showing a point cloud included in the light point cloud map corresponding to the work object shown in FIG. 7A in a case where the work content is cutting the work object. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] The underwater work device of the present invention can provide high-resolution, colored images to workers even when the water becomes so turbid near the work object during underwater work that optical measuring instruments cannot make accurate measurements, allowing work on the work object to be continued without interruption.

[0013] Hereinafter, an underwater work device according to an embodiment of the present invention will be described with reference to the drawings. In the drawings used in this specification, the same or corresponding components are designated by the same reference numerals, and repeated description of these components may be omitted. EXAMPLES

[0014] First, a brief description will be given of underwater work using the underwater work device.

[0015] 1A and 1B are diagrams showing an example of how an underwater work device performs work underwater.

[0016] 1A, the underwater work device includes an operating device 30 and a work device 10, and is operated by an operator to perform work on a work target 50. The work device 10 includes an optical measuring instrument 12. The work target 50 is an object that is the target of work by the underwater work device, and exists underwater.

[0017] 1B, when the work apparatus 10 is performing work underwater, the water may become turbid 20. In a conventional underwater work apparatus, when the water becomes turbid 20, particularly around a work target 50, it is difficult to recognize the work target 50 with the optical measuring instrument 12 due to scattering caused by the turbidity 20, and it is difficult to perform work on the work target 50.

[0018] First Embodiment An underwater work device according to a first embodiment of the present invention will be described.

[0019] 2 is a block diagram showing an example of the configuration of an underwater work apparatus according to this embodiment. The underwater work apparatus according to this embodiment comprises a work apparatus 10, an operating device 30, and a control device 40, and performs a predetermined operation on a work target 50 (FIG. 1) existing underwater.

[0020] The work device 10 is located underwater, and is equipped with an acoustic measuring device 11, an optical measuring device 12, a position measuring unit 13, and a turbidity estimating unit 14, and is capable of grasping information relating to the work device 10 and the surroundings of the work device 10. The work device 10 performs work on a work object 50 by operating the operation device 30 by an operator of the underwater work device.

[0021] The working device 10 can be equipped with a mechanism according to the type of work, for example, a working mechanism such as an articulated arm or a cutting tool. The working device 10 can also be equipped with a moving mechanism such as a thruster or crawler, and can move underwater. Furthermore, the working device 10 may be equipped with a mechanism for changing the orientation of the acoustic measuring device 11 and the optical measuring device 12.

[0022] Acoustic measuring instrument 11 is a measuring instrument that measures the surroundings of working device 10 using sound and acquires point cloud data that represents the surfaces of objects present around working device 10. Acoustic measuring instrument 11 acquires point cloud data expressed in three-dimensional coordinates. Objects present around working device 10 include work target 50. Acoustic measuring instrument 11 can be configured, for example, by combining one transducer and a two-dimensional array sensor, or can be configured by two one-dimensional array sensors that are orthogonal to each other.

[0023] Hereinafter, the point cloud data acquired by the acoustic measuring instrument 11 and expressed in three-dimensional coordinates will be referred to as "sound point cloud data."

[0024] The optical measuring instrument 12 is a measuring instrument that uses light to measure the surroundings of the operating device 10 and acquires point cloud data that represents the surfaces of objects present around the operating device 10. The optical measuring instrument 12 acquires point cloud data that is expressed in three-dimensional coordinates and has color information. In other words, the optical measuring instrument 12 acquires colored point cloud data. The objects present around the operating device 10 include a work target 50. The optical measuring instrument 12 can be configured, for example, by combining a Lidar sensor or a ToF sensor with a video camera, or can also be configured with a stereo camera.

[0025] Hereinafter, the point cloud data acquired by the optical measuring instrument 12, which is expressed in three-dimensional coordinates and has information about color, will be referred to as "light point cloud data."

[0026] The position measurement unit 13 acquires three-dimensional coordinates of the acoustic measuring instrument 11 and the optical measuring instrument 12 with respect to the working device 10 or the working environment as position data. The position measurement unit 13 can be configured with software or hardware. The position measurement unit 13 can estimate the positions of the acoustic measuring instrument 11 and the optical measuring instrument 12 with respect to the working device 10, for example, based on sound point cloud data and light point cloud data. The position measurement unit 13 can also estimate the positions of the acoustic measuring instrument 11 and the optical measuring instrument 12 with respect to the working environment, for example, based on the position of the working device 10 acquired by a positioning sensor capable of communicating with a reference point set in the working environment, or the position of the working device 10 acquired from an encoder that measures the angle and translation amount of the working device 10.

[0027] Note that, if the operating device 10 moves to the vicinity of the work target 50 at the start of the work and does not move during the work, the position measurement unit 13 does not need to acquire the position data of the acoustic measuring instrument 11 and the optical measuring instrument 12. In this case, the positions of the acoustic measuring instrument 11 and the optical measuring instrument 12 do not change during the work and can therefore be obtained from the position of the operating device 10.

[0028] The turbidity estimation unit 14 determines the turbidity of the water in which the work target 50 exists. The turbidity estimation unit 14 can determine the turbidity of the water using existing technology. For example, the turbidity estimation unit 14 can estimate the turbidity of the water from the intensity of the signal from the acoustic measuring instrument 11 or the optical measuring instrument 12, the degree of agreement of the coordinates of points between the sound point cloud data and the light point cloud data, etc. The turbidity estimation unit 14 may also be provided with a turbidity meter and measure the turbidity of the water with the turbidity meter.

[0029] The operating device 30 is located in the air, and is used by a worker to operate the working device 10. The operating device 30 has an input unit through which the worker inputs operation instructions. The working device 10 is operated by these operation instructions. The operating device 30 also has an image display unit that displays images, and can also display images of the work target 50 and its surroundings to the worker. The operating device 30 can have, for example, buttons, a motion controller, etc. as the input unit, and a display, a stereoscopic image display device, etc. as the image display unit.

[0030] The control device 40 is located in the air or underwater, includes a measurement data processing unit 41, and communicates with the working device 10 and the operation device 30. The control device 40 controls the working device 10 and the operation device 30, and processes data received from these devices. The control device 40 includes, for example, a central processing unit, a main storage device, and an auxiliary storage device, and executes a program for controlling the working device 10. Note that the components of the control device 40 may be distributed, or may be present inside the working device 10 or the operation device 30.

[0031] FIG. 3 is a block diagram showing an example of the flow of data in the measurement data processing unit 41 included in the control device 40. As shown in FIG.

[0032] The measurement data processing unit 41 acquires sound point cloud data 51 from the acoustic measuring instrument 11, light point cloud data 52 from the optical measuring instrument 12, position data 53 from the position measuring unit 13, and water turbidity 54 from the turbidity estimating unit 14. Then, based on the sound point cloud data 51, light point cloud data 52, and position data 53, the measurement data processing unit 41 updates the stored sound point cloud map 61 and light point cloud map 62 while switching processing according to the turbidity 54, and creates an image 80 to be presented to the worker. The measurement data processing unit 41 outputs the created image 80 to the operation device 30.

[0033] The sound point cloud map 61 and the light point cloud map 62 are maps that represent the environment around the operating device 10 (the environment including the work target 50) as a point cloud of three-dimensional coordinates. The sound point cloud map 61 is a map of objects existing around the operating device 10 created using the sound point cloud data 51, and includes position information of these objects. The light point cloud map 62 is a map of objects existing around the operating device 10 created using the light point cloud data 52, and includes position information and color information of these objects.

[0034] The measurement data processing unit 41 creates and stores the sound point cloud map 61 and the light point cloud map 62 of the required range in advance before the operating device 10 executes the operation on the work object 50. The measurement data processing unit 41 may create and store the sound point cloud map 61 and the light point cloud map 62 together with the operation of the operating device 10 while the operating device 10 is executing the operation on the work object 50.

[0035] Next, the processing executed by the measurement data processing unit 41 will be described.

[0036] FIG. 4 is a flowchart showing the flow of processing executed by the measurement data processing unit 41 in the first embodiment.

[0037] In step 100, the measurement data processing unit 41 acquires sound point cloud data 51, light point cloud data 52, position data 53, and water turbidity 54. Then, based on the position data 53, the measurement data processing unit 41 converts each of the sound point cloud data 51 and the light point cloud data 52 from coordinates in the measuring instrument coordinate system to coordinates in the environmental coordinate system or the working device coordinate system.

[0038] The measuring instrument coordinate system is a local coordinate system set for each of the acoustic measuring instrument 11 and the optical measuring instrument 12. The environmental coordinate system is a coordinate system set for the environment around the working device 10 (for example, a coordinate system based on a reference point set in the working environment). The working device coordinate system is a coordinate system set for the working device 10 (for example, a coordinate system based on the position of the working device 10).

[0039] In addition, if the coordinates of the acoustic measuring instrument 11 and the optical measuring instrument 12 do not change until the end of the work, coordinate conversion is not performed in step 100, and the sound point cloud data 51 and the light point cloud data 52 can remain expressed in coordinates in the measuring instrument coordinate system.

[0040] In step 101, the measurement data processing unit 41 determines whether the water is highly turbid based on the turbidity 54. If the water is highly turbid, it is difficult for the optical measuring instrument 12 to accurately acquire the light point cloud data 52 due to the effect of scattering caused by the turbidity.

[0041] If the turbidity 54 is equal to or less than a predetermined reference value, the measurement data processing unit 41 determines that the water is not very turbid and executes the process of step 102, and if the turbidity 54 is greater than this reference value, it determines that the water is very turbid and executes the process of step 103. The reference value of the turbidity 54 can be determined arbitrarily, for example, depending on the sensitivity of the optical measuring instrument 12.

[0042] In step 102, the measurement data processing unit 41 replaces the data of the stored light point cloud map 62 for the measurement region of the optical measuring instrument 12 at the current time with the light point cloud data 52, thereby updating the data of the light point cloud map 62. The measurement region of the optical measuring instrument 12 can be obtained from the viewing angle and position (coordinates) of the optical measuring instrument 12. The coordinates of the optical measuring instrument 12 can be obtained from the position data 53.

[0043] If the measurement data processing unit 41 does not store the light point cloud map 62 , the measurement data processing unit 41 creates a new light point cloud map 62 based on the light point cloud data 52 .

[0044] In step 103, the measurement data processing unit 41 creates a relational equation Em for expressing points included in the light point cloud map 62 using points included in the sound point cloud map 61. More specifically, the measurement data processing unit 41 creates a relational equation Em for expressing the coordinates of points included in the stored light point cloud map 62 using the coordinates of one or more points included in the stored sound point cloud map 61. The measurement data processing unit 41 creates this relational equation Em for each point included in the light point cloud map 62. Due to the coordinate conversion in step 100, the sound point cloud data 51 and the light point cloud data 52 are expressed in the same coordinate system.

[0045] If the measurement data processing unit 41 does not store the light point cloud map 62, the measurement data processing unit 41 creates a new light point cloud map 62 based on the light point cloud data 52. If the measurement data processing unit 41 does not store the sound point cloud map 61, the measurement data processing unit 41 creates a new sound point cloud map 61 based on the sound point cloud data 51.

[0046] Using the relational expression Em, the coordinates of a point included in the light point cloud map 62 are expressed by the coordinates of a point included in the sound point cloud map 61. For example, using the relational expression Em, the position vector of a point P included in the light point cloud map 62 can be expressed as the sum of the position vector of a point included in the sound point cloud map 61 and located closest to point P and an arbitrary constant vector. Also, for example, the position vector of point P can be expressed as a linear sum of the position vectors of three points included in the sound point cloud map 61 and located close to point P. Also, using a parametric surface S (for example, a Bezier surface) created from a plurality of points included in the sound point cloud map 61 and located near point P, the position vector of point P can be expressed as the sum of a point on surface S and a constant vector.

[0047] In step 104, the measurement data processing unit 41 creates a correspondence relationship Rs that associates points included in the sound point cloud data 51 with points included in the sound point cloud map 61. More specifically, the measurement data processing unit 41 creates a correspondence relationship Rs between points included in the sound point cloud data 51 acquired in step 100 and points included in the stored sound point cloud map 61. The measurement data processing unit 41 creates this correspondence relationship Rs for each point included in the sound point cloud data 51. From the correspondence relationship Rs, it is possible to know which point among the points included in the sound point cloud data 51 each point included in the sound point cloud map 61 corresponds to.

[0048] The measurement data processing unit 41, for example, regards the data of the sound point cloud map 61 as data from one time before, regards the sound point cloud data 51 as data from the current time, regards one point of the sound point cloud data 51 as one point included in the sound point cloud map 61 having moved, and creates the correspondence relationship Rs. The measurement data processing unit 41 can create the correspondence relationship Rs using any existing algorithm. For example, the measurement data processing unit 41 may create the correspondence relationship Rs by using an ICP algorithm to make the correspondence between two corresponding points such that the sum of the distances between the two corresponding points is minimized, or may create the correspondence relationship Rs using an existing algorithm for making two points at different times correspond to each other (taking into account the deformation of the work object 50) for time-series point cloud data.

[0049] Note that, among the points included in the sound point cloud data 51, if it is determined that a point included in the sound point cloud map 61 is not a moved point, but is a point newly obtained by measurement by the acoustic measuring instrument 11, this point is excluded from the correspondence relationship Rs. The point to be excluded can be determined, for example, based on the distance between a point included in the sound point cloud data 51 and a point included in the sound point cloud map 61 that is determined to correspond to this point, or by comparing the distribution of points in the sound point cloud map 61 with the measurement area of ​​the acoustic measuring instrument 11 when the sound point cloud data 51 was measured.

[0050] In step 105, the measurement data processing unit 41 updates the light point cloud map 62 using the sound point cloud data 51 and the sound point cloud map 61. Specifically, the measurement data processing unit 41 replaces the point Q1 of the sound point cloud map 61 for which the relational expression Em has been created with the point Q2 of the sound point cloud data 51 (the point acquired in step 100) using the correspondence relationship Rs. Then, the measurement data processing unit 41 updates the light point cloud map 62 using the point Q2 of the sound point cloud data 51 and the relational expression Em (more specifically, using the point Q1 of the sound point cloud map 61 replaced with the point Q2 of the sound point cloud data 51 and the relational expression Em).

[0051] More specifically, the measurement data processing unit 41 first replaces point Q1, for which a correspondence relationship Rs has been created, among the points included in the sound point cloud map 61 and for which a relational expression Em has been created, with point Q2 included in the sound point cloud data 51 and corresponding to point Q1, using the correspondence relationship Rs. Next, the measurement data processing unit 41 uses the relational expression Em to determine the position in the light point cloud map 62 of point Q1 of the sound point cloud map 61 that has been replaced with point Q2 of the sound point cloud data 51, and updates the light point cloud map 62.

[0052] The processing from step 103 to step 105 can be briefly summarized as follows. Steps 103 to 105 are the processing for the case where the water is very turbid. In step 103, a correspondence relationship (relationship formula Em) between the light point cloud map 62 and the sound point cloud map 61 is created. In step 104, a correspondence relationship Rs between the sound point cloud map 61 and the sound point cloud data 51 is created. By steps 103 and 104, the correspondence relationship between the sound point cloud data 51 and the light point cloud map 62 can be grasped. In step 105, the light point cloud map 62 (for example, a point cloud map created before the current time) is updated using the sound point cloud data 51 (for example, point cloud data measured at the current time).

[0053] In step 106, the measurement data processing unit 41 replaces the data of the stored sound point cloud map 61 with the sound point cloud data 51 for the measurement area of ​​the sound measuring instrument 11 at the current time, thereby updating the data of the sound point cloud map 61. The measurement area of ​​the sound measuring instrument 11 can be obtained from the viewing angle and position (coordinates) of the sound measuring instrument 11. The coordinates of the sound measuring instrument 11 can be obtained from the position data 53.

[0054] If the measurement data processing unit 41 does not store the sound point cloud map 61 , the measurement data processing unit 41 creates a new sound point cloud map 61 based on the sound point cloud data 51 .

[0055] In step 107, the measurement data processing unit 41 creates an image 80 to be presented to the worker, based on the light point cloud map 62. Then, the measurement data processing unit 41 outputs the created image 80 to the operation device 30. The operation device 30 displays the image 80.

[0056] The image 80 shows the environment around the working device 10 (the environment including the work target 50) in color. Therefore, the worker can obtain color information of the work target 50. In addition, the image 80 is a three-dimensional image, and the worker can observe the work target 50 while operating the operation device 30 to change the display (for example, while enlarging, reducing, rotating, etc.).

[0057] When the process of step 107 is completed, the process returns to step 100 and the process of step 100 is executed.

[0058] In this embodiment, for example, from the start of work until the water becomes more turbid, a process (step 102) of updating the light point cloud map 62 with the light point cloud data 52 is performed, and after the water becomes more turbid, a process (steps 103, 104, 105) of updating the light point cloud map 62 with the sound point cloud data 51 is performed, and when the water becomes less turbid thereafter, step 102 is performed again. This process switching can be repeated until the end of work. As a result, even if the water becomes turbid or the turbidity changes due to the deformation or movement of the work object 50 during work, the light point cloud map 62 can be updated while reflecting the deformation or movement of the work object 50 without being affected by the turbidity of the water.

[0059] As described above, the underwater work device according to this embodiment can perform work even if the water becomes turbid during underwater work, and can provide the worker with a high-resolution, colored image 80. This image 80 is created from the light point cloud map 62 obtained by light-based measurement by the optical measuring instrument 12, and therefore is a colored image with high spatial resolution that can obtain color information of the work object.

[0060] In addition, when the water is very turbid, if the working device 10 moves while the update process of the light point cloud map 62 is being performed in steps 103, 104, and 105, the measurement areas of the sound measuring device 11 and the optical measuring device 12 change. For this reason, it is possible that the points of the sound point cloud data 51 and the points of the sound point cloud map 61 cannot be associated with each other in step 104, and the number of points for which the correspondence relationship Rs can be created is small, or there may be no points for which the correspondence relationship Rs can be created. In order to prevent such a situation from occurring, it is preferable that the measurement data processing unit 41 creates the sound point cloud map 61 and the light point cloud map 62 using the sound point cloud data 51 and the light point cloud data 52 for all of the work areas expected to be required before the water becomes very turbid. EXAMPLES

[0061] An explanation will be given of the operation performed by the underwater work apparatus according to the second embodiment of the present invention. The following mainly explains the differences between the underwater work apparatus according to the second embodiment and the underwater work apparatus according to the first embodiment.

[0062] In the underwater work device of this embodiment, when the water is very murky, the measurement data processing unit 41 of the control device 40 updates the light point cloud map 62 by adding points to the light point cloud map 62 depending on the work content or by taking into account constraints depending on the work content.

[0063] 5 is a block diagram showing an example of the configuration of an underwater work apparatus according to this embodiment. In the underwater work apparatus according to this embodiment, the work apparatus 10 further comprises a work mechanism 15 and a work area acquisition unit 16 in addition to the underwater work apparatus according to the first embodiment.

[0064] The working mechanism 15 is a mechanism that performs work such as cutting and relocation on the work target 50, and includes one or more components. The working mechanism 15 can be composed of, for example, a multi-joint arm, a gripper (grasping member), and a cutting tool. The working mechanism 15 performs work on the work target 50 by an operator of the underwater work device operating the operation device 30.

[0065] The working area acquisition unit 16 acquires the position (three-dimensional coordinates) of each element of the working mechanism 15 relative to the working device 10 and the working environment as working area data. When the working device 10 is performing work on the work object 50, the working area data is data indicating the working area of ​​the working mechanism 15 on the work object 50. The working area acquisition unit 16 can be configured, for example, with a positioning sensor capable of communicating with an externally installed GPS or transponder, or an encoder that measures the angle and translation amount of each element of the working mechanism 15. For example, when the working mechanism 15 is a multi-joint arm, the working area data is joint data of the multi-joint arm.

[0066] Note that, if the working area data can be acquired by estimation from operation instructions to the working mechanism 15 inputted by the worker to the operation device 30, the working device 10 does not need to be equipped with the working area acquisition unit 16. In this case, the working device 10 acquires the coordinates of each element of the working mechanism 15 as working area data from the operation instructions to the working mechanism 15 by the worker. The operation device 30 may also be equipped with a mechanism that allows the worker to directly select a task or working area.

[0067] Fig. 6 is a flowchart showing the flow of processing executed by the measurement data processing unit 41 in the embodiment 2. The flowchart shown in Fig. 6 has step 200 between step 101 and step 103 in the flowchart shown in Fig. 4 in the embodiment 1.

[0068] Steps 200 and 103 will now be described.

[0069] In step 200, the measurement data processing unit 41 acquires working area data from the working area acquisition unit 16, and obtains the working area of ​​the working mechanism 15 from the working area data. Then, the measurement data processing unit 41 adds points to the inside of the working area in the light point cloud map 62, or creates constraint conditions (i.e., constraint conditions for the relational expression Em) when expressing points included in the light point cloud map 62 using points included in the sound point cloud map 61, depending on the work content of the working mechanism 15 on the work target object 50. The measurement data processing unit 41 creates the relational expression Em taking into account these constraint conditions.

[0070] The shape and state of the work object 50 change depending on the work content, such as cutting or relocation. In order to represent such changes in the work object 50, points are added to the light point cloud map 62, and constraint conditions for the relational equation Em are created. Points to be added to the light point cloud map 62 are, for example, new points generated by work performed by the work mechanism 15 (points generated by changes in the work object 50). The measurement data processing unit 41 creates constraint conditions for each relational equation Em (i.e., for each point included in the light point cloud map 62).

[0071] For example, when the work content is cutting the work object 50, the measurement data processing unit 41 can add points within the work area of ​​the light point cloud map 62 to represent a missing area or a cut surface due to the cutting. Also, for example, when the work content is relocation of the work object 50, the measurement data processing unit 41 can add points representing an object that is to be newly measured due to the relocation of the work object 50. Also, for example, when the work content is relocation of the work object 50, the measurement data processing unit 41 can calculate the area T in which the work object 50 held by the working mechanism 15 exists, and create constraint conditions for the relational expression Em so that points within the area T in the light point cloud map 62 are expressed using the center of gravity of multiple points included in the area T in the sound point cloud map 61.

[0072] The measurement data processing unit 41 can determine the points to be added within the working area and the constraint conditions for the relational expression Em based on the characteristics of the working mechanism 15 that have been grasped in advance. For example, when the work content is cutting, the working mechanism 15 is a cutting tool, and the characteristics of the cutting tool such as the shape of the teeth and the position and shape of the cut surface to be generated are known in advance, so the measurement data processing unit 41 adds points representing the cut surface based on these characteristics of the working mechanism 15 (cutting tool). Also, when the work content is relocation, the working mechanism 15 is a gripper, and the characteristics of the gripper such as the size of the gripper and the range over which the fingers spread are known in advance, so the measurement data processing unit 41 creates constraint conditions for the relational expression Em based on these characteristics of the working mechanism 15 (gripper).

[0073] In step 103, when the constraint condition is created in step 200, the measurement data processing unit 41 creates the relational equation Em taking into consideration the created constraint condition. The measurement data processing unit 41 creates the relational equation Em for expressing points included in the light point cloud map 62 using points included in the sound point cloud map 61, taking into consideration the constraint condition created in step 200 for points for which the constraint condition exists. More specifically, the measurement data processing unit 41 creates the relational equation Em for expressing the coordinates of points included in the stored light point cloud map 62 using the coordinates of one or more points included in the stored sound point cloud map 61, taking into consideration the constraint condition. The constraint condition is, for example, a condition that limits points included in the sound point cloud map 61 that can be used to create the relational equation Em.

[0074] In this embodiment, by doing this, the coordinates of the points included in the light point cloud map 62 can be updated according to the work content on the work object 50, and a more accurate light point cloud map 62 and an image 80 based on this light point cloud map 62 can be presented to the worker.

[0075] In the following, an example of the process in step 200 will be described for the case where the work content is cutting the work target 50.

[0076] 7A and 7B are diagrams for explaining an example of the process in step 200 of the flowchart shown in Fig. 6 when the work content is cutting the work object 50. Fig. 7A is a schematic diagram showing the top surface and cross section of the work object 50 to be cut. Fig. 7B is a schematic diagram showing a point cloud G included in the point cloud map of light 62 corresponding to the work object 50 shown in Fig. 7A.

[0077] First, the measurement data processing unit 41 determines the trajectory of the working mechanism 15 from the working area of ​​the working mechanism 15 (e.g., a cutting tool) determined from the working area data and the operation instructions for the working mechanism 15 input by the operator to the operation device 30. Then, the measurement data processing unit 41 calculates the cutting plane U of the work object 50 based on the trajectory of the working mechanism 15 and the characteristics of the working mechanism 15 (e.g., the shape of the teeth of the cutting tool and the position and shape of the cutting plane to be generated). As already mentioned, the characteristics of the working mechanism 15 can be known in advance.

[0078] FIG. 7A shows an example of a cut surface U of the work object 50 calculated by the measurement data processing unit 41.

[0079] Next, the measurement data processing unit 41 calculates a missing area D caused by cutting the work object 50 from the cut surface U and the characteristics of the working mechanism 15 (e.g., a cutting tool). Then, the measurement data processing unit 41 adds multiple points H inside or on the boundary of the missing area D. The number of points H and the intervals between them can be determined arbitrarily. The points H are added to the light point cloud map 62, and the color of the points H can be represented by a value (e.g., a negative value) determined in advance to represent the missing area D caused by the cutting.

[0080] Figure 7B shows an example of a point cloud G contained in the light point cloud map 62, a missing area D of the work object 50 calculated by the measurement data processing unit 41, and a point H added to the light point cloud map 62.

[0081] Finally, the measurement data processing unit 41 creates constraint conditions to be considered when creating the relational expression Em for the points included in the light point cloud map 62. As already described, the relational expression Em is an expression that expresses the points included in the light point cloud map 62 using the points included in the sound point cloud map 61.

[0082] In this embodiment, for example, when a point included in the light point cloud map 62 is defined as point P2 and a point included in the sound point cloud map 61 is defined as point P1, and point P2 is to be expressed using point P1, if a line connecting point P1 and point P2 intersects with a cross section U of the work object 50, a constraint condition is created that prohibits expressing point P2 using point P1. This constraint condition is a condition that prohibits two points sandwiching the cross section U from being related to each other, and is a condition that indicates that the cross section U is a surface obtained by cutting the work object 50 (the cross section U represents an opening of the work object 50). Due to this constraint condition, two points located on either side of the cross section U cannot be related to each other across the cross section U.

[0083] In this manner, the measurement data processing unit 41 adds points to the light point cloud map 62 and creates constraints on the relational equation Em in step 200 of FIG. 6, and then in step 105, updates the light point cloud map 62 taking into account the constraints that limit the points that can be related to each other.

[0084] In this embodiment, the light point cloud map 62 can be updated taking into consideration constraints according to the work content. For example, in cutting work on the work object 50, the position of the cutting surface U and the shape of the opening of the cutting surface U can be presented to an operator of the underwater work equipment.

[0085] The present invention is not limited to the above-described embodiments, and various modifications are possible. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to an embodiment having all of the configurations described. It is also possible to replace a part of the configuration of one embodiment with the configuration of another embodiment. It is also possible to add the configuration of another embodiment to the configuration of one embodiment. It is also possible to delete a part of the configuration of each embodiment, or to add or replace another configuration. [Explanation of symbols]

[0086] 10...work device, 11...acoustic measuring device, 12...optical measuring device, 13...position measuring unit, 14...turbidity estimation unit, 15...work mechanism, 16...work area acquisition unit, 20...turbidity, 30...operation device, 40...control device, 41...measurement data processing unit, 50...work object, 51...sound point cloud data, 52...light point cloud data, 53...position data, 54...turbidity, 61...sound point cloud map, 62...light point cloud map, 80...image, D...missing area, Em...relationship, G...point cloud, Rs...correspondence relationship, U...cut surface.

Claims

1. A work device that is equipped with an acoustic measuring device and an optical measuring device and performs work on a work object present underwater; A control device for controlling the working device; An operating device used to operate the working device; Equipped with The acoustic measuring instrument acquires point cloud data representing a surface of an object present around the working device as sound point cloud data, using sound; The optical measuring instrument uses light to acquire point cloud data representing a surface of an object present around the working device and having information about a color of the surface, as light point cloud data; the control device includes a sound point cloud map which is a map of objects present around the working device, the sound point cloud map being created from the sound point cloud data, and a light point cloud map which is a map of objects present around the working device, the light point cloud data being created from the light point cloud data, the control device obtains a relationship between the light point cloud map and the sound point cloud map, and a relationship between the sound point cloud map and the sound point cloud data, and updates the light point cloud map using the sound point cloud data and the sound point cloud map. An underwater work device characterized by:

2. The work device includes a turbidity estimation unit that determines the turbidity of water in which the work object exists, The control device includes: If the turbidity is greater than a predetermined reference value, the light point cloud map is updated using the sound point cloud data and the sound point cloud map; If the turbidity is equal to or less than the reference value, the data of the light point cloud map is replaced with the light point cloud data, and the light point cloud map is updated.

2. An underwater work device according to claim 1.

3. The control device creates the light point cloud map using the light point cloud data, and creates the sound point cloud map using the sound point cloud data.

2. An underwater work device according to claim 1.

4. The working device includes a working mechanism that performs a work on the work target, and a working area acquisition unit that acquires a working area of ​​the working mechanism, the control device adds a point to the inside of the work area in the light point cloud map according to the work content of the work mechanism.

2. An underwater work device according to claim 1.

5. The control device obtains a relationship between the light point cloud map and the sound point cloud map by creating a relational equation for expressing points included in the light point cloud map using points included in the sound point cloud map.

2. An underwater work device according to claim 1.

6. The working device includes a working mechanism that performs a work on the work target, the control device creates constraint conditions when expressing points included in the light point cloud map using points included in the sound point cloud map according to the work content of the work mechanism; The control device creates the relational expression taking into account the constraint conditions.

6. An underwater work device according to claim 5.

Images