Measurement device, measurement method, and program

Abstract

This measurement device (1) comprises an imaging control unit (21) for causing an imaging unit (14) to image a prescribed object of interest in the water and a vibration measurement unit (23) for measuring sound waves in the water by measuring the vibration frequency of the object of interest on the basis of images that were made to be captured by the imaging control unit (21).

Description

[Technical Field]

[0001] The present technology relates to a measurement device, a measurement method, and a program, and in particular to a technology for measuring underwater acoustic waves. [Background technology]

[0002] A measuring device has been proposed that measures the abundance of phytoplankton by irradiating it with excitation light of a predetermined wavelength to excite the phytoplankton and measuring the intensity of the fluorescence emitted from the excited phytoplankton (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2019-165687 Summary of the Invention [Problem to be solved by the invention]

[0004] Meanwhile, various sound waves are emitted in the ocean (underwater), such as low-frequency sound waves caused by crustal activity and sound waves emitted by marine organisms such as whales. A hydrophone using a piezoelectric microphone is known as a device for measuring acoustic waves generated in the sea.

[0005] However, hydrophones have difficulty detecting extremely low frequency sound waves, for example, those below 1 Hz, and are therefore not yet capable of efficiently measuring sound waves in water.

[0006] Therefore, the purpose of this technology is to efficiently measure sound waves propagating through water. [Means for solving the problem]

[0007] The measuring device according to the present technology includes an imaging control unit that causes an imaging unit to capture an image of a predetermined target object in water, and a By measuring the vibration of the target object and a measuring unit that measures sound waves in water. This allows the measurement device to measure underwater sound waves with a simple configuration. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a diagram illustrating the configuration of a measurement device 1 according to an embodiment. [Figure 2] 1A and 1B are diagrams illustrating a target object and the movement of the target object. [Figure 3] FIG. 10 is a diagram illustrating an example of a measurement setting. [Figure 4] FIG. 10 is a diagram illustrating an example of an operation time sheet. [Figure 5] 1 is a diagram illustrating a target object vibrating due to sound waves generated in water. FIG. [Figure 6] FIG. 10 is a diagram illustrating a vibration measurement process. [Figure 7] FIG. 10 is a diagram illustrating a rule-based distance measurement process. [Figure 8] FIG. 10 is a diagram illustrating an image serving as training data. [Figure 9] This is a model diagram of deep learning. [Figure 10] FIG. 1 is a diagram illustrating a first example of use. [Figure 11] FIG. 10 is a diagram illustrating a second example of use. [Figure 12] FIG. 10 is a diagram illustrating the configuration of a measurement device according to a modified example. [Figure 13] FIG. 10 is a diagram illustrating the configuration of a measurement device according to a modified example. DETAILED DESCRIPTION OF THE INVENTION

[0009] The embodiments will be described below in the following order. <1. Configuration of the measuring device> <2.About the target object> <3. Measurement method as an embodiment> <4. Vibration measurement processing> <5. Distance measurement processing> <6. Usage example> <7. Other configuration examples of measuring device> <8. Summary of embodiments> <9. This Technology>

[0010] <1. Configuration of the measuring device> First, the configuration of a measurement device 1 according to an embodiment of the present technology will be described. The measuring device 1 is a device that measures sound waves propagating in water by measuring the vibrations of a target object, such as a microorganism or a fine particle present in water, such as in the sea.

[0011] Here, the microorganisms that serve as the target objects include aquatic microorganisms such as phytoplankton, zooplankton, and larvae of aquatic organisms that exist in water. Furthermore, the fine particles that serve as the target objects include microplastics, dust, sand, marine snow, air bubbles, etc. However, these are merely examples, and the target objects may also be other objects.

[0012] The sound waves that propagate through water include various types of sound waves, such as low-frequency sound waves caused by tectonic activity and sound waves emitted by marine organisms such as whales, and are low-frequency sound waves, for example, between 0.1 Hz and 20 Hz, although they may also have frequencies of 20 Hz or higher.

[0013] Fig. 1 is a diagram illustrating the configuration of a measurement device 1. As shown in Fig. 1, the measurement device 1 includes a main body 2 and an illumination unit 3. Note that the illumination unit 3 may be provided within the main body 2.

[0014] The main body 2 includes a control unit 10 , a memory 11 , a communication unit 12 , a gravity sensor 13 , an imaging unit 14 , and a lens 15 .

[0015] The control unit 10 includes a microcomputer having, for example, a CPU (Central Processing Unit), a ROM (Read Only Memory), and a RAM (Random Access Memory). and performs overall control of the measurement device 1. In the first embodiment, the control unit 10 functions as an imaging control unit 21, a class identification unit 22, and a vibration measurement unit 23. The imaging control unit 21, the class identification unit 22, and the vibration measurement unit 23 will be described in detail later. The control unit 10 also reads data stored in the memory 11 , stores data in the memory 11 , and transmits and receives various data to and from external devices via the communication unit 12 .

[0016] The memory 11 is composed of a nonvolatile memory. The communication unit 12 performs wired or wireless data communication with external devices. The gravity sensor 13 detects gravitational acceleration (direction of gravity) and outputs the detection result to the control unit 10. Note that the measuring device 1 does not necessarily have to include the gravity sensor 13.

[0017] The imaging unit 14 includes a vision sensor 14a and / or an imaging sensor 14b. The vision sensor 14a is a sensor called a DVS (Dynamic Vision Sensor) or an EVS (Event-Based Vision Sensor). The vision sensor 14a captures an image of a predetermined underwater imaging range through a lens 15.

[0018] The vision sensor 14a is an asynchronous image sensor in which a plurality of pixels each having a photoelectric conversion element are arranged two-dimensionally, and a detection circuit for detecting address events in real time is provided for each pixel. Note that an address event is an event that occurs in response to the amount of light incident on each address assigned to each of the plurality of pixels arranged two-dimensionally, such as when the current value based on the charge generated in the photoelectric conversion element, or the amount of change in the current, exceeds a certain threshold.

[0019] The vision sensor 14a detects whether an address event has occurred for each pixel, and if an address event has been detected, reads out the pixel signal from the pixel where the address event occurred as pixel data. That is, the vision sensor 14a asynchronously acquires pixel data according to the amount of light incident on each of the multiple pixels arrayed two-dimensionally.

[0020] In the vision sensor 14a, a pixel signal read operation is performed for a pixel where an address event is detected, which allows for much faster readout than a synchronous image sensor in which a readout operation is performed for all pixels at a predetermined frame rate, and also allows for a smaller amount of data to be read out for one frame.

[0021] Therefore, by using the vision sensor 14a, the measurement device 1 can detect the movement of the target object more quickly. Furthermore, the vision sensor 14a can reduce the amount of data and also reduce power consumption.

[0022] The imaging sensor 14b is, for example, a CCD (Charge Coupled Device) or CMOS (Complementary Metal-Oxide-Semiconductor) type image sensor, and has a plurality of pixels, each having a photoelectric conversion element, arranged two-dimensionally. The imaging sensor 14b captures an image of a predetermined imaging range through the lens 15 at regular intervals according to the frame rate to generate image data. Note that in the measuring device 1, a zone plate, a pinhole plate, or a transparent plate can be used instead of the lens 15.

[0023] Vision sensor 14a and image sensor 14b are arranged so as to capture images of substantially the same imaging range through lens 15. For example, a half mirror (not shown) may be arranged between vision sensor 14a and image sensor 14b and lens 15, so that one light split by the half mirror is incident on vision sensor 14a and the other light is incident on image sensor 14b.

[0024] The illumination unit 3 is driven under the control of the control unit 10, and irradiates light onto the imaging range of the imaging unit 14. The illumination unit 3 can switch between irradiating light of different wavelengths, and irradiates light of different wavelengths in increments of 10 nm, for example.

[0025] <2.About the target object> 2 is a diagram illustrating a target object and its movement. In FIG. 2, an image of the target object is shown in the upper part, and the direction of movement of the target object is indicated by an arrow in the lower part.

[0026] As shown in Figure 2, the target objects include microorganisms, marine snow, seabed sand, smoke, and air bubbles. It is known that some microorganisms exhibit tacticity when irradiated with light of a specific wavelength. Here, tacticity is an innate behavior of an organism in response to light (an external stimulus). Therefore, when irradiating a microorganism with tacticity with light of a specific wavelength, the microorganism moves in accordance with its tacticity.

[0027] Marine snow is made up of particles such as plankton excreta, dead bodies, or decomposed matter that exist in the ocean, and moves through the ocean by sinking (in the direction of gravity). The seabed sand is, for example, particles such as sand that have settled on the seabed, and moves in a swirling manner due to the seabed current. Smoke is a phenomenon that occurs when geothermally heated water erupts from hydrothermal vents on the ocean floor. The hot water erupting from hydrothermal vents can reach temperatures of several hundred degrees Celsius and contains a wealth of dissolved heavy metals and hydrogen sulfide. When the water reacts with the seawater, black or white smoke forms, swirling upwards. The bubbles may be natural gases such as methane or carbon dioxide leaking (erupting) from the seabed, or carbon dioxide leaking from reservoirs artificially injected for CCS (carbon dioxide capture and storage), and move upward from the seabed.

[0028] In this way, target objects, including not only microorganisms but also microparticles, exist that move in a known direction, and in the measurement device 1 of the first embodiment, microorganisms and microparticles whose movement direction is known are identified as target objects.

[0029] <3. Measurement method as an embodiment> Next, an outline of a method for measuring a target organism as an embodiment will be described. FIG. 3 is a diagram illustrating an example of a measurement setting.

[0030] The control unit 10 performs measurements in accordance with pre-specified measurement settings as shown in Fig. 3. The measurement settings specify a measurement start condition, an operation time sheet for the lighting unit 3, an identification program (identification method), a vibration measurement program (vibration measurement method), and a measurement end condition.

[0031] The measurement start conditions specify the conditions for starting the measurement, such as the time to start the measurement or receiving a measurement start command input via the communication unit 12.

[0032] The operation time sheet specifies a time sheet for operating the lighting unit 3. For example, the operation time sheet specifies that the wavelength should be changed in 10 nm increments in the range from 400 nm to 700 nm, i.e., 400 nm, 410 nm, . . . , 690 nm, and 700 nm, with an off period between each wavelength.

[0033] In this way, the operation time sheet specifies what wavelength of light the illumination unit 3 should irradiate onto the imaging range and at what timing. Note that the reason for providing a time when the illumination unit 3 is turned off, i.e., when no light is irradiated, is to capture an image of the light when the target object is emitting (excited). In addition, by inserting an off time between each wavelength, it becomes easier for the asynchronous vision sensor 14a to detect events by wavelength.

[0034] The identification program specifies a program (method) for identifying the type of target object, such as identification by machine learning or rule-based identification.

[0035] The vibration measurement program specifies a program (method) for measuring the vibration of a target object, such as measurement by machine learning or rule-based measurement.

[0036] The measurement termination conditions specify the conditions for terminating the measurement, such as the time to terminate the measurement or receiving a measurement termination command input via the communication unit 12.

[0037] 4 is a flowchart showing the processing steps of the measurement method. As shown in FIG. 4, in step S1, the control unit 10 reads external environment information, which will be described in detail later. Then, in step S2, the control unit 10 determines whether the measurement start condition specified in the measurement settings is met. Then, the control unit 10 repeats steps S1 and S2 until the measurement start condition is met.

[0038] On the other hand, if the measurement start condition is met (Yes in step S2), in step S3 the imaging control unit 21 switches between irradiating light of different wavelengths from the illumination unit 3 in accordance with the operation time sheet specified in the measurement settings. Furthermore, the imaging control unit 21 causes the imaging unit 14 to capture an image of the imaging range each time the wavelength of the light irradiated from the illumination unit 3 and its on / off state are switched, and acquires pixel data and image data. Thereafter, in step S4 the class identification unit 22 executes class identification processing.

[0039] In the class identification process, the class identification unit 22 identifies (specifies) the type of target object based on the image (pixel data and image data) captured by the imaging unit 14. The class identification unit 22 derives identification information from the image captured by the imaging unit 14 and compares it with definition information stored in the memory 11 to detect the target object.

[0040] Definition information is provided for each target object and is stored in memory 11. The definition information includes the type of target object, its moving direction, and image information.

[0041] The movement information is information detected mainly based on an image captured by the vision sensor 14a, and is information based on the movement of a target object as shown in the lower part of Fig. 3. When the target object is a microorganism, the movement information is information such as the direction (positive or negative), speed, and trajectory of movement relative to the light source. When the target object is a fine particle, the movement information is information such as the direction, speed, and trajectory of movement.

[0042] The image information is information detected mainly based on an image captured by the image sensor 14b, and is information about the external shape of the target object. Note that the image information may also be information detected based on an image captured by the vision sensor 14a.

[0043] The definition information may also include the direction of gravity detected by the gravity sensor 13 and external environment information acquired via the communication unit 12. Possible external environment information includes depth, position coordinates (latitude and longitude of the measurement point, and plane rectangular coordinates), electrical conductivity, temperature, pH, gas (e.g., methane, hydrogen, helium) concentration, metal concentration (e.g., manganese, iron), etc.

[0044] The class identification unit 22 detects an object present in the imaging range based on the image (pixel data) captured by the vision sensor 14a. For example, the class identification unit 22 creates one image (frame data) based on pixel data input within a predetermined period, and detects a group of pixels within a predetermined range in which movement is detected in the image as one object.

[0045] Furthermore, the class identification unit 22 tracks the object between multiple frames by pattern matching, etc. Then, based on the tracking results of the object, the class identification unit 22 derives the moving direction, speed, and trajectory as identification information.

[0046] The period in which the class identifying unit 22 generates an image from pixel data may be the same as or shorter than the period (frame rate) in which the image sensor 14b acquires image data.

[0047] Furthermore, for the object for which identification information has been derived, the class identification unit 22 extracts an image portion corresponding to the object from the image data input from the image sensor 14b. Then, based on the extracted image portion, the class identification unit 22 derives external features as identification information by image analysis. Note that the image analysis can be performed using a known method, and therefore a description thereof will be omitted here.

[0048] The class identification unit 22 determines which type of target object it is by comparing the wavelength of light emitted by the illumination unit 3 and the identification information derived for the detected object (movement direction, trajectory, speed, external features) with definition information according to a specified identification program. Here, for example, if the derived identification information of the object is within the range indicated in the definition information of the target object, the class identification unit 22 will identify the derived object as being of the type indicated in the definition information.

[0049] The definition information is stored in the memory 11 by a method that differs for each identification program. For example, in a rule-based identification program, the definition information is set in advance by the user and stored in the memory 11. In addition, in a machine learning identification program, the definition information is generated and updated by machine learning in a learning mode and stored in the memory 11.

[0050] Thereafter, in step S5, the vibration measurement unit 23 determines whether the type of the detected object is an object whose vibration is to be measured, based on the type of object identified by the class identification unit 22. Here, it is determined whether the object does not move by itself, that is, whether it is an object (e.g., a fine particle) that moves only due to the current in the water (vibration caused by sound waves).

[0051] If the type of the detected object is not a target object whose vibration is to be measured (No in step S5), the process proceeds to step S7. On the other hand, if the type of the detected object is a target object whose vibration is to be measured (Yes in step S5), in step S6, the vibration measuring unit 23 performs a vibration measurement process to detect and measure sound waves by measuring the vibration of the object based on the image captured by the imaging unit 14. An example of the vibration measurement process will be described later.

[0052] In step S7, the control unit 10 determines whether the measurement termination condition specified in the measurement settings is met. The control unit 10 then repeats steps S3 to S7 until the measurement termination condition is met, and if the measurement termination condition is met (Yes in step S7), the control unit 10 ends the process.

[0053] <4. Vibration measurement processing> Fig. 5 is a diagram illustrating a target object vibrating due to sound waves generated in water. Fig. 6 is a diagram illustrating the vibration measurement process. In Fig. 5, the target object is shown as a white circle, and the vibrating target object is shown with a white circle superimposed on it.

[0054] For example, suppose that multiple target objects are lined up in the depth direction underwater, as shown in Figure 5. The third, fourth, and fifth target objects from the top are vibrating due to sound waves generated in the water. The fourth target object from the top has a higher vibration frequency than the third and fifth target objects from the top.

[0055] In such a case, the first, second, sixth and seventh target objects from the top do not appear in the images captured by the vision sensor 14a at predetermined intervals because they do not move. On the other hand, the third, fourth, and fifth objects from the top will appear in different positions (pixels) in each image. In this way, a target object vibrating due to sound waves generated in water will appear to move in images captured at predetermined intervals.

[0056] Therefore, the vibration measurement unit 23 performs a distance measurement process to measure the distance to the target object in the imaging direction for images captured continuously by the vision sensor 14a at predetermined time intervals. Furthermore, the vibration measurement unit 23 calculates the acceleration between frames based on the measured distance in the imaging direction, as shown on the left side of Fig. 6. The distance measurement process will be described in detail later.

[0057] Thereafter, the vibration measuring unit 23 performs normalization on the calculated acceleration waveform by overlap processing, window function processing, etc. Then, the vibration measuring unit 23 performs a fast Fourier transform on the normalized acceleration waveform as shown in the center of Fig. 6, and calculates the amplitude for each frequency as shown on the right side of Fig. 6.

[0058] Then, the vibration measuring unit 23 identifies (measures) frequency components (f1 to f4 in the figure) whose amplitudes are greater than a predetermined threshold value, among the frequency components whose amplitudes have been calculated, as the frequencies of sound waves emitted in water.

[0059] <5. Distance measurement processing> Next, the distance measurement process will be described. The vibration measurement unit 23 executes the distance measurement process based on a rule-based or machine learning distance measurement program. Here, specific examples of the rule-based distance measurement process and the machine learning distance measurement process will be described. Note that the method for calculating the distance to the target object is not limited to these, and other methods may also be used.

[0060] 5.1 Rule-based distance measurement processing 7 is a diagram illustrating the rule-based distance measurement process, in which the focal length f of the vision sensor 14a is stored in the memory 11 as known information.

[0061] Furthermore, statistical information (average size H) for each target object is stored in the memory 11. This information is registered in advance by the user as a database.

[0062] Once the target object is identified from the image based on the pixel data, the vibration measurement unit 23 reads the average size H of the target object and the focal length f of the vision sensor 14a from the memory 11. Thereafter, the vibration measurement unit 23 calculates the longitudinal length s of the image 42 of the target object captured on the imaging surface 40 based on, for example, the number of pixels in which the image 42 is captured.

[0063] Furthermore, the vibration measuring unit 23 calculates the distance D from the measuring device 1 to the target object 41 in the imaging direction using equation (1). D=fH / s (1)

[0064] In this way, the vibration measuring unit 23 calculates (measures) the distance D from the measuring device 1 to the actual target object 41 every time an image based on pixel data is acquired (every time a target object is detected from the image).

[0065] [5.2 Machine learning distance measurement processing] Fig. 8 is a diagram illustrating an image serving as training data, and Fig. 9 is a model diagram of deep learning.

[0066] In the distance measurement process of machine learning, machine learning is performed using images as training data, such as those shown in FIG. 8, to generate a model (architecture) for the distance measurement process.

[0067] Specifically, images of a known target object captured by the vision sensor 14a are prepared in advance in five patterns where the distance from the measuring device 1 to the target object in the imaging direction is 1 mm, 5 mm, 10 mm, 100 mm, and 200 mm, and 31 patterns where the wavelength of the irradiated light is changed in 10 nm increments from 400 nm to 700 nm, for a total of 153 patterns.

[0068] Then, for each of the prepared images, the vibration measurement unit 23 detects a group of pixels within a predetermined range where movement has been detected as a target object, and resizes the group of pixels to 32 pixels x 32 pixels to generate an image that serves as training data, as shown in Figure 8.

[0069] Note that Fig. 8 shows a portion of an image that is training data. Here, in the ocean, the attenuation rate of light of approximately 500 nm is low, and the attenuation rates of light of wavelengths smaller than approximately 500 nm and larger than approximately 500 nm increase with distance from approximately 500 nm. Furthermore, the greater the distance from the measuring device 1 to the target object, the lower the light arrival rate.

[0070] 8, in an image of a target object, the closer the target object is to the measuring device 1 and the closer the wavelength of the irradiated light is to 500 nm, the clearer the target object appears. On the other hand, the farther the target object is from the measuring device 1 and the closer the wavelength of the irradiated light is to 500 nm, the less clear the target object appears or the image is not captured at all.

[0071] When the images serving as training data are resized, the vibration measurement unit 23 performs machine learning on the training data consisting of these images using a deep neural network, as shown in FIG. 9. This model is composed of, for example, five convolution layers (Conv1 to Conv5), three pooling layers (Max Pooling), and two fully connected layers (FC). Then, by performing machine learning, As a result, a model is generated that finally outputs a one-dimensional classification vector having five elements from Distance 1 mm to Distance 200 mm, and the model is stored in the memory 11.

[0072] Machine learning in such a deep neural network is performed for each target object, and a model for each target object is generated and stored in memory 11.

[0073] Once the class identification unit 22 has identified the type of target object, the vibration measurement unit 23 reads out a model of the identified type from the memory 11. The vibration measurement unit 23 also resizes the target object portion in the image captured by the vision sensor 14a to 32 pixels by 32 pixels and inputs the resized image into the read-out model. This outputs the value of a one-dimensional classification vector having five elements ranging from Distance 1 mm to Distance 200 mm. The vibration measurement unit 23 then outputs (measures) the element with the highest value among the five elements (any of Distance 1 mm to Distance 200 mm) as the distance in the imaging direction of the target object.

[0074] <6. Usage example> In the following, examples of use of the measurement device 1 will be described with reference to use example 1 and use example 2.

[0075] [6.1 Usage example 1] Figure 10 is a diagram explaining Usage Example 1. On the left side of Figure 10, the target object is indicated by a black circle, and the vibration of the target object is indicated by black circles displayed side by side. Figure 10 also shows three different layers (layers A, B, and C) distributed in the ocean depth direction, and shows how the sound waves (vibrations) detected in each layer are different.

[0076] It is known that the speed of sound in the ocean is determined by three parameters: water temperature, salinity, and water pressure. Salinity has the least influence as a parameter determining the speed of sound, and it varies little from place to place. On the other hand, the water temperature decreases as the depth increases from the surface to a depth of about 800 m. The speed of sound also slows as the water temperature decreases. This layer where the water temperature decreases with depth is called the thermocline (Layer A). Furthermore, from the ocean surface to a depth of 1,500 m or more, the water temperature remains constant at about 4°C, while the water pressure increases as the depth increases. The speed of sound also increases as the water pressure increases. This layer is called the deep-sea isothermal layer (Layer C). Between the thermocline and the deep-sea isothermal layer (800m to 1500m deep), there is a depth where the speed of sound is minimal, and this acts as a kind of lens on the sound propagation path, so energy radiated within this layer tends to remain within the layer. Therefore, even with moderate acoustic power, sound can propagate over very long distances in this layer. This layer, where the speed of sound is slower than in other layers, is called the deep-sea sound channel or SOFAR layer.

[0077] Therefore, in Use Example 1, the measurement device 1 is used to explore the SOFAR layer, where sound waves propagate over long distances.

[0078] 10, the measuring device 1 is moved in the water depth direction (gravity direction). While the measuring device 1 is being moved, images are sequentially captured by the imaging unit 14. The class identification unit 22 identifies the type of object each time an image is captured, and when the object to be measured for vibration is identified, the vibration measurement unit 23 measures the vibration of the object as described above.

[0079] As a result, as shown on the right side of Figure 10, sound waves with different frequencies and amplitudes are measured in the SOFAR layer of Layer B than in the thermocline of Layer A and the deep-sea isothermal layer of Layer C. In particular, sound waves with frequencies with large amplitudes are measured in the SOFAR layer of Layer B.

[0080] Therefore, the vibration measuring unit 23 detects the SOFAR layer by comparing the frequency and amplitude of the sound waves that would be measured in the SOFAR layer with the frequency and amplitude of the measured sound waves.

[0081] The frequency and amplitude of the sound waves that will be measured in the SOFAR layer may be stored in advance in the memory 11 by the user (rule-based), or may be learned by machine learning.

[0082] [6.2 Usage example 2] Figure 11 is a diagram illustrating Usage Example 2. Usage Example 2 involves exploring hydrothermal vents. Here, sound waves of a predetermined frequency are generated from the hydrothermal vents. Therefore, in use case 2, hydrothermal vents are explored by detecting and approaching the sound waves emitted from the vents.

[0083] In Usage Example 2, the measuring device 1 is mounted on a moving unit 100 such as an underwater robot or a submarine. The moving unit 100 may be moved by a user operation, or may be moved under the control of the control unit 10. Here, a case where the moving unit 100 moves based on the control of the control unit 10 will be described as an example.

[0084] First, the control unit 10 moves the moving unit 100, for example, randomly, as shown by the arrow in Fig. 11. Then, during the random movement, images are sequentially captured by the imaging unit 14. The class identification unit 22 identifies the type of object each time an image is captured, and once the object whose vibration is to be measured is identified, the vibration measurement unit 23 measures the vibration of the object as described above.

[0085] The vibration measurement unit 23 also determines whether the frequency f11 of the sound waves generated from the hydrothermal vent is being detected. Then, for example, at time t1, the sound waves of frequency f11 are not being detected, so the moving unit 100 is further moved randomly.

[0086] Thereafter, when a sound wave of frequency f11 is detected at time t2, for example, the control unit 10 moves the moving unit 100 in a direction in which the amplitude of frequency f11 increases.

[0087] In this way, by moving the moving unit 100 in a direction in which the amplitude of the frequency f11 increases, the location where the amplitude of the frequency f11 is greatest at time t3, for example, is detected as a hydrothermal vent.

[0088] In this way, the control unit 10 moves the imaging unit 14 (measuring device 1) based on the measured sound waves of the predetermined frequency. The control unit 10 also moves the imaging unit 14 in a direction that increases the amplitude of the measured sound waves of the predetermined frequency.

[0089] <7. Other configuration examples of measuring device> The embodiment is not limited to the specific example described above, and various modified configurations can be adopted.

[0090] In the above embodiment, the measurement device 1 is provided with one illumination unit 3. However, the number of illumination units 3 is not limited to one, and a plurality of illumination units 3 may be provided.

[0091] Fig. 12 is a diagram illustrating the configuration of a modified measurement device 200. As shown in Fig. 12, the modified measurement device 200 includes one main body 2 and two illumination units 3. The two illumination units 3 are arranged so that they can irradiate light in directions perpendicular to each other, and can irradiate the imaging range with light of different wavelengths. In such a measuring device 200, since light of different wavelengths can be irradiated from two lighting units 3, identification information of target objects (microorganisms) that exhibit mobility in light of different wavelengths can be derived in a single measurement, allowing measurements to be performed efficiently.

[0092] Fig. 13 is a diagram illustrating the configuration of a modified measuring device 300. As shown in Fig. 13, the modified measuring device 300 includes two main body units 2 and one illumination unit 3. The two main body units 2 are arranged so as to be able to capture images in directions orthogonal to each other. In such a measuring device 300, images can be captured using two main body units 2 (imaging units 14), making it possible to detect three-dimensional movement of the target object and perform measurements more efficiently. When two main bodies 2 are provided, one of the main bodies 2 may be provided with only the imaging unit 14.

[0093] In the above-described embodiment, the imaging unit 14 includes the vision sensor 14a and the image sensor 14b. However, the imaging unit 14 may include only one of the vision sensor 14a or the image sensor 14b, as long as it can capture an image that allows measurement of information about the position of the target object in the imaging direction. The imaging unit 14 may also include a SPAD (Single Photon Avalanche Diode) sensor instead of the vision sensor 14a and the image sensor 14b.

[0094] In the above embodiment, the type of target object is identified by deriving identification information based on the pixel data acquired by the vision sensor 14a and the image data acquired by the imaging sensor 14b. However, other methods may be used to identify the type of target object as long as the type can be identified based on at least one of the pixel data acquired by the vision sensor 14a and the image data acquired by the imaging sensor 14b.

[0095] In the above embodiment, the vibration of the target object is measured after the type of the target object is identified by the class identification unit 22. However, the measurement device 1 may also be configured to measure the vibration of the target object without identifying the type of the target object.

[0096] <8. Summary of embodiments> As described above, the measuring device 1 of the embodiment includes an imaging control unit 21 that causes the imaging unit 14 to capture an image of a specified target object in water, and a measuring unit (vibration measuring unit 23) that measures sound waves in water based on the image captured by the imaging unit 14. This allows the measuring device 1 to measure underwater sound waves with a simple configuration. The measuring device 1 can also measure sound waves with an ultra-low frequency of less than 1 Hz. Therefore, the measurement device 1 can efficiently measure sound waves propagating through water.

[0097] In the measuring device 1 according to the present technology described above, the imaging unit 14 may include a vision sensor 14a that asynchronously acquires pixel data in accordance with the amount of light incident on each of a plurality of pixels arranged two-dimensionally. This makes it possible to read out only the pixel data of the pixel where the event occurred and to measure the target object based on that pixel data. Therefore, the measuring device 1 can achieve high-speed imaging, reduced power consumption, and low calculation costs for image processing due to automatic separation from the background.

[0098] In the measuring device 1 according to the present technology described above, it is conceivable that the measuring unit measures the frequency of the sound wave by measuring the frequency of the vibration of the target object. This allows the measuring device 1 to measure underwater sound waves based on the vibrations of the target object that appears in the image captured by the imaging unit 14.

[0099] The measuring device 1 according to the present technology described above may include a classification unit (class classification unit 22) that identifies the type of target object based on the image captured by the imaging unit, and the measuring unit may measure sound waves based on the type of target object identified by the classification unit. This makes it possible to avoid measuring sound waves based on the vibrations of a target object of a type that is not vibrated (moved) by sound waves when such a target object is detected. Therefore, the measuring device 1 can measure the sound wave with high accuracy.

[0100] In the measuring device 1 according to the present technology described above, it is conceivable that the measuring unit measures sound waves when the type of target object identified by the identifying unit is a type that does not move by itself. This makes it possible to avoid measuring sound waves based on movements that are not caused by sound waves. Therefore, the measuring device 1 can measure the sound wave with high accuracy.

[0101] In the measuring device 1 according to the present technology described above, the measuring section may measure the frequency of vibration of the target object based on images captured by the imaging section at predetermined time intervals. This allows the measurement device 1 to measure underwater sound waves with a simple configuration.

[0102] In the measuring device 1 according to the present technology described above, it is conceivable that the imaging section sequentially captures images while moving the imaging range. This makes it possible to detect sound waves over a wide area underwater.

[0103] The measuring device 1 according to the present technology described above may include a movement control unit (control unit 10) that moves the imaging unit, and the movement control unit may move the imaging unit based on sound waves of a predetermined frequency measured in the measuring unit. This makes it possible to search for the source of the sound waves to be detected and to find the layer from which the sound waves to be detected are generated.

[0104] In the measuring device 1 according to the present technology described above, the movement control section may move the measuring section in a direction that increases the amplitude of the sound wave of the predetermined frequency measured by the measuring section. This makes it possible to identify the position of the source of the sound wave to be detected.

[0105] In the measuring device 1 according to the present technology described above, it is conceivable that the measuring unit detects a specific layer in the water based on the measured sound waves. In the above-described measuring device 1 according to the present technology, the layer to be detected is considered to be a layer in which the speed of sound is slower than other layers. This makes it possible to find the layer where the sound waves to be detected are generated.

[0106] In the measurement method according to the present technology described above, an image of a predetermined target object in water is captured by an imaging unit, and sound waves in water are measured based on the image captured by the imaging unit. The program according to the present technology causes an imaging unit to capture an image of a predetermined target object in water, and causes an information processing device to execute a process of measuring underwater sound waves based on the image captured by the imaging unit.

[0107] Such a program can be recorded in advance on a HDD as a recording medium built into a device such as a computer, or on a ROM in a microcomputer having a CPU. Alternatively, the software may be temporarily or permanently stored (recorded) on a removable recording medium such as a flexible disk, a CD-ROM (Compact Disc Read Only Memory), an MO (Magneto Optical) disk, a DVD (Digital Versatile Disc), a Blu-ray Disc (registered trademark), a magnetic disk, a semiconductor memory, a memory card, etc. Such removable recording media may be provided as a so-called package software. Such a program can be installed onto a personal computer or the like from a removable recording medium, or can be downloaded from a download site via a network such as a LAN (Local Area Network) or the Internet.

[0108] Furthermore, such a program is suitable for providing a wide range of information processing devices according to the embodiments. For example, by downloading the program to a mobile terminal device such as a smartphone or tablet, a mobile phone, a personal computer, a game device, a video device, a PDA (Personal Digital Assistant), or the like, the device can function as the information processing device of the present disclosure.

[0109] The effects described in this specification are merely examples and are not limiting, and other effects may also be present.

[0110] <9. This Technology> The present technology can also be configured as follows. (1) an imaging control unit that causes the imaging unit to capture an image of a predetermined target object in water; a measurement unit that measures underwater sound waves based on the image captured by the imaging unit; A measuring device comprising: (2) The imaging unit Equipped with a vision sensor that asynchronously acquires pixel data according to the amount of light incident on each of multiple pixels arranged two-dimensionally. The measuring device described in (1). (3) The measurement unit The frequency of the sound wave is measured by measuring the frequency of the vibration of the target object. A measuring device according to (1) or (2). (4) an identification unit that identifies the type of target object based on the image captured by the imaging unit; The measurement unit measures the sound waves based on the type of the target object identified by the identification unit. A measuring device according to any one of (1) to (3). (5) The measurement unit measures the sound waves when the type of the target object identified by the identification unit is a type that does not move by itself. (4) The measuring device according to (4). (6) The measurement unit The vibration frequency of the target object is measured based on the images captured by the imaging unit at predetermined time intervals. (3) A measuring device according to the present invention. (7) The imaging unit sequentially captures images while moving the imaging range. A measuring device according to any one of (1) to (6). (8) a movement control unit that moves the imaging unit, The movement control unit The moving unit is moved based on the sound wave of the predetermined frequency measured by the measuring unit. A measuring device according to any one of (1) to (7). (9) The movement control unit The sound wave of the predetermined frequency measured by the measuring unit is moved in a direction in which the amplitude of the sound wave increases. (8) The measuring device according to (8). (10) The measurement unit Detecting specific layers in the water based on measured sound waves A measuring device according to any one of (1) to (9). (11) The layer to be detected is one in which the speed of sound is slower than other layers. (10) The measuring device according to (10). (12) an imaging unit to capture an image of a predetermined target object in the water; Measures underwater acoustic waves based on the image captured by the imaging unit. Measurement method. (13) an imaging unit to capture an image of a predetermined target object in the water; Measures underwater acoustic waves based on the image captured by the imaging unit. A program that causes a measuring device to execute processing. [Explanation of symbols]

[0111] 1. Measuring equipment 3. Lighting section 10 Control Unit 14 Imaging unit 14a Vision sensor 14b Image sensor 21 Imaging control unit 22 Class Identification Unit 23 Vibration measurement section

Claims

1. an imaging control unit that causes the imaging unit to capture an image of a predetermined target object in water; a measurement unit that measures underwater sound waves by measuring vibrations of the target object based on the image captured by the imaging unit; A measuring device comprising:

2. The imaging unit Equipped with a vision sensor that asynchronously acquires pixel data according to the amount of light incident on each of multiple pixels arranged two-dimensionally. The measuring device according to claim 1 .

3. The measurement unit The frequency of the sound wave is measured by measuring the frequency of the vibration of the target object. The measuring device according to claim 1 .

4. an identification unit that identifies the type of target object based on the image captured by the imaging unit; The measurement unit measures the sound waves based on the type of the target object identified by the identification unit. The measuring device according to claim 1 .

5. The measurement unit measures the sound waves when the type of the target object identified by the identification unit is a type that does not move by itself.

5. The measuring device according to claim 4.

6. The measurement unit The vibration frequency of the target object is measured based on the images captured by the imaging unit at predetermined time intervals. The measuring device according to claim 3 .

7. The imaging unit sequentially captures images while moving the imaging range. The measuring device according to claim 1 .

8. a movement control unit that moves the imaging unit, The movement control unit The moving unit is moved based on the sound wave of the predetermined frequency measured by the measuring unit. The measuring device according to claim 1 .

9. The movement control unit The sound wave of the predetermined frequency measured by the measuring unit is moved in a direction in which the amplitude of the sound wave increases.

9. The measuring device according to claim 8.

10. The measurement unit Detecting specific layers in the water based on measured sound waves The measuring device according to claim 1 .

11. The layer to be detected is one in which the speed of sound is slower than other layers. The measuring device according to claim 10.

12. an imaging unit to capture an image of a predetermined target object in the water; The vibration of the target object is measured based on the image captured by the imaging unit, thereby measuring the sound waves in the water. Measurement method.

13. an imaging unit to capture an image of a predetermined target object in the water; The vibration of the target object is measured based on the image captured by the imaging unit, thereby measuring the sound waves in the water. A program that causes a measurement device to execute processing.

Images