Learning device for detecting underwater targets and device for detecting underwater targets

The system addresses the challenge of varying observation conditions in underwater target detection by employing multiple learning models selected based on observation conditions, enhancing detection accuracy.

JP7579068B2Active Publication Date: 2024-11-07HITACHI LTD
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
JP2020098166
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Filing Date
2020-06-05
Publication Date
2024-11-07
Estimated Expiration
2040-06-05

AI Technical Summary

Technical Problem

Existing underwater target detection systems using artificial intelligence face challenges in maintaining high detection accuracy under various observation conditions due to the dependence on single model data, which is not suitable for all conditions.

Method used

A system that prepares multiple learning models based on observation conditions and selects the appropriate model for detection using a distance calculation and model selection process, incorporating sonar data and observation conditions to enhance accuracy.

Benefits of technology

Enables higher detection accuracy of underwater targets by adapting to varying observation conditions through the use of multiple learning models, improving the precision of target detection.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007579068000001
    Figure 0007579068000001
  • Figure 0007579068000002
    Figure 0007579068000002
  • Figure 0007579068000003
    Figure 0007579068000003
Patent Text Reader

Abstract

To provide a system that realizes highly accurate target detection using many learning models that depend on observation conditions on an underwater target detector that uses a neural network.SOLUTION: A learning model aggregate 114 in which multiple learning models for observation data of underwater targets accumulated together with observation conditions 101 thereof is created, distance calculation means 109 that calculates the distance among calculation means 101 is provided, one or more learning models supported by model selection means 108 are selected based on the distance among the conditions of the learning model aggregate 114, and an underwater target is estimated by inference means 122 based on the extracted learning model.SELECTED DRAWING: Figure 1
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to a technique for detecting underwater targets using observation data, and in particular to a technique for performing learning by machine learning for use in detecting underwater targets.

[0002] The observation data includes sound waves and image data converted from the observed sound waves. [Background technology]

[0003] Conventionally, a conventional technique related to an active sonar for detecting underwater targets is disclosed in Patent Document 1. In Patent Document 1, multiple continuous pulses are transmitted to the target, and the intensity of the reflected echoes is displayed on a screen showing distance on the vertical axis and direction on the horizontal axis, and an assessor with specialized knowledge determines the presence of the target.

[0004] Here, it is conceivable to use a neural network, which is the result of machine learning, to detect underwater targets using active sonar. Specifically, a screen showing the intensity of the reflected echo, with distance on the vertical axis and direction on the horizontal axis, i.e., a B-scope screen, is used as an input image, and the echoes of targets contained therein are detected by a neural network instead of a person to substitute for the detection of the underwater target. Patent Documents 2 and 3 are examples of conventional technologies that apply neural networks to sonar devices that detect underwater targets. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Publication No. 8-129065 [Patent Document 2] JP 2019-200175 A [Patent Document 3] JP 2010-266280 A Summary of the Invention [Problem to be solved by the invention]

[0006] In the above Patent Documents 2 and 3, in the detection of underwater targets using sound waves, the observed sonar data is highly dependent on the observation conditions, and it is difficult to maintain the required high detection accuracy for data under all conditions with a single model data. For this reason, it is conceivable to prepare multiple learning models according to individual conditions and switch between the learning models depending on the conditions.

[0007] It has been difficult for the above-mentioned underwater target detection system using artificial intelligence (AI) to achieve higher detection accuracy in response to various observation conditions. Therefore, an object of the present invention is to provide such an underwater target detection device. Another object of the present invention is to provide a method for constructing such an underwater target detection device. [Means for solving the problem]

[0008] In order to solve the above problems, the present invention prepares multiple learning models for detecting underwater targets, and selects one or more learning models based on the distance between the observation conditions of the observation data and the observation conditions of the multiple learning models.

[0009] More specifically, this is an underwater target detection learning device for detecting underwater targets using observation data, which has a memory unit that stores multiple learning data used to detect the underwater target, each corresponding to an observation condition for the underwater target, an input unit that accepts observation data of the underwater target and the observation conditions of the observation data, a distance calculation unit that calculates the distance between the accepted observation conditions and each of the multiple observation conditions stored in the memory unit, a model selection unit that selects a learning model from the memory unit based on the distance calculated by the distance calculation unit, and a learning unit that uses the learning model selected by the model selection unit to learn about the detection of the underwater target, and which enables inferences regarding detection of the underwater target accepted by the input unit using the learning model learned by the learning unit.

[0010] Another aspect of the present invention includes an underwater target detection device that detects underwater targets related to the navigation of an underwater vehicle or a ship using observation data, the underwater target detection device having a memory unit that stores multiple learning data used to detect the underwater target, each corresponding to an observation condition for the underwater target, an input unit that accepts observation data of the underwater target and the observation conditions of the observation data, a distance calculation unit that calculates the distance between the accepted observation conditions and each of the multiple observation conditions stored in the memory unit, a model selection unit that selects a learning model from the memory unit based on the distance calculated by the distance calculation unit and the navigation route along which the underwater vehicle or the ship is planned to navigate, and an inference unit that performs inference regarding detection of the underwater target accepted by the input unit based on the learning model selected by the model selection unit. Effect of the Invention

[0011] According to the present invention, it is possible to detect underwater targets with higher detection accuracy by applying AI and adapting to various observation conditions. [Brief description of the drawings]

[0012] [Figure 1] FIG. 2 is a diagram showing a functional configuration of the first embodiment. [Diagram 2] FIG. 2 is a diagram showing a list of observation data used in the first embodiment. [Diagram 3] FIG. 4 is a diagram showing the configuration of an example of observation conditions used in the first embodiment. [Figure 4] FIG. 11 is a diagram illustrating an example of a method for converting a condition distance in the first embodiment. [Diagram 5] FIG. 2 is a diagram showing an example of one expression format of a condition space map used in the first embodiment. [Figure 6] FIG. 1 is a diagram showing an example in which a learning model collection used in the first embodiment is depicted on a condition space map. [Figure 7] FIG. 1 is a diagram showing the contents of a learning model collection used in the first embodiment cut out in a certain condition space. [Figure 8] FIG. 4 is a diagram showing the contents of condition data used in the first embodiment of the present invention. [Figure 9] FIG. 11 is a diagram showing an example of an evaluation value of a condition distance used in the first embodiment of the present invention. [Figure 10] FIG. 2 is a diagram showing inference performance data used in the first embodiment of the present invention. [Figure 11] FIG. 2 is a diagram showing an example of a set of multi-class learning models used in the first embodiment of the present invention. [Figure 12] FIG. 11 is a diagram showing a functional configuration of a second embodiment. [Figure 13] FIG. 11 is a diagram showing the configuration of a loop program used in the second embodiment. [Figure 14] FIG. 1 is a system configuration diagram of a first embodiment. [Figure 15] 1 is a flowchart showing an outline of a process according to a first embodiment. [Figure 16] FIG. 1 is a system configuration diagram of a second embodiment. [Figure 17] 13 is a flowchart showing an outline of a process according to the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] Each embodiment of the present invention will be described below. In each embodiment, an underwater target is detected in order to detect obstacles in the navigation of an underwater vehicle or ship, or to perform a search using an underwater vehicle or ship. Examples of underwater targets include schools of fish, mines, people, and other floating objects and drifting objects. These are merely examples, and do not exclude other objects. In each embodiment, at least a part of an underwater target may appear on the water or on the sea. Note that underwater vehicles or ships include moving bodies that navigate on the water or on the sea, as well as on the water or underwater. In the following description, when simply referring to a ship, it is assumed to mean an underwater vehicle or ship. EXAMPLES

[0014] First, a first embodiment of the present invention will be described. Fig. 1 is a functional configuration diagram of this embodiment. In Fig. 1, 100 denotes sonar data, 101 denotes observation conditions when the sonar data 100 is measured or observed, 102 denotes annotation data, 103 denotes model data, 104 denotes performance data, and 105 denotes a target position inference result. In addition, 106 denotes a system interface, 107 denotes a model read / write means, 108 denotes a model selection means, 109 denotes a distance calculation means, 110 denotes a data division means, 111 denotes a learning process means, 112 denotes a preprocessing means, and 113 denotes an inference process means. In addition, 114 denotes a learning model collection, 115 denotes a condition map, 116 denotes condition data, 117 denotes replicated raw data, 118 denotes raw data, 119 denotes a multi-class learning model, and 120 denotes inference performance data. Furthermore, 121 denotes a learning means, 122 denotes an inference means, 123 denotes an output multidimensional score of the learning means, 124 denotes an output multidimensional score of the inference means, 125 denotes an evaluation function of the score, 126 denotes an evaluation function of the score, and 127 denotes a performance analysis means. Note that a pointer to the raw data 118 is also included in this system.

[0015] This system can be implemented in an underwater target detection system shown in Fig. 14. Before describing each function and process of this system, the system configuration of this embodiment will be described with reference to Fig. 14. In Fig. 14, this system is composed of an underwater target detection learning device 10 and a terminal device 20 installed in a data center or the like, and an underwater target detection device 34 provided on a ship 30 capable of navigating on the water. Furthermore, the ship 30 is provided with a sonar transmission / reception device 31, sensors 32, and a wireless device 33.

[0016] Here, the sonar transceiver 31 has a function of converting the detected sound waves into an electrical signal. Then, the sonar transceiver 31 itself or another device converts the electrical signal into sonar data 100, which is image data. This sonar transceiver 31 includes a so-called active sonar.

[0017] The sensors 32 also have a function of detecting at least a part of the observation conditions 101. For example, this can be realized by a GPS sensor that measures the position information of the sea area, and a water thermometer that measures the underwater temperature profile, which is the water temperature at each water depth. Note that, among the observation conditions 101, the sea quality, etc. may be determined by using map information previously stored in the underwater target detection device 34.

[0018] The underwater target detection learning device 10 and the underwater target detection device 34 are connected via a network 40. In this case, the underwater target detection device 34 is connected to the network 40 via a wireless device 33 installed on the ship 30. The network 40 may be a public line such as the Internet, or a dedicated line.

[0019] By being connected in this manner, the underwater target detection device 34 can notify the underwater target detection learning device 10 of sonar data 100 and observation conditions 101 measured or identified by the underwater target detection device 34. Furthermore, the learning results of the underwater target detection learning device 10 can be notified to the underwater target detection device 34, enabling the underwater target detection device 34 to make inferences regarding the detection of underwater targets. Note that the above-mentioned sonar data 100, observation conditions 101, and learning results may have large data volumes. In this case, it is desirable to exchange these via a storage medium 60 such as an HDD or DVD, rather than via the network 40.

[0020] Next, the details of the learning device 10 for detecting underwater targets will be described. The learning device 10 for detecting underwater targets can be realized by a so-called computer, and is composed of a processing unit 11 such as a CPU, a storage unit 12 such as a memory, and an input / output unit 13 such as an interface for inputting and outputting data. Each function is realized by the processing unit 11 performing calculations according to a program deployed in the storage unit 12. That is, the model selection unit 1080, the distance calculation unit 1090, the data division unit 1100, the learning process unit 1110, the preprocessing unit 1120, the learning unit 1210, and the performance analysis unit 1270 are realized as programs or as entities performing calculations according to the programs. Here, each of these components in the processing unit 11 has the following correspondence with each functional block shown in FIG. 1.

[0021] The model selection unit 1080 corresponds to the model selection means 108. The distance calculation unit 1090 corresponds to the distance calculation means 109. The data division unit 1100 corresponds to the data division means 110. The learning process unit 1110 corresponds to the learning process means 111. The preprocessing unit 1120 corresponds to the preprocessing means 112. The learning unit 1210 corresponds to the learning means 121. The performance analysis unit 1270 corresponds to the performance analysis means 127.

[0022] Moreover, it is preferable that the underwater target detection learning device 10 is configured as a computer, particularly a server. In this case, the underwater target detection learning device 10 will receive input from a terminal device 20 implemented as a computer and output processing results to the terminal device 20. Furthermore, the underwater target detection learning device 10 is connected to a database 50. The database 50 stores various pieces of information also shown in FIG. 1.

[0023] The underwater target detection device 34 can also be implemented as a computer. That is, it has a processing unit, a storage unit, and an input / output unit. The processing unit has an inference processor 1130 and an inference unit 1220. The inference processor 1130 and the inference unit 1220 are realized as a program or as a subject of calculation according to the program, similar to the configuration of the processing unit 11. The inference processor 1130 corresponds to the inference process means 113 in FIG. 1, and the inference unit 1220 corresponds to the inference means 122.

[0024] Furthermore, the input / output unit 13 in Fig. 14 corresponds to the system interface 106 and the model read / write means 107 in Fig. 1. Note that some functions of the system interface 106 and the model read / write means 107 also correspond to the input / output unit of the underwater target detection device 34.

[0025] The underwater target detection learning device 10 and the underwater target detection device 34 may be integrated. In this case, it is desirable to implement the functions of the underwater target detection learning device 10 in the underwater target detection device 34. Furthermore, the division of functions between the underwater target detection learning device 10 and the underwater target detection device 34 is not limited to that shown in FIG. 14. For example, the underwater target detection device 34 may have a distance calculation unit 1090.

[0026] Next, an overview of the processing in this embodiment will be described, followed by a detailed description of various types of information used in this embodiment and the processing using each piece of information.

[0027] 15 is a flow chart showing an outline of the processing in this embodiment. In the following explanation, the processing will be mainly performed by each functional block shown in FIG.

[0028] First, in step S1, the system interface 106 receives sonar data 100, observation conditions 101, and annotation data 102, which is teaching data.

[0029] Next, in step S2, the distance calculation means 109 calculates the distance between the observation condition input in step S1 and each observation condition of the learning model collection 114. This enables the distance calculation means 109 to identify the distance to the learning model associated with each observation condition of the learning model collection 114.

[0030] Next, in step S3, the model selection means 108 functions differently during learning and during inference. During learning, one position within the learning model collection 114 for storing the learning model obtained as a result of learning is determined. At this time, the observation conditions are stored in the condition data 116, and the position of the condition map 115 is specified. During inference, a learning model to be used for learning in step S4 is selected from the learning model collection 114 based on the distance of each learning model specified in step S2.

[0031] Next, it is preferable that step S4 is executed only during learning, and the learning process means 111 and the learning means 121 cooperate to execute the learning process. It is preferable that step S4 is skipped during inference, and the process proceeds to step S5.

[0032] Finally, step S5 is preferably executed only during inference, and the inference process means 113 and the inference means 122 use the learning model selected in step S3 to execute inference processing for detecting underwater targets in the sonar data 100. In step S5, it is preferable to skip this processing during learning.

[0033] This concludes the description of the outline of the processing in this embodiment. The details of the processing in this embodiment will now be described with reference to FIG.

[0034] Here, the functional block for executing step S1 in Fig. 15 will be described. On this premise, the operation of the sonar transceiver 31 of this embodiment will be described. Note that storage of information described below means appropriate storage in the database 50, memory unit 12, memory unit of the underwater target detection device 34, etc. shown in Fig. 1.

[0035] In an active sonar device, which is a type of sonar transceiver 31, sound waves are generated at an underwater target, and the strength of the reflected sound waves is analyzed based on the response time to estimate the direction and distance of the target. The observation data, i.e., sonar data, is converted into image data in B-scan format, and a system is formed in which the target is detected based on the shape of the reflected wave from the target. When displaying an image in B-scan format, resolution conversion and emphasis processing are performed for display in order to project it on a display device. The sonar data 100 before this conversion is specifically called raw data. Raw data is mainly used when learning and inference is performed with AI. Data with converted resolution may be used depending on the type of sonar data and the AI ​​processing.

[0036] Sonar data 100, observation conditions 101, and annotation data 102 are input via a system interface 106 of this system, and learning by machine learning is performed by a learning means 121. As described above, sonar data 100 is in a data format in which image data, i.e., two-dimensional array data whose values ​​are signal strength, is input in a time series. Observation conditions 101 include position information of the ocean area where observation is being performed, the depth of the target, sea quality (sand, mud, etc.), water temperature, speed of the observation vessel, attitude of sonar transmitter / receiver 31, etc.

[0037] The annotation data 102 is teacher data for learning, and is information on the type, position, and attitude of a preset target. The annotation data 102 is data obtained by converting the latitude, longitude, and depth information of a target on a GPS system into position information on two-dimensional image data, or the results of calculating the position and attitude of an underwater target for each frame from sonar data displayed on a B-scope by an experienced surveyor.

[0038] The machine learning model used in this system adopts a model format suitable for object detection in image data. AI methods have been published that detect objects of a fixed shape and any size from an image, and by using one of these, it is possible to estimate the positions of multiple objects and the image range of the objects. In this case, multiple objects with different shapes are recognized as independent classes. A learning model that supports multiple classes is called multi-class compatible.

[0039] Other inputs and outputs of the system interface 106 include model data 103, performance data 104, and target position inference result 105. The model data 103 indicates a learning model (trained data) generated by this system or the learning model itself stored in an external system. The performance data 104 is data related to the performance of the learning model held by this system, such as the target detection accuracy, and is input and output to and from an external system (not shown). The target position inference result 105 is data on the inference result of a target that has been inferred by setting a learning model in this system and inputting new sonar data 100.

[0040] 1 are functional blocks that execute the internal processing of this system. An overview of this internal processing will be given below.

[0041] The model read / write means 107 is a functional block that executes input / output of the learning model, and performs input / output based on the observation conditions 101 for the learning model generated by the learning means 121 and the learning models included in the learning model collection 114 .

[0042] Next, a functional block for executing step S2 in Fig. 15 will be described. The distance calculation means 109 calculates the distance between the observation condition 101 input via the system interface 106 and each observation condition included in the learning model collection 114. In this way, the distance calculation means 109 specifies the calculated distance as the distance of each learning model stored in the learning model collection 114. Here, the distance calculation means 109 calculates the condition distance for each learning model stored in the learning model collection 114, and stores the result as a condition map 115.

[0043] The distance calculation means 109 further stores the input condition data 116 and reads and processes it as necessary. To calculate the condition distance, two pieces of condition data are required: the input observation condition 101 and the observation condition of the learning model to be processed. Here, the condition data 116 is shown in FIG. 8.

[0044] Next, a functional block for executing step S3 in Fig. 15 will be described. Model selection means 108 is a functional block for selecting one or more learning models that meet suitable conditions from the learning model collection 114. This condition determination uses the distance between two conditions calculated by distance calculation means 109, that is, the condition distance. This condition distance includes the distance of each learning model described above.

[0045] The evaluation functions 125 and 126 are functions that calculate a single scalar value from the multidimensional scores. In cases where a target is detected as belonging to one of multiple classes, the evaluation function can use a function that extracts the maximum value of the multidimensional scores. In this case, the class corresponding to the maximum score is estimated as the class of the target.

[0046] The performance analysis means 127 is a functional block that compares the inference results of the target selected by the evaluation function 126 with the annotation data, which is the teaching data, and evaluates the degree of agreement across the entire group of inference data to calculate the inference accuracy.

[0047] The inference performance data 120 is data indicating the above-mentioned inference accuracy and is used when selecting a learning model by the model selection means 108. This inference performance data 120 is shown in Fig. 10. The model selection means 108 takes into consideration the inference performance indicated by the inference performance data 120, and excludes a learning model from those to be selected if the inference performance does not satisfy a predetermined inference performance threshold value.

[0048] Next, step S4 in Fig. 15 will be described. Data division means 110 executes processing to divide a data set of sonar data and annotation data that are input to learning process means 111 and inference process means 113. Time-series sonar data 100 is assigned to learning or inference on an image frame basis for each time period, and divided into the respective data sets.

[0049] The learning process means 111 operates the learning means 121 for the above-mentioned learning sonar data set, and controls a series of processes for generating a learning model. The generated learning model is stored in the learning model collection 114 based on the observation conditions and is used.

[0050] The learning means 121 uses annotation data corresponding to the input observation data. In its internal processing, the learning means 121 calculates a loss function using the annotation data 102 and executes optimization of the learning model.

[0051] Furthermore, the preprocessing means 112 performs preprocessing on the data to be handled when performing various processes such as learning. The preprocessing includes data shaping and the like.

[0052] Next, a description will be given of the functional block that executes step S5 in Fig. 15. Inference process means 113 operates inference means 122 for the above-mentioned sonar data set for inference, and controls a series of processes for calculating an inference result.

[0053] The inference means 122 outputs data on target candidates. Specifically, it outputs a score that quantifies the positions of a finite number of target candidates and their likelihood in response to a set threshold value for inference. If there are multiple targets, it outputs a multidimensional score corresponding to multiple classes. This concludes the explanation of the functional blocks shown in FIG. 1, and below, we will further explain the details of various information, including the processing content. In particular, the details of the distance calculation in step S2 will also be shown below.

[0054] First, the correspondence between the observation conditions 101 and the sonar data 100 will be described with reference to FIG. 2. Here, the description will be given taking as an example two types of conditions, namely, sea area identification information A1 to A4 and depths D1 to D4. Reference numeral 201 denotes the type of sea area, reference numeral 202 denotes the depth, and the names of sonar data corresponding to these two conditions (identification reference numeral 203, etc.) are shown in correspondence with each other. In FIG. 2, sonar data corresponding to all conditions is shown as having been observed and being available. However, in the actual field, not all conditions have been acquired, and there is also unacquired data (or missing data) (specifically, the missing data is identified and exception processing is performed).

[0055] Figure 3 shows four types of observation conditions as an example: ocean area information, depth, target type, and target size. Ocean area information a consists of latitude, longitude, and bottom sediment. Depth b uses the depth of the target (here, it is assumed that depth1, etc. will be replaced with a specific numerical value in meters). Target type c is data that stores the name or identification code to distinguish them. Size d stores the size of the target as a numerical value in centimeters.

[0056] The details of the ocean area information in Figure 2 correspond to observation conditions a in Figure 3, and the details of the depth in Figure 2 correspond to depth b in Figure 3.

[0057] Here, the information on the distance calculation in step S2 and the details of this distance calculation will be described with reference to FIG. 4, FIG. 5, and FIG. 9. FIG. 4 shows an example of the conversion of the above four types of observation conditions to distances, that is, a method of converting condition distances. Each condition is associated with a scalar quantity (distance). a, b, c, and d are conversion tables to scalar quantities X1, X2, X3, and X4 corresponding to the sea area (A), depth (D), type of target (O), and size of target (S), respectively. If each type of condition is considered as an independent dimension, a multidimensional space can be imagined with X1 to X4 shown in FIG. 4 as coordinate axes. FIG. 4 can be considered as a method of calculating distances along each coordinate axis. It is necessary to define separately the method of measuring distances in the above multidimensional space.

[0058] FIG. 9 is an example showing the result of the condition distance calculated using the condition distance conversion method of FIG. 4, that is, the evaluation value. The numerical value of the condition distance shown here is based on the sea area A1 and depth D1. The numerical range of the condition distance is from 0 to 330. It is not necessary to calculate the condition distance using all the conditions entered in the observation conditions 101. Any calculation method of the condition distance that is compatible with the application of the learning model will suffice.

[0059] An example of the calculation method of the condition distance used in the calculation of FIG. Condition distance (d1, d2) = k1 * |X1(d1) - X1(d2)| + k2 * | X2(d1) - X2(d2) | (where k1 = 1, k2 = 1) (Equation 1) For two observation points d1 and d2, the condition distance (d1, d2) is calculated based on a formula that depends only on the two conditions, the distance between the ocean conditions X1, and the distance between the depths X2. Here, k1 and k2 are weighting coefficients for the types (dimensions) of each condition.

[0060] It is possible to associate observed data and the learning model that is the result of that learning with a multidimensional space that uses the dimensions of each of the above conditions, and these are called a condition map.

[0061] Figure 5 is an example of a condition map, where observation data is associated with each observation point. The data is plotted at positions corresponding to the scalar quantities X1 and X2 of each dimension. If there is a condition for which no observation has been performed, that point is a missing point.

[0062] Next, the selection conditions in the model selection in step S3 will be described. The model selection means 108 selects one or more learning models based on the calculated distance in accordance with the selection conditions stored in advance. The selection conditions include, for example, "neighborhood" and "whole."

[0063] First, the neighborhood will be described. When the neighborhood is used, the model selection means 108 selects the learning model with the shortest calculated distance. In this case, the model selection means 108 may select multiple learning models in order of shortest distance as a neighborhood condition. Furthermore, the model selection means 108 may be configured to select one or more learning models within a predetermined distance as a neighborhood condition. In this case, if there is no learning model within the predetermined distance, it is desirable for the model selection means 108 to select one or more learning models in order of shortest distance. This makes it possible to select a learning model that precisely matches the observation conditions 101 observed at that time.

[0064] Next, the whole will be described. Here, the model selection means 108 calculates and uses a representative value of the distance (nearest distance) of each learning model in the learning model collection 114 to the nearest model. For this purpose, the model selection means 108 performs selection so that the representative value of the nearest distance of the selected learning model is equivalent to all of the learning models in the learning model collection 114. Here, equivalent includes not only the same value, but also being within a predetermined difference. Moreover, the representative value includes the average value, the median value, etc. By using the whole of the above, it becomes possible to select a more versatile learning model.

[0065] As described above, the selection condition may be any condition for selecting one or more, that is, a single or multiple learning models.

[0066] For this reason, the model selection means 108 can use a combination of multiple selection conditions. For example, the model selection means 108 selects a learning model with the shortest distance and multiple learning models with the same representative value. This makes it possible to select a learning model that is balanced between the neighborhood and the whole.

[0067] Furthermore, in each of the above selection conditions, an upper limit may be set on the number of learning models to be selected, or that number may be set to a fixed number.

[0068] 1, the fact that the learning means 121 outputs three output multidimensional scores 123 indicates that learning has been performed for three learning models. In other words, this indicates that the model selection means 108 has selected three learning models. Similarly, the fact that the inference means 122 outputs three multidimensional scores 124 also indicates that the model selection means 108 has selected three learning models.

[0069] Next, information regarding the learning process in step S4 will be described.

[0070] FIG. 6 shows learning models plotted on the condition map 115. That is, it is the result of learning performed by the learning process means 111 and the learning means 121. In this figure, learning models are plotted at the position of the condition where no observation was performed. They are learning models 504, 505, and 506. These are the results of learning performed using the union of two or more pieces of learning data. Learning model 504 is the result of using the union of sea area A1 and sea area A2 at depth D1, and learning models 505 and 506 are similar.

[0071] FIG. 7 is a diagram showing the contents of a learning model collection. Here, the names of the learning models corresponding to each condition are listed for the dimensions of two conditions. The learning model from the union of the observation data shown in the explanation of FIG. 6 is also added to the data contents of the learning model collection. As in this example, the learning model collection can be used as a mechanism for expanding the data contents through operation. As an example of the names of the learning models, M1111 is a learning model corresponding to sea area A1 and depth D1, and as another example, M12-1-1-1 is a learning model corresponding to the condition of the union of sea areas A1 and A2 and depth D1.

[0072] Finally, a description will be given of the estimation process in step S5 in Fig. 15. A process will be described in which new observation conditions 101 and their sonar data 100 are input to the system of this embodiment and inference is performed using the collection of generated learning models.

[0073] Sonar data 100 and observation conditions 101 are input to system interface 106, and model selection means 108 reads one or more learning models that are optimal for the input observation conditions from learning model collection 114 and temporarily stores them in multi-class learning model 119. Inference process means 113 operates inference means 122 for each frame of the input sonar data, outputs multidimensional score 124, and calculates an inference result using evaluation function 126. The inference process means 113 outputs the calculated inference result to target position inference result 105 via system interface 106.

[0074] An example of the configuration of the multi-class learning model 119 is shown in FIG. 11. In this example, three learning models M1, M2, and M3 are used. In the above process, the inference means 122 performs inference using the three learning models included in this multi-class learning model, and outputs the score values ​​corresponding to each model as a multidimensional score. The process of the evaluation function 126 uses the process of calculating the maximum value described above. As a result, one of the three selected learning models becomes a candidate for the target, and its estimated position and the probability of the target are calculated.

[0075] As described above, according to the first embodiment, a learning model is generated for the sonar data 100 for each observation condition 101, a collection of learning models is constructed using the learning models, and one or more optimal learning models corresponding to the observation conditions can be selected for newly input sonar data. Therefore, even for sonar data 100 of an underwater target whose learning model depends on the observation conditions 101, the underwater target can be detected with high accuracy in accordance with the observation conditions at the input point. EXAMPLES

[0076] Next, a second embodiment of the present invention will be described. In this embodiment, an underwater target detection device 34' installed on a ship 30 executes model selection. Fig. 16 shows a system configuration including this underwater target detection device 34'. The underwater target detection device 34' is connected to a sonar transceiver 31, sensors 32, and a wireless device 33, similar to the underwater target detection learning device 10 of the first embodiment. Although not shown in Fig. 16, information can be exchanged with the underwater target detection learning device 10 via a network 40 or a storage medium 60, similar to the first embodiment.

[0077] Furthermore, the underwater target detection device 34' is connected to a terminal device 36 and a database 35 that are capable of inputting and outputting information to the underwater target detection device 34'. The terminal device 36 and the database 35 may be configured integrally with the underwater target detection device 34'.

[0078] Next, the underwater target detection device 34' will be described in detail. The underwater target detection device 34' is realized by a computer, similar to the underwater target detection learning device 10 and the underwater target detection device 34 of the first embodiment. That is, the underwater target detection device 34' has a processing unit 341, a memory unit 342, and an input / output unit 343. Therefore, each function of the underwater target detection device 34' is realized by the processing unit 341 performing calculations in accordance with the program deployed in the memory unit 342.

[0079] Here, a program for realizing each function of the underwater target detection device 34' will be described. The underwater target detection device 34' has the following configuration in addition to the inference process unit 1130 and the inference unit 1220 provided in the underwater target detection device 34 of the first embodiment.

[0080] First, the underwater target detection device 34' has a model selection unit 1080, a distance calculation unit 1090, a data division unit 1100, and a preprocessing unit 1120. These are also included in the underwater target detection learning device 10 of the first embodiment. In addition to these, the underwater target detection device 34' has a route processing unit 14021. Note that, apart from the route processing unit 14021, the functions are the same as those of the first embodiment. The function of the route processing unit 14021 will be described later with reference to Figs. 12 and 13.

[0081] Next, various information stored in the database 35 will be described. The database 35 has annotation data 102, performance data 104, target position inference result 105, learning model collection 114, condition map 115, and condition data 116. These are the same information stored in the database 50 in the first embodiment. In addition, in the second embodiment, the database 35 stores a route program 1403.

[0082] Here, the functional configuration of the second embodiment will be described with reference to Fig. 12. The functional configuration of Fig. 12 realizes the functions of the underwater target detection device 34' shown in Fig. 16. Here, Fig. 12 differs from Fig. 1 in the following ways. A route processing means 1402 and a route program 1403 are added. Here, the route processing means 1402 corresponds to the route processing unit 14021 in Fig. 16. Also, in Fig. 12, the learning process means 111, the duplicated data 117, the raw data 118, the inference performance data 120, the learning means 121, the output multidimensional score 123, the score evaluation function 125, and the performance analysis means 127 are omitted from Fig. 1.

[0083] Next, the processing of this embodiment will be described with reference to the flowchart of Fig. 17. Due to the difference in configuration described above, this embodiment executes the same processing as steps S1, S2, and S5 in Fig. 15. Here, the description of these steps will be omitted, and steps S3-1 and S3-2, which are different from the first embodiment, will be described.

[0084] 17, (a) steps S1 to S3-1 indicate advance preparation processing, and (b) steps S3-2 and S5 indicate processing during navigation. Therefore, (a) may be executed by the underwater target detection learning device 10.

[0085] As in the first embodiment, the processing entity will be described using the functional block shown in FIG.

[0086] First, in step S3-1, the route processing means 1402 creates a route program 1403 that associates a navigation route, observation conditions, and a learning model. For this purpose, the model selection means 108 extracts a learning model based on the distance calculated in step S2. This extraction is performed in the same manner as the model selection in step S3 in the first embodiment.

[0087] However, the route processing means 1402 receives input information such as observation conditions according to a planned navigation route, and extracts each learning model for each navigation route in unit time order.

[0088] Then, the route processing means 1402 creates a route program 1403 in which the extracted learning model and the corresponding observation conditions are associated with the travel route, and stores this in the database 35. An example of this route program 1403 is shown in Fig. 13. In Fig. 13, a route (underwater route information) indicating a navigation route is associated with an ocean area (A), depth (D), and type of underwater target (O) indicating observation conditions, and a model indicating a learning model. Note that, in Fig. 13, one observation condition and one learning model are associated with a navigation route, but multiple observation conditions and learning models may be associated with each other.

[0089] The route processing means 1402 also sorts and records the route program 1403 in chronological order of the unit time for which the travel route is scheduled. The route processing means 1402 may also record the scheduled travel time in the route program 1403.

[0090] Next, during navigation, in step S3-2, the model selection means 108 appropriately selects a learning model corresponding to the route or navigation time from the route program 1403. That is, the model selection means 108 selects a learning model corresponding to the planned navigation route from the route program 1403 in the order sorted in chronological order.

[0091] Then, during navigation, the inference process is executed in step S5 using the selected learning model.

[0092] As in the first embodiment, the processing in the second embodiment may also function in cooperation with the learning device 10 for detecting underwater targets.

[0093] According to the second embodiment, a learning model that is appropriately selected according to the time series of observation conditions corresponding to a preset navigation route is stored, and a learning model that corresponds to the observation conditions at each navigation point is read out to estimate the target. Therefore, in this embodiment, it is possible to detect underwater targets with high accuracy according to the navigation route.

[0094] In each of the above embodiments, the system involving learning can prevent an increase in operational costs due to the maintenance and management of learning data and learning models.

[0095] Furthermore, although each embodiment is directed to a ship, the present invention is also applicable to a moving body or means for detecting an underwater target, such as an aircraft. [Explanation of symbols]

[0096] 100... sonar data, 101... observation condition input, 102... annotation data, 105... target position inference result, 114... learning model collection, 108... model selection means, 109... distance calculation means, 115... condition map, 121... learning means, 122... inference means, 119... multi-class learning model

Claims

1. 1. An underwater target learning device for detecting an underwater target using observation data, comprising: a storage unit that stores a plurality of learning data used for detecting the underwater target in association with observation conditions for the underwater target; an input unit that receives observation data of the underwater target and observation conditions of the observation data; a distance calculation unit that calculates a distance between the received observation condition and each of a plurality of observation conditions stored in the storage unit; a model selection unit that selects a learning model from the storage unit based on the distance calculated by the distance calculation unit; a learning unit that performs learning for detecting the underwater target using the learning model selected by the model selection unit, An underwater target detection learning device that enables inference regarding detection of the underwater target received by the input unit using the learning model learned by the learning unit.

2. 2. The learning device for detecting underwater targets according to claim 1, The model selection unit is an underwater target detection learning device that selects a learning model associated with the observation condition under which the distance calculated by the distance calculation unit is shortest.

3. 2. The learning device for detecting underwater targets according to claim 1, The model selection unit is an underwater target detection learning device that selects a plurality of learning models based on the distance calculated by the distance calculation unit.

4. 4. The learning device for detecting underwater targets according to claim 3, The model selection unit is Calculating a first representative value of distances under a plurality of observation conditions stored in the storage unit; Identifying a plurality of observation conditions that result in a second representative value equivalent to the first representative value; A learning device for detecting underwater targets that selects a learning model associated with the identified plurality of observation conditions.

5. 5. The learning device for detecting underwater targets according to claim 3, The model selection unit further selects a plurality of learning models in order of shortest distance calculated by the distance calculation unit.

6. An underwater target detection device that detects underwater targets related to underwater navigation or ship navigation using observation data, a storage unit that stores a plurality of learning data used for detecting the underwater target in association with observation conditions for the underwater vehicle or the underwater target, respectively; an input unit that receives observation data of the underwater target and observation conditions of the observation data; a distance calculation unit that calculates a distance between the received observation condition and each of a plurality of observation conditions stored in the storage unit; a model selection unit that selects a learning model from the memory unit based on the distance calculated by the distance calculation unit and a navigation route along which the ship is scheduled to navigate; An underwater target detection device having an inference unit that performs inference regarding detection of the underwater target received by the input unit based on the learning model selected by the model selection unit.

7. 7. The underwater target detection device according to claim 6, Further, the present invention has a route processing means for extracting a learning model based on the distance calculated by the distance calculation unit, creating a route program in which the extracted learning model is associated with observation conditions and the navigation route corresponding to the learning model, and storing the route program in the storage unit; The model selection unit is an underwater target detection device that selects the learning model from the route program.

8. 8. The underwater target detection device according to claim 7, The route processing means is an underwater target detection device that extracts a learning model associated with the observation conditions under which the distance calculated by the distance calculation unit is the shortest.

9. 8. The underwater target detection device according to claim 7, The route processing means is an underwater target detection device that extracts multiple learning models based on the distance calculated by the distance calculation unit.

10. 10. The underwater target detection device according to claim 9, The route processing means includes: Calculating a first representative value of distances under a plurality of observation conditions stored in the storage unit; Identifying a plurality of observation conditions that result in a second representative value equivalent to the first representative value; An underwater target detection device that extracts a learning model associated with the identified multiple observation conditions.

11. 11. The underwater target detection device according to claim 9, The route processing means further extracts a plurality of learning models in order of the shortest distance calculated by the distance calculation unit of the underwater target detection device.

Citation Information

Patent Citations

  • B-scope display device of active sonar

    JP1996129065A

  • Discrimination device

    JP2010266280A

  • Machine control device

    JP2019138152A

  • Fish species estimation system, fish species estimation method, and program

    JP2019200175A

  • Deep Learning and Intelligent Sensing System Integration

    US20200050202A1