Server, fish species identification system, fish species identification method, and program

Abstract

To provide a server, a fish species discrimination system, a fish species discrimination method, and a program that can improve the accuracy of fish species discrimination results by a machine learning model.SOLUTION: A server 20 capable of communicating with a plurality of underwater detection devices 10 acquires feedback information that associates echo data with fish species from each underwater detection device 10, stores the acquired feedback information in association with an attribute including a fishing method in which at least the underwater detection devices 10 are used, and generates an individual model for fish species discrimination for each attribute by machine learning that uses multiple pieces of feedback information having the same attribute.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a server that generates a machine learning model for fish species discrimination, a fish species discrimination system including the server, a fish species discrimination method for discriminating fish species using the machine learning model, and a program for causing a computer to execute a function of generating a machine learning model for fish species discrimination.

Background Art

[0002] Conventionally, a fish school detection device for detecting a fish school in water has been known. In this type of fish school detection device, ultrasonic waves are transmitted into water, and the reflected waves are received. Echo data corresponding to the intensity of the received reflected waves is generated, and an echo image is displayed based on the generated echo data. A user can confirm a fish school from the echo image and smoothly proceed with the capture of the fish school.

[0003] In this case, it is preferable that the fish species of the fish school on the echo image can be further discriminated. Thereby, the user can efficiently capture the fish of the fish species desired by the user.

[0004] For fish species discrimination, for example, a machine learning algorithm can be used. In this case, the echo data output from the fish school detector is used as input data, and the fish species of the fish school on the echo data is used as teacher data, and learning for the machine learning algorithm is performed to generate a learned model. The fish species (teacher data) of the fish school on the echo data is input by the user based on actual fishing, for example. The following Patent Document 1 describes this type of fish species estimation system.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0006] The method described above can generate a unique machine learning model for each user. However, because the training data is input by the user based on actual catches, the amount of training data is limited. Therefore, it is difficult to improve the accuracy of the fish species classification results by the machine learning model.

[0007] One possible solution to this problem is to aggregate standard data generated by experts for machine learning purposes and use this aggregated standard data as training data for training machine learning algorithms.

[0008] However, the characteristics of the echo data acquired by the fish finder differ depending on the fishing method in which it is used. In other words, the swimming speed and depth of fish differ depending on whether the fish finder is installed in a fixed net or on a fishing boat, and therefore the characteristics of the echo data acquired by the fish finder will differ. For this reason, even if standard data is used directly to train a machine learning algorithm, it is difficult to improve the accuracy of the fish species identification results.

[0009] In view of these challenges, the present invention aims to provide a server, a fish species identification system, a fish species identification method, and a program that can improve the accuracy of fish species identification results using machine learning models. [Means for solving the problem]

[0010] A first aspect of the present invention relates to a server capable of communicating with a plurality of underwater detection devices. The server according to this aspect acquires feedback information from each of the underwater detection devices that associates echo data with fish species, stores the acquired feedback information linked to attributes including at least the fishing method in which the underwater detection device is used, and generates individual models for fish species discrimination for each attribute by machine learning using a plurality of the same set of feedback information. death , The system receives customized information, including changes to the output conditions of the individual model, input by the user of each of the underwater detection devices, and modifies the output conditions of the individual model for the underwater detection device based on the received customized information. The modification of the output conditions includes, even if the predicted probability of the individual model for a specific fish species specified by the user is lower than that for other fish species, if the predicted probability of the specific fish species is above a predetermined threshold, the system prioritizes outputting the specific fish species as the identification result.

[0011] According to the server in this embodiment, feedback information is acquired and aggregated from multiple underwater detection devices, thereby increasing the amount of feedback information available for machine learning. Furthermore, since only feedback information with the same attributes, including fishing methods, is used for machine learning among the aggregated feedback information, it is possible to suppress the use of echo data with different characteristics in the machine learning of individual models. As a result, the accuracy of fish species discrimination results by individual models can be improved. Furthermore, even if the predicted probability of a specific fish species is lower than that of other species, the system will still output a result identifying that specific fish species, thus increasing the frequency of outputting such results. As a result, users are less likely to miss out on catching their desired specific fish species, and can increase the catch volume of that species.

[0012] In the server according to this embodiment, the attribute may further include the region to which the user using the underwater detection device belongs.

[0013] Even with the same fishing method, fish characteristics can differ depending on the region, and consequently, the characteristics of the echo data may also differ. Therefore, by further restricting the feedback information used to train individual models by region, the accuracy of fish species classification by individual models can be further improved.

[0014] The server according to this embodiment may be configured to generate a standard model for fish species discrimination using machine learning with standard data prior to the generation of the individual model, and to generate the individual model by obtaining information from the underwater detection device as feedback information indicating the user's modifications to the fish species discrimination result by the standard model.

[0015] This configuration allows the user to receive fish species identification results from a standard model, while simultaneously receiving user modifications to these results as feedback information from the underwater detection device. Therefore, individual models can be generated appropriately and efficiently while maintaining user convenience.

[0016] The server according to this embodiment may be configured to update the individual model by acquiring information indicating a user's modification to the fish species identification result by the individual model as feedback information.

[0017] According to this configuration, the individual model can be gradually updated to adapt to the user's attributes. Therefore, the convenience of the user can be enhanced.

[0022] Further, the change of the output condition may include changing the lower limit value of the prediction probability for outputting the discrimination result for a specific fish species specified by the user.

[0023] Thereby, the user can, for example, lower the lower limit value for a specific fish species to output the discrimination result of the specific fish species even if the prediction probability of the specific fish species is low, and can more surely catch the fish the user wants to catch. Alternatively, the user can increase the lower limit value for a specific fish species to reduce the frequency at which the discrimination result of the specific fish species is output, and can more efficiently confirm the fish the user wants to catch.

[0026] The second aspect of the present invention relates to a fish species discrimination system. The fish species discrimination system according to this aspect includes the server according to the first aspect and the underwater detection device.

[0027] According to the fish species discrimination system according to this aspect, the same effects as those of the first aspect are achieved.

[0028] In the fish species discrimination system according to this aspect, the underwater detection device may be configured to include a display unit, an input unit, and a control unit that displays the discrimination result of the fish species by the individual model on the display unit, accepts correction of the discrimination result via the input unit, and transmits the correction to the server as feedback information.

[0029] According to this configuration, the correction of the discrimination result of the fish species by the individual model can be provided to the server side at any time. Therefore, the individual model can be gradually updated to adapt to the user's attributes, and the convenience of the user can be enhanced.

[0030] In this configuration, the control unit receives, via the input unit, an input of customization information including a change in the output conditions of the individual model, transmits the input customization information to the server, and the server may be configured to change the output conditions of the individual model for the underwater detection device based on the received customization information.

[0031] According to this configuration, the individual model can be customized so that it is easy for the user to use.

[0032] A third aspect of the present invention relates to a fish species discrimination method. The fish species discrimination method according to this aspect acquires feedback information associating echo data with fish species from a plurality of underwater detection devices, associates the acquired feedback information with an attribute including at least the fishing method used by the underwater detection device, and stores the information, and generates an individual model for fish species discrimination for each attribute by machine learning using a plurality of the feedback information having the same attribute. The system receives customized information, including changes to the output conditions of the individual model, input by the user of each of the underwater detection devices, and modifies the output conditions of the individual model for the underwater detection device based on the received customized information. The modification of the output conditions includes, even if the predicted probability of the individual model for a specific fish species specified by the user is lower than that for other fish species, if the predicted probability of the specific fish species is above a predetermined threshold, the system prioritizes outputting the specific fish species as the identification result.

[0033] According to the fish species discrimination system according to this aspect, the same effect as that of the first aspect is achieved.

[0034] A fourth aspect of the present invention relates to a program for causing a computer to execute a predetermined function. The program according to this aspect includes a function of acquiring feedback information associating echo data with fish species from a plurality of underwater detection devices, a function of associating the acquired feedback information with an attribute including at least the fishing method used by the underwater detection device and storing the information in a storage unit, and a function of generating an individual model for fish species discrimination for each attribute by machine learning using a plurality of the feedback information having the same attribute. a function to receive customized information, including changes to the output conditions of the individual model, entered by the user of each of the underwater detection devices, and a function to change the output conditions of the individual model for the underwater detection device based on the received customized information. and includes. The modification of the output conditions includes, even if the prediction probability of the individual model for a specific fish species specified by the user is lower than that of other fish species, if the prediction probability of the specific fish species is above a predetermined threshold, the specific fish species will be given priority in outputting the discrimination result.

[0035] According to the program according to this aspect, the same effect as that of the first aspect is achieved.

Effects of the Invention

[0036] As described above, the present invention provides a server, a fish species identification system, a fish species identification method, and a program that can improve the accuracy of fish species identification results using machine learning models.

[0037] The effects and significance of the present invention will become even clearer from the description of the embodiments shown below. However, the embodiments shown below are merely examples of how to implement the present invention, and the present invention is not limited in any way to those described in the embodiments below. [Brief explanation of the drawing]

[0038] [Figure 1] Figure 1 is a diagram showing the configuration of a fish species identification system according to an embodiment. [Figure 2] Figure 2 is a block diagram showing the configuration of a fish species identification system according to an embodiment. [Figure 3] Figure 3 is a diagram showing the management status of various types of information in the storage unit of a server according to this embodiment. [Figure 4] Figure 4(a) is a diagram showing the configuration of user management information according to the embodiment. Figures 4(b) to 4(d) are diagrams showing the configuration of individual data according to the embodiment, respectively. [Figure 5] Figure 5 is a schematic diagram illustrating a fish species discrimination process using a neural network according to an embodiment. [Figure 6] Figure 6(a) is a flowchart showing the application process of a machine learning model according to the embodiment. Figure 6(b) is a flowchart showing the fish species discrimination process according to the embodiment. [Figure 7] Figure 7 is a schematic diagram showing an example of displaying an echo image including the fish species identification result according to the embodiment. [Figure 8] Figure 8(a) is a flowchart showing the feedback information transmission process performed by the control unit of the underwater detection device according to the embodiment. Figure 8(b) is a flowchart showing the feedback information reception process performed by the control unit of the server according to the embodiment. [Figure 9]Figure 9 is a schematic diagram showing a screen for receiving corrections of fish species from a user, according to an embodiment. [Figure 10] Figure 10(a) is a flowchart showing the transmission process of customized information performed by the control unit of the underwater detection device according to the embodiment. Figure 10(b) is a flowchart showing the reception process of customized information performed by the control unit of the server according to the embodiment. [Figure 11] Figure 11 is a schematic diagram showing an example of how the fish species identification results are displayed when the output conditions of an individual model are changed based on customized information, according to the embodiment. [Modes for carrying out the invention]

[0039] Figure 1 shows the configuration of the fish species identification system 1.

[0040] The fish species identification system 1 comprises an underwater detection device 10 and a server 20. The underwater detection device 10 is a fish finder installed on the ship 2. The underwater detection device 10 can communicate with the server 20 via an external communication network 30 (for example, the internet) and a base station 40. The underwater detection device 10 and the server 20 each hold address information for communicating with each other. This address information is set in the underwater detection device 10 and the server 20 during initial setup.

[0041] The underwater detection device 10 comprises a transducer 11 and a control unit 12. The transducer 11 is installed on the bottom of the ship 2, and the control unit 12 is installed in the ship's wheelhouse or elsewhere. The transducer 11 and the control unit 12 are connected by a signal cable (not shown). The transducer 11 is equipped with an ultrasonic transducer for transmitting and receiving waves. In response to control from the control unit 12, the transducer 11 transmits ultrasonic waves 3 (transmitted waves) toward the seabed 4 using the ultrasonic transducer and receives the reflected waves. The transducer 11 transmits a received signal based on the received reflected waves to the control unit 12.

[0042] The control unit 12 processes the received signal and generates echo data showing the echo intensity at each depth. The control unit 12 arranges the echo intensity at each depth based on the echo data in chronological order to generate an echo screen for one display. The control unit 12 displays the generated echo screen on the display unit. The control unit 12 updates the echo screen for each ultrasonic wave transmission and reception. By referring to the echo screen, the user can understand the presence and location of the school of fish 5.

[0043] Furthermore, the control unit 12 transmits the generated echo data to the server 20 as needed. The server 20 stores the received echo data and generates echo images similar to those of the control unit 12. The server 20 uses a machine learning model applied to the receiving underwater detection device 10 to identify the species of fish in the school of fish included in the echo image. The server 20 transmits the identification result of the fish species, along with the range (depth, time) of the school of fish to be identified, to the receiving underwater detection device 10 of the echo data.

[0044] The underwater detection device 10 overlays the fish species identification result onto the corresponding area of ​​the echo image based on the received identification result and the range (depth, time) of the fish school. This allows the user to confirm the fish species of each fish school on the echo image, enabling them to smoothly proceed with catching the desired fish.

[0045] If the user's fish species identification result provided by the server 20 differs from the actual fish species caught, the user sends feedback information to the server 20 to correct the fish species. For example, after finishing fishing for the day, the user performs an operation to acquire echo data for a predetermined time period of that day from the server 20 via the input unit of the underwater detection device 10. As a result, the server 20 transmits the echo data for the specified day and time period, along with the fish species identification result (including the range of the fish school to be identified), to the underwater detection device 10. Based on the received echo data and identification result, the underwater detection device 10 displays an echo image including the identification result on its display unit.

[0046] The user performs an operation via the input unit to correct the fish species identification result displayed on the echo image to the species of the fish they have caught. As a result, the underwater detection device 10 transmits feedback information to the server 20, including the user's correction to the identification result and the range (depth, time) of the fish school corresponding to that identification result. The server 20 uses the received feedback information as training data to perform machine learning on the machine learning model applied to the underwater detection device 10. This optimizes the machine learning model so that it reflects the user's fishing results.

[0047] Furthermore, as described later, this machine learning is performed using not only feedback information from the underwater detection device 10, but also feedback information transmitted to the server 20 from other underwater detection devices with the same attributes as the underwater detection device 10 (including at least the fishing method). This improves the fish species discrimination accuracy of the machine learning model applied to the underwater detection device 10.

[0048] Furthermore, the user can, as appropriate, input customization information, including changes to the output conditions of the machine learning model of the underwater detection device 10, via the input section of the underwater detection device 10. The underwater detection device 10 transmits the input customization information to the server 20. Based on the received customization information, the server 20 changes the output conditions of the machine learning model for the underwater detection device 10. This allows the user to display fish species identification results in the echo image according to their preferences.

[0049] Although only one underwater detection device 10 is shown in Figure 1, in reality, numerous underwater detection devices 10 can communicate with the server 20 via the external communication network 30 and the nearest base station. Furthermore, the underwater detection devices 10 that communicate with the server 20 include not only those installed on the ship 2 as shown in Figure 1, but also several types of underwater detection devices used in different fishing methods, such as those installed on fixed nets.

[0050] Figure 2 is a block diagram showing the configuration of the fish species identification system 1.

[0051] The underwater detection device 10 comprises a control unit 101, a display unit 102, an input unit 103, a transmitter / receiver unit 104, a signal processing unit 105, a communication unit 106, and a position detection unit 107.

[0052] The control unit 101 consists of a microcomputer and memory, etc. The control unit 101 controls each part of the underwater detection device 10 according to a program stored in memory. This program includes functions for receiving and displaying fish species identification results, as well as functions for receiving and transmitting feedback information and customization information, as described later.

[0053] The display unit 102 includes a monitor and displays a predetermined image under control from the control unit 101. The input unit 103 includes a trackball for moving a cursor on the image displayed on the display unit 102, operation keys, etc., and outputs signals to the control unit 101 in response to user operations. The display unit 102 and the input unit 103 may be integrated using a liquid crystal touch panel or the like.

[0054] The transmitter / receiver unit 104 includes a transducer 11 as shown in Figure 1, a transmitting circuit for supplying a transmission signal to the transducer 11, and a receiving circuit for processing the received signal output from the transducer 11 and outputting it to the signal processing unit 105. The transmitting circuit and the receiving circuit are included in the control unit 12 shown in Figure 1.

[0055] The transmitter / receiver unit 104 transmits a transmission wave (ultrasound) according to the control unit 101. In this sequence, two types of transmission waves with different frequencies are transmitted sequentially in a time-division multiplexing manner. The transmitter / receiver unit 104 receives the reflected waves of each transmitted frequency and outputs a received signal. The receiving circuit extracts the received signal for each transmission frequency and outputs it to the signal processing unit 105.

[0056] The reason for transmitting and receiving signals at two different frequencies is to enable more accurate fish species identification, as will be explained later. For example, the presence or absence of a swim bladder causes differences in the echo intensity at each frequency. Therefore, by referring to the differences in echo intensity from a school of fish, the species of fish in that school can be identified with high accuracy.

[0057] The signal processing unit 105 generates echo data indicating the intensity of reflected waves according to depth from the received signals of each frequency input from the transmitter / receiver unit 104, and outputs the two types of generated echo data to the control unit 101. The elapsed time from the timing of transmission of each frequency wave corresponds to the depth. Here, the intensity of the reflected wave attenuates as the depth increases. Therefore, in order to handle the echo data quantitatively regardless of the difference in depth, the signal processing unit 105 corrects the intensity of the reflected wave that attenuates according to the elapsed time, and outputs the echo data with the corrected intensity to the control unit 101.

[0058] The control unit 101 generates an echo image based on the received echo data and displays it on the display unit 102. The control unit 101 generates echo data using echo data corresponding to either one of the frequencies. The user may switch as appropriate which frequency of echo data is used to generate the echo image. The control unit 101 generates a single row of images in the depth direction from the echo data, representing the echo intensity at each depth in gradation using a color scale. The control unit 101 integrates the images in each row from the present time to a predetermined time ago in the time direction to generate a single screen of echo image.

[0059] In the following explanation, when we refer to echo data (including echo data included in feedback information), we mean echo data of two different frequencies unless otherwise specified.

[0060] The communication unit 106 is a communication module capable of wireless communication with the base station 40. The position detection unit 107 is equipped with GPS and detects the position of the underwater detection device 10. The position detection unit 107 outputs the detected position information to the control unit 101.

[0061] As explained with reference to Figure 1, the control unit 101 periodically transmits echo data, feedback information, and customization information to the server 20 via the communication unit 106, and also receives the fish species identification result from the server 20 via the communication unit 106. The control unit 101 may further transmit location information detected by the location detection unit 107 to the server 20.

[0062] As shown in Figure 2, in addition to the underwater detection device 10, numerous other underwater detection devices 10a, 10b, ... can communicate with the server 20 via the external communication network 30 and the nearest base stations 40a, 40b, .... As described above, the underwater detection devices 10 that communicate with the server 20 include not only those installed on the ship 2 as shown in Figure 1, but also several types of underwater detection devices used for different fishing methods, such as underwater detection devices installed on fixed nets. The basic configuration of the other underwater detection devices is the same as that of the underwater detection device 10 in Figure 2.

[0063] However, an underwater detection device installed in a fixed net may consist of an offshore unit installed in the fixed net and a terminal that can communicate with this offshore unit via an external communication network, allowing the user to remotely monitor the condition of fish within the net. Echo data acquired by the offshore unit is transmitted to the terminal via the external communication network. This displays the echo image on the terminal. The terminal may be a personal computer, a mobile phone, a tablet, or any other portable device owned by the user.

[0064] In this case, the terminal may transmit echo data to the server 20, or the offshore unit may transmit echo data to the server 20 in parallel with transmitting echo data to the terminal. Feedback information and customization information may be input via the terminal and transmitted from the terminal to the server 20. The fish species identification result may be transmitted directly from the server 20 to the terminal without going through the offshore unit. Furthermore, the server 20 may transmit echo data to the terminal. That is, the server 20 may receive echo data from the offshore unit and transmit the received echo data to the terminal.

[0065] The server 20 comprises a control unit 201, a storage unit 202, and a communication unit 203. The control unit 201 is composed of a CPU, etc. The storage unit 202 is composed of ROM, RAM, hard disk, etc. The storage unit 202 stores a program for fish species identification. The control unit 201 controls each part according to the program stored in the storage unit 202. The communication unit 203 communicates with the underwater detection device 10 via the external communication network 30 and base station 40 under control from the control unit 201.

[0066] The control unit 201 generates a machine learning model to be applied to each underwater detection device 10 using the above program. The control unit 201 also stores the echo data, feedback information, and customization information received from each underwater detection device 10 in the storage unit 202, associating them with each underwater detection device 10. The control unit 201 updates the machine learning model applied to each underwater detection device 10 using the feedback information and customization information received from each underwater detection device 10.

[0067] Furthermore, the echo data transmitted from each underwater detection device 10 to the server 20 may be thinned down to a predetermined granularity before transmission in order to reduce communication traffic and the capacity load on the server 20. In this case, the server 20 uses the echo data corrected for thinning by interpolation to perform fish species discrimination and machine learning. Alternatively, fish species discrimination and machine learning may be performed using the thinned echo data. However, in order to perform fish species discrimination and machine learning with higher accuracy, it is preferable to use echo data corrected for thinning by interpolation for fish species discrimination and machine learning.

[0068] Furthermore, in order to quantitatively process the echo data received from each underwater detection device 10, the server 20 may perform corrections on the echo data received from each underwater detection device 10 based on underwater acoustic theory, taking into account the characteristics of the underwater detection device 10 and the transducer 11 (e.g., sensitivity, amplification factor, etc.) to perform fish species discrimination and machine learning. This makes it possible to perform fish species discrimination by machine learning models and machine learning on machine learning models with greater accuracy.

[0069] Figure 3 shows the management status of various types of information in the storage unit 202 of the server 20.

[0070] The memory unit 202 stores standard data 301, standard model 302, user management information 303, individual data 311, 321, and individual models 312, 322.

[0071] Standard Data 301 is standard training data for machine learning. Standard Data 301 combines echo data of fish school ranges (depth, time) with data on the fish species within those schools. Standard Data 301 is continuously generated by specialists and registered by administrators. As a result, the amount of data in the standard dataset gradually increases.

[0072] Standard model 302 is a machine learning model generated by machine learning using standard data 301. When standard data 301 is updated, machine learning is performed on standard model 302, and standard model 302 is updated.

[0073] In this embodiment, machine learning using a neural network is applied. For example, a neural network using deep learning, which combines neurons in multiple stages, is applied. However, the machine learning applied is not limited to this, and other machine learning methods such as support vector machines and decision trees may also be applied.

[0074] User management information 303 is information for managing users of the underwater detection device 10.

[0075] Figure 4(a) shows the configuration of user management information 303.

[0076] User management information 303 is composed of user ID, username, region, fishing method, equipment type, and application model, with each corresponding to the other.

[0077] The User ID is information used to identify the user (underwater detection device 10). For example, the product code of the underwater detection device 10 may be used as the User ID, or a randomly assigned code may be used as the User ID. The User Name is the user's name (full name, etc.). The Region is the region to which the user belongs. The Region may be, for example, a prefecture. The Region may also be a regional name such as Kinki, or a city or town.

[0078] The fishing method is information used to identify the fishing method in which the underwater detection device 10 is used. For example, the fishing method is identified by the type of fishing method, such as purse seine fishing or fixed net fishing. The device type is information indicating the type of device. The device type may be, for example, a model code. If the underwater detection device 10 is specialized for any particular fishing method, the type of the underwater detection device 10 may be used to identify the fishing method.

[0079] The applied model indicates whether the machine learning model used for fish species discrimination is a standard model or an individual model. The standard model is, as described above, a standard machine learning model generated by machine learning using standard data. The individual model is a machine learning model generated for each user (each underwater detection device 10) by machine learning using individual data (feedback information), as described below.

[0080] Of the user management information shown in Figure 4(a), information other than the applicable model is registered by the service technician to the server 20 when setting up the underwater detection device 10 with the fish species identification system 1. At this time, the service technician sets the user ID included in the registered user management information on the underwater detection device 10. When the underwater detection device 10 communicates with the server 20, it includes its own user ID in the communication header.

[0081] Returning to Figure 3, individual data 311 and 321 are data acquired from each user's underwater detection device 10. Individual models 312 and 322 are machine learning models applied to each user's underwater detection device 10. In Figure 3, individual data 311 and individual model 312 are for user U1, and individual data 321 and individual model 322 are for user U2. Similarly, individual data and individual models are managed for each user other than U1 and U2.

[0082] Figures 4(b) to 4(d) show the structure of the individual data.

[0083] As shown in Figures 4(b) to 4(d), various types of individual data are managed in association with the user ID. Figure 4(b) shows individual data related to echo data. The underwater detection device 10 sequentially transmits the echo data obtained by one sequence of transmit and receive waves to the server 20 along with the date and time of acquisition. The storage unit 202 of the server 20 stores the start date and time and end date and time of acquisition of the echo data, as well as a group of echo data acquired during that period and the date and time of acquisition of those data, in association with the user ID.

[0084] Furthermore, the fish species classification results obtained by a machine learning model from a group of echo data from the start date to the end date are further associated with the echo data of each group. If multiple fish species classification results are obtained, all of these results are associated with a group of echo data. Each fish species classification result consists of the range of the fish group (depth, time) and the classification result (fish species).

[0085] Figure 4(c) shows individual data related to feedback information. Here, feedback information acquired from the underwater detection device 10 corresponding to the user ID is stored in chronological order. Figure 4(d) shows individual data related to customization information. Here, customization information acquired from the underwater detection device 10 corresponding to the user ID is stored in chronological order.

[0086] Figure 5 schematically illustrates the fish species discrimination process using a neural network.

[0087] The control unit 201 of server 20 extracts the range (depth, time) of a school of fish from the echo data for one screen to be processed. The range of a school of fish is extracted as a range on the echo image where the echo intensity is above a predetermined threshold and there is a connection between the echo intensities. The method for extracting the range of a school of fish may be incorporated by reference from the description in International Publication No. 2019 / 003759, which was previously filed by the applicant.

[0088] The control unit 201 applies the echo data of the extracted fish school range to the input 401a of the machine learning algorithm (neural network) 401 shown in Figure 5. The output 401b of the machine learning algorithm 401 is assigned items for fish species such as sardines, horse mackerel, and mackerel. When the echo data of the fish school range is applied to the input 401a of the machine learning algorithm 401, the probability (prediction probability) that the fish species in the fish school is the species of the respective item is output from each item in the output 401b of the machine learning algorithm 401. In the example in Figure 5, a prediction probability of 80% is output from the sardine item and a prediction probability of 8% is output from the horse mackerel item.

[0089] The predicted probability for each item is compared against output condition 402. Output condition 402 applies, for example, a condition that outputs the fish species of the item whose predicted probability is above a predetermined lower limit and is the highest ranked (highest) as the classification result 403. The lower limit is set to prevent fish species with low accuracy from being output as classification results. In the example in Figure 5, sardines, which have a predicted probability of 80%, are output as the fish species classification result 403.

[0090] Machine learning for the machine learning algorithm 401 is performed by sequentially applying a series of training data to the input 401a and output 401b of the machine learning algorithm 401. Specifically, the input 401a of the machine learning algorithm 401 is the echo data of a school of fish contained in one training data set, and the output 401b of the machine learning algorithm 401 is set to 100% for the item corresponding to the fish species contained in this training data, and to 0% for the other items, and machine learning is performed.

[0091] The standard model in Figure 3 is generated by sequentially setting the standard data (echo data of the fish school range, fish species) as input 401a and output 401b of the machine learning algorithm 401 and performing machine learning. The individual models in Figure 3 are generated by sequentially setting the feedback information contained in the individual data (echo data of the fish school range, fish species) as input 401a and output 401b of the machine learning algorithm 401 and performing machine learning. If the amount of feedback information that can be used for machine learning is small, the standard data may be used further in training the individual models.

[0092] Furthermore, in addition to the echo data of the fish school, the input 401a of the machine learning algorithm 401 may also contain other information that can be used for fish species identification, such as the location where the echo data was obtained, and the water temperature, salinity, and current velocity at that location. In this case, the underwater detection device 10 is configured to acquire this other information and transmits it to the server 20 along with the echo data. The control unit 201 of the server 20 stores this received other information in the storage unit 202 as individual data as shown in Figure 3. In this case, the standard data may also include this other information.

[0093] Figure 6(a) is a flowchart showing the process of applying a machine learning model.

[0094] When a new user is registered in the user management information 303 (see Figure 3), the control unit 201 of the server 20 sets the standard model as the applicable model for that user (user ID) (S101). In this case, the applicable model for that user (user ID) in Figure 4(a) is set to the standard model. Thereafter, until feedback information is received from the user's underwater detection device 10 (S102:NO), the applicable model for that user (user ID) remains the standard model (S101).

[0095] When the control unit 201 receives feedback information from the user's underwater detection device 10 (S102:YES), it extracts feedback information obtained from other users with the same attributes as the user from the storage unit 202 (S103). In step S103, the control unit 201 extracts feedback information from the storage unit 202 that is associated with other users (user IDs) who use at least the same fishing method, as feedback information with the same attributes, in the individual data of Figure 4(a).

[0096] The control unit 201 uses the extracted feedback information from other users and the feedback information from the user in question received in step S102 as training data to train the machine learning algorithm shown in Figure 5 and generate an individual model for the user in question (S104). If the amount of feedback information extracted in step S103 is still insufficient, standard data may also be used as training data when training the machine learning algorithm.

[0097] Once an individual model for the user is generated, the control unit 201 sets the generated individual model as the applicable model for that user (user ID) (S105). As a result, the applicable model for the user in Figure 4(a) is changed from the standard model to the individual model. The generated individual model is stored in the storage unit 202, associated with the user, as shown in Figure 3. For example, the parameter values ​​and output conditions 402 of the machine learning algorithm 401 in Figure 5 may be stored as the individual model.

[0098] Furthermore, the identity of attributes in step S103 is not necessarily limited to the same fishing method; other elements may be included, insofar as feedback information (echo data) with different characteristics can be appropriately excluded. For example, even if the fishing method is the same, if the user belongs to a different region (fishing ground), the characteristics of the fish may differ, and the characteristics of the echo data may also differ. Therefore, other feedback information used in the machine learning of individual models may be further restricted by the region shown in Figure 4(a). This allows for the generation of even more accurate individual models.

[0099] Furthermore, other feedback information used in the machine learning of individual models may be further restricted by the device type shown in Figure 4(a). This allows individual models to be generated using feedback information with similar characteristics obtained from underwater detection devices 10 of the same device type. As a result, even more accurate individual models can be generated.

[0100] Once the individual model is set as the applicable model (S105), the control unit 201 returns to step S102. As a result, each time new feedback information is received from the user (underwater detection device 10) (S102:YES), the control unit 201 updates the user's individual model (S103~S105) using the received feedback information and newly received feedback information from other underwater detection devices 10 with the same attributes as training data.

[0101] Furthermore, after the accuracy of the individual model has improved through repeated updates (for example, the frequency of corrections to the fish species determination result based on feedback information has fallen below a predetermined threshold), step S103 may be omitted, and in step S104, the individual model may be updated using only the feedback information received in step S102. This allows the user's individual model to be updated to better suit the user's fishing grounds, etc.

[0102] Furthermore, in the flowchart of Figure 6(a), the generation and updating of the individual model was triggered by the receipt of feedback information from the user's underwater detection device 10. However, the timing of the generation and updating of the individual model is not limited to this. For example, the individual model may be updated at regular intervals using feedback information received from the user's underwater detection device 10 and feedback information received from other users' underwater detection devices 10 that have the same attributes as the user's underwater detection device 10 as training data.

[0103] Furthermore, if, at the time the user is registered with the server, a sufficient amount of feedback information from other users with the same attributes as the user has already been collected by the server 20, then a standard model may not be applied to the user (user ID). Instead, an individual model may be generated using the feedback information from other users (user IDs) with the same attributes as training data, and the generated individual model may be applied to the user (user ID).

[0104] Figure 6(b) is a flowchart showing the fish species identification process.

[0105] When the control unit 201 of the server 20 begins receiving echo data from the underwater detection device 10 (S201:YES), it stores the received echo data in the storage unit 202 as individual data as shown in Figure 3 (S202). The control unit 201 also sequentially constructs an echo image from the received echo data and identifies the range (depth, time) of the fish school on the echo image. The control unit 201 then applies the echo data of the identified fish school range to the application model (standard model / individual model) of the underwater detection device 10 (user ID) to determine the fish species of the fish school (S203). The control unit 201 transmits the fish species determination result, along with the range (depth, time) of the fish school from which the determination result was obtained, to the underwater detection device 10, and further stores this information in the storage unit 202 (S204).

[0106] Subsequently, the control unit 201 repeatedly executes the processes in steps S202 to S204 until it finishes receiving echo data from the underwater detection device 10 (S205:NO). As a result, each time a new range (depth, time) of a school of fish is identified from the echo image, the species of fish in that school is determined by the applicable model. The newly obtained fish school determination result and the range (depth, time) of that school are transmitted to the underwater detection device 10 as needed and stored in the storage unit 202.

[0107] Thus, when the reception of echo data from the underwater detection device 10 is completed (S205:YES), the control unit 201 terminates the process shown in Figure 6(b). As a result, one row of individual data shown in Figure 4(b) is stored in the storage unit 202. As described above, the echo data column in Figure 4(b) holds all the echo data received from the underwater detection device 10 during the process shown in Figure 6(b). In addition, the fish species identification result column in Figure 4(b) holds all the fish species identification results obtained through the process shown in Figure 6(b), along with the range (depth, time) of the fish school.

[0108] Figure 7 schematically shows an example of the display of an echo image P1 including the fish species identification result. For convenience, in Figure 7, depth lines are added only to the areas with high echo intensity.

[0109] When the control unit 201 of the underwater detection device 10 receives the discrimination result and the range (depth, time) of the fish school from the server 20, it displays a frame-shaped marker M0 indicating the range of the fish school in an area on the echo image P1 corresponding to the depth width and time width corresponding to the received range of the fish school. Furthermore, the control unit 201 displays a label L0 indicating the discrimination result of the received fish species around this marker M0. In the example in Figure 7, based on the discrimination result and the range (depth, time) of the fish school received from the server 20, markers M0 are displayed for fish schools F1 to F8, and furthermore, labels L0 indicating the discrimination result of the fish species are displayed around these markers M0. The current date and time are displayed near the upper left corner of the echo image P1.

[0110] In the example shown in Figure 7, the applied model did not output a fish species classification result for fish school F9, and therefore no markers or labels are displayed for fish school F9. This can occur, for example, if the fish species classification result for fish school F9 by the applied model does not meet the output conditions in Figure 5. For example, if the output conditions are to output the fish species with the highest (first-rank) predicted probability that is above a predetermined lower limit, then if the first-rank predicted probability among the predicted probabilities generated for each fish species output item of the machine learning algorithm is below this lower limit, no classification result will be output for this fish school. In such a case, the classification result for this fish school will not be sent from the server 20 to the underwater detection device 10, and as shown for fish school F9 in Figure 7, the fish species classification result will not be displayed.

[0111] Figure 8(a) is a flowchart showing the feedback information transmission process performed by the control unit 101 of the underwater detection device 10. Figure 8(b) is a flowchart showing the feedback information reception process performed by the control unit 201 of the server 20.

[0112] For example, if the user detects a discrepancy between the species identification result of a predetermined school of fish displayed in the echo image P1 and the species of that school of fish that they actually caught, or if they actually catch a school of fish for which no species identification result is displayed in the echo image P1 and determine the species of that school, the user performs an operation (feedback operation) to the underwater detection device 10 via the input unit 103 in Figure 2 to send feedback information to the server 20 to correct the species of these schools of fish. In this case, the user inputs the date and time range of the echo image containing the school of fish to be corrected via the input unit 103, and then performs an operation to obtain the species identification result and echo data (hereinafter referred to as "history information") for that range from the server 20.

[0113] Referring to Figure 8(a), when the control unit 101 of the underwater detection device 10 receives feedback input from the user via the input unit 103 (S301:YES), it sends a request to the server 20 to transmit history information including the date and time range entered by the user, and obtains the history information from the server 20 (S302). Referring to Figure 8(b), when the control unit 201 of the server 20 receives the request to transmit history information transmitted in step S302 of Figure 8(a) (S401:YES), it extracts the history information (echo data and fish species identification result) for the date and time range included in the transmission request from the individual data of the underwater detection device 10 that sent the transmission request, and transmits the extracted history information to the underwater detection device 10 that sent the request (S402).

[0114] Referring to Figure 8(a), when the control unit 101 of the underwater detection device 10 receives history information from the server 20 (S302), it displays an echo screen based on the received history information on the display unit 102 and accepts corrections of the fish species from the user (S303).

[0115] Figure 9 schematically shows the screen used to receive a correction request for the fish species from the user in step S303 of Figure 8(a).

[0116] The control unit 101 of the underwater detection device 10 displays the echo image and discrimination result for the first time period within the time range specified by the user on the display unit 102 when a request for transmission of history information is received. The user operates the scroll bar B0 via the input unit 103 to transition the echo images in the time direction and display a screen containing the echo image and discrimination result for the desired time period. This displays the time period screen shown in Figure 9.

[0117] On this screen, the user specifies the marker M0 of the fish school they wish to modify via the input unit 103. In the screen shown in Figure 9, the user has specified the marker M0 of fish school F5, which has been identified as mackerel. As a result, the marker M0 of fish school F5 is highlighted, and the candidate fish species C0 are displayed around this marker M0 along with a scroll bar. The user operates the scroll bar of the candidate C0 to display the desired fish species, and then selects the fish species they wish to change. In the example in Figure 9, sea bream is selected as the fish species of fish school F5.

[0118] Furthermore, on this screen, if the user wants to input a fish species for a school of fish for which no identification result has been assigned, they specify the range of the school of fish via the input unit 103. As shown in Figure 7, no fish species identification result has been obtained for school of fish F9. When the user wants to input a fish species for school of fish F9, they specify the range of school of fish F9 via the input unit 103. As a result, as shown in Figure 9, a new marker M1 is displayed in the specified range of school of fish F9, and around this marker M1, the candidate fish species C0 are displayed along with a scroll bar. The user operates the scroll bar of the candidate fish species C0 to display the desired fish species, and then selects the fish species they want to input. In the example in Figure 9, Spanish mackerel is selected as the fish species for school of fish F9.

[0119] After performing operations to change or set the fish species, the user inputs an operation to confirm these operations via the input unit 103.

[0120] Referring to Figure 8(a), when the user inputs a confirmation operation (S304:YES), the control unit 101 sends feedback information to the server 20 in step S303, including the range of the fish school specified by the user and the fish species entered by the user for that fish school (S305). With this, the control unit 101 terminates the process shown in Figure 8(a).

[0121] Referring to Figure 8(b), when the control unit 201 of the server 20 receives feedback information from the control unit 101 of the underwater detection device 10 (S403: YES), it stores the received feedback information in the storage unit 202 as individual data for the underwater detection device 10 (S404). As a result, one line of feedback information as shown in Figure 4(c) is stored in the storage unit 202. Thus, the control unit 201 completes the process shown in Figure 8(b).

[0122] Furthermore, if feedback information is received in step S403 of Figure 8(b), the determination in step S102 of Figure 6(a) becomes YES. As a result, the processing from step S103 onward in Figure 6(a) is executed as described above.

[0123] Figure 10(a) is a flowchart showing the process of transmitting customized information, which is performed by the control unit 101 of the underwater detection device 10. Figure 10(b) is a flowchart showing the process of receiving customized information, which is performed by the control unit 201 of the server 20.

[0124] Users can input and send customized information at any time. The customized information is information for adjusting the fish species identification results of the underwater detection device 10 to the user's preference, and includes changes to the output conditions 402 of the individual model in Figure 5.

[0125] Referring to Figure 10(a), when the control unit 101 of the underwater detection device 10 receives a customization operation input from the user via the input unit 103 (S501:YES), it displays a predetermined input screen on the display unit 102 and accepts customization information input from the user (S502). After the user has entered the customization information and a confirmation operation is entered (S503:YES), the control unit 101 transmits the entered customization information to the server 20. With this, the control unit 101 terminates the process shown in Figure 10(a).

[0126] As described above, the customization information includes changes to the output conditions of individual models. Here, changes to the output conditions may include prioritizing the output of a specific fish species as a classification result if the prediction probability of that specific fish species is above a predetermined threshold (a value greater than zero), even if the prediction probability of that specific fish species is lower than that of other fish species. Alternatively, changes to the output conditions may include changing the lower limit of the prediction probability required to output a classification result for a specific fish species. Furthermore, changes to the output conditions may include not outputting a classification result for a specific fish species specified by the user.

[0127] Referring to Figure 10(b), when the control unit 201 of the server 20 receives customization information from the control unit 101 of the underwater detection device 10 (S601: YES), it stores the received customization information in the storage unit 202 as individual data for the underwater detection device 10 (S602). As a result, one line of feedback information in Figure 4(d) is stored in the storage unit 202. Furthermore, the control unit 201 changes the output conditions of the individual model of the underwater detection device 10 based on the received customization information (S603). With this, the control unit 201 terminates the process shown in Figure 10(b).

[0128] Figure 11 schematically shows an example of how the fish species identification results are displayed when the output conditions of an individual model are changed based on customized information.

[0129] Here, the output conditions for the discrimination result have been modified so that even if the prediction probability of a specific fish species designated by the user is lower than that of other fish species, if the prediction probability of the specific fish species is above a predetermined threshold (a value greater than zero), this specific fish species will be prioritized and output as the discrimination result. In this case, the specific fish species to be prioritized is specified as sea bream. Therefore, in the screen shown in Figure 7, the fish species discrimination result for fish school F5 was mackerel, while in the screen shown in Figure 11, the fish species discrimination result for fish school F5 is sea bream.

[0130] In other words, in the fish species classification of fish school F5 using the machine learning algorithm 401 in Figure 5, the predicted probability of mackerel was the highest, but since the predicted probability of sea bream was above the threshold, the classification result 403 output was sea bream according to the modified output condition 402. Here, the threshold may be specified by the user as customization information. This allows the user to prioritize outputting the classification result of sea bream if the predicted probability of sea bream is above their desired threshold, and they can check the fish school F5 that may contain sea bream on the screen in Figure 11.

[0131] In this case, it may be further indicated for this school of fish F5 that the predicted probability of sea bream is lower (not the first) than any other fish species. For example, as shown in Figure 11, the label L0 may include the symbol "?" along with the indication of sea bream to show that the predicted probability of sea bream for school F5 is not the first. Alternatively, the rank of the predicted probability of sea bream may be indicated by a number instead of the symbol "?". This allows the user to make an appropriate decision on whether or not to fish for school F5.

[0132] Furthermore, in the example display in Figure 11, the output conditions have been modified so that the lower limit of the predicted probability for outputting the discrimination result is lower for the specific fish species specified by the user compared to other fish species. Here, the specific fish species for which the lower limit is lowered is specified as Spanish mackerel. As a result, in the screen of Figure 7, the predicted probability of Spanish mackerel, the first-priority fish for fish school F9, was lower than the standard lower limit, so the fish species discrimination result for fish school F9 was not displayed. However, in the screen of Figure 11, the predicted probability of Spanish mackerel, the first-priority fish for fish school F9, was above the lower limit specified by the user, so the fish species discrimination result for fish school F9 is displayed as Spanish mackerel. This allows the user to increase the frequency with which the discrimination result for Spanish mackerel, which they intend to catch, is displayed in the echo image P1, enabling them to efficiently proceed with Spanish mackerel fishing.

[0133] In this case, the lower limit for outputting a determination result for a specific fish species was set lower than the lower limit of the standard applied to other fish species. However, the lower limit for outputting a determination result for a specific fish species may also be set higher than the standard lower limit. This allows the user to reduce the frequency with which the species determination result for a specific fish species not intended for capture is displayed in the echo image P1.

[0134] Furthermore, in the example display in Figure 11, the output conditions have been changed so that the identification results for a specific fish species specified by the user are not output. Here, the specific fish species for which the identification results are not output is specified as sardines. For this reason, in the screen of Figure 7, marker M0 and the sardine label L0 are displayed for fish schools F1 and F2, whereas in the screen of Figure 11, these displays are omitted for fish schools F1 and F2. This allows the user to suppress the display of identification results for a specific fish species (sardines in this case) that is not intended as a target for capture in the echo image P1, and to smoothly confirm the range of fish species they desire in the echo image P1.

[0135] <Effects of the Embodiment> According to the embodiment, the following effects may be achieved.

[0136] As shown in Figures 1 to 6(a), the server 20 acquires feedback information from each underwater detection device 10 that associates echo data with fish species, stores the acquired feedback information linked to attributes including at least the fishing method in which the underwater detection device 10 is used, and generates individual models for fish species discrimination for each attribute using machine learning with multiple pieces of feedback information with the same attributes.

[0137] With this configuration, feedback information is acquired and aggregated by the server 20 from multiple underwater detection devices 10, thereby increasing the amount of feedback information available for machine learning. Furthermore, since only feedback information with the same attributes, including fishing method (see Figure 4(a)), is used for machine learning among the aggregated feedback information, it is possible to suppress the use of echo data with different characteristics in the machine learning of individual models. As a result, the accuracy of fish species discrimination results by individual models can be improved.

[0138] Here, the attributes may further include the region to which the user using the underwater detection device 10 belongs (see Figure 4(a)). Even with the same fishing method, the characteristics of the fish may differ depending on the region, and therefore the characteristics of the echo data may also differ. Therefore, by further restricting the feedback information used for training individual models to the region, the accuracy of the fish species classification results by individual models can be further improved.

[0139] As shown in Figures 3 and 6(a), the server 20 generates a standard model 302 for fish species discrimination using machine learning with standard data 301 (see Figure 3) prior to generating individual models 312 and 322, applies it to the user (underwater detection device 10), obtains information indicating the user's modifications to the fish species discrimination result by the standard model 302 as feedback information from the underwater detection device 10 (Figure 6(a): step S102), and generates individual models 312 and 322 (steps S103 to S105). This allows the server 20 to provide the user with the fish species discrimination result by the standard model 302 while smoothly obtaining the user's modifications to this result as feedback information from the underwater detection device 10. Thus, individual models 312 and 322 can be generated appropriately and efficiently while maintaining user convenience.

[0140] As shown in Figure 6(a), Server 20 is individual model 312, 322 (see Figure 3) Information indicating user modifications to the fish species identification results is obtained as feedback information (S102), and individual models 312 and 322 are updated. This allows individual models 312 and 322 to be gradually updated to adapt to the user's attributes. Thus, user convenience can be improved.

[0141] As shown in Figure 10(b), the server 20 receives customization information (S601) including changes to the output conditions 402 (see Figure 5) of individual models 312 and 322 (see Figure 3) entered by the user of each underwater detection device 10, and modifies the output conditions 402 of the individual models 312 and 322 for the underwater detection device 10 based on the received customization information. This allows the individual models 312 and 322 to be customized for ease of use by the user.

[0142] In this case, changing the output condition 402 may include prioritizing the output of a specific fish species as a discrimination result, even if the prediction probability of the individual models 312 and 322 for that specific fish species is lower than that of other fish species, as long as the prediction probability of that specific fish species is above a predetermined threshold. As a result, as shown in Figure 11, even if the prediction probability of a specific fish species (in this case, sea bream) is lower than that of other fish species, the discrimination result for that specific fish species (label L0 of fish school F5) is output, and the frequency of outputting the discrimination result for that specific fish species can be increased. Therefore, users are less likely to miss out on catching their desired specific fish species and can increase the catch amount of that specific fish species.

[0143] Alternatively, changing output condition 402 may include changing the lower limit of the predicted probability for outputting a discrimination result. This allows the user to, for example, lower the lower limit for a specific fish species, as shown in Figure 11, to output a discrimination result (label L0 of fish school F9) for that specific fish species (in this case, Spanish mackerel) even if the predicted probability for that species is low, thereby more reliably catching the fish they want to catch. Alternatively, by raising the lower limit for a specific fish species, the user can reduce the frequency with which a discrimination result for that specific fish species is output, allowing them to more efficiently identify the fish they want to catch.

[0144] Alternatively, changing output condition 402 may include not outputting the identification results for a specific fish species specified by the user. As shown in Figure 11, this allows the user to suppress the output of identification results for fish species they are not interested in (in this case, sardines) (in this case, identification results for fish schools F1 and F2), and to smoothly and efficiently identify fish schools of fish species they are interested in.

[0145] As shown in Figure 2, the underwater detection device 10 comprises a display unit 102, an input unit 103, and a control unit 101. Here, the control unit 101 displays the fish species identification result by the individual model on the display unit 102 through the process shown in Figure 8(a), and accepts corrections to the identification result via the input unit 103 (S301-S303), and transmits the correction as feedback information to the server 20 (S305). This allows the user to provide corrections to the fish species identification result by the individual model to the server 20 at any time. Thus, the individual model can be gradually updated to adapt to the user's attributes, improving user convenience.

[0146] Here, through the process shown in Figure 10(a), the control unit 101 further receives customization information via the input unit 103, including a change in the output conditions 402 (see Figure 5) of the individual model (S501, S502), and transmits the input customization information to the server 20 (S504). Through the process shown in Figure 10(b), the server 20 changes the output conditions 402 of the individual model for the underwater detection device 10 based on the received customization information (S601: YES). This allows the individual model to be customized for ease of use by the user.

[0147] <Example of changes> The present invention is not limited to the above embodiments, and various modifications are possible to the embodiments of the present invention other than the above configuration.

[0148] For example, in the above embodiment, two types of ultrasonic waves of different frequencies were transmitted and received in one sequence, but the types of frequencies transmitted and received in one sequence are not limited to two. For example, only ultrasonic waves of one frequency may be transmitted and received in one sequence, or only ultrasonic waves of three or more frequencies may be transmitted and received in one sequence.

[0149] Furthermore, in the above embodiment, the fish species discrimination process using a standard model or an individual model was performed on the server 20 side, but this discrimination process may also be performed on the underwater detection device 10 side. In this case, the server 20 transmits the standard model or an individual model generated for each underwater detection device 10 to each underwater detection device 10, and each underwater detection device 10 performs fish species discrimination using the standard model or individual model received from the server 20. In this case as well, the server 20 aggregates individual data (including feedback information and customization information) from each underwater detection device 10, as in the above embodiment, and updates and customizes the individual model based on the aggregated individual data using the same processing as in the above embodiment. The server 20 transmits the updated and customized individual model to each underwater detection device 10 as needed, and each underwater detection device 10 performs fish species discrimination processing using the updated and customized individual model.

[0150] Furthermore, in the above embodiment, fishing method, region, and equipment type (see Figure 4(a)) were shown as attributes for restricting the feedback information used in machine learning, but other parameters may also be included as attributes.

[0151] For example, the sea area from which echo data was obtained may be included as an attribute for restricting the feedback information used for machine learning. In this case, the server 20 receives the position information detected by the position detection unit 107 of the underwater detection device 10 along with the echo data from the underwater detection device 10, and further restricts the feedback information (echo data) used for machine learning of individual models to feedback information that includes position information in the sea area specified by the user. This allows the individual models to be updated to be adapted to the sea area specified by the user (the user's fishing grounds).

[0152] Furthermore, in the above embodiment, user feedback information was input via the screen shown in Figure 9, but the method of inputting feedback information is not limited to this. Also, the feedback information does not necessarily have to be a correction of the discrimination result by the standard model or individual model; the user may independently specify an arbitrary region (a region of fish school) on the echo image and input the fish species in that region.

[0153] Furthermore, the customization information does not have to be information related to changes in the output condition 402 as described above; it may also be information related to changes in output condition 402 other than those described above. For example, the customization information may simply be information for changing the lower limit of the output condition, regardless of the fish species.

[0154] Furthermore, customizing individual models using customization information may include modifications to individual models other than changes to output condition 402, for example, by specifying feedback information to be used for machine learning of the individual model. In addition, the user may specify, through customization information, that standard data be used along with feedback information of the same attributes for machine learning of their own individual model, and in this case, they may also specify the ratio of feedback information to standard data used for machine learning, or the number of pieces of feedback information and standard data used for machine learning.

[0155] Furthermore, in the above embodiment, the user's individual model was customized based on customization information provided by the user. However, the standard model may also be customized based on customization information provided by the user while the user is using the standard model.

[0156] Furthermore, in the above embodiment, the underwater detection device 10 was a fish finder, but the underwater detection device 10 may be a device other than a fish finder, such as a sonar.

[0157] In addition, embodiments of the present invention can be modified in various ways as appropriate within the scope of the claims. [Explanation of symbols]

[0158] 1. Fish Species Identification System 10 Underwater detection equipment 20 servers 101 Control Unit 102 Display section 103 Input section 301 Standard Data 302 Standard Model 312, 322 individual models 402 Output Conditions 403 Judgment result

Claims

1. A server capable of communicating with multiple underwater detection devices, Feedback information, which associates echo data with fish species, is acquired from each of the aforementioned underwater detection devices. The acquired feedback information is stored in association with attributes including at least the fishing method in which the underwater detection device is used. By machine learning using multiple pieces of feedback information with the same attribute, an individual model for fish species discrimination is generated for each attribute. The system receives customized information, including changes to the output conditions of each individual model, which are input by the user of each of the underwater detection devices. Based on the received customization information, the output conditions for the individual model for the underwater detection device are changed. The modification of the output conditions includes, even if the prediction probability of the individual model for a specific fish species specified by the user is lower than that of other fish species, if the prediction probability of the specific fish species is above a predetermined threshold, the specific fish species will be given priority in the output of the discrimination result. A server characterized by the following features.

2. A server capable of communicating with a plurality of underwater detection devices, Feedback information, which associates echo data with fish species, is acquired from each of the aforementioned underwater detection devices. The acquired feedback information is stored in association with attributes including at least the fishing method in which the underwater detection device is used. By machine learning using multiple pieces of feedback information with the same attribute, an individual model for fish species discrimination is generated for each attribute. The system receives customized information, including changes to the output conditions of each individual model, which are input by the user of each of the underwater detection devices. Based on the received customization information, the output conditions for the individual model for the underwater detection device are changed. The aforementioned change in output conditions includes changing the lower limit of the predicted probability for outputting the discrimination result for a specific fish species designated by the user. A server characterized by the following features.

3. In the server according to claim 1 or 2, The aforementioned attributes further include the region to which the user using the underwater detection device belongs. A server characterized by the following features.

4. In the server according to claim 1 or 2, Using machine learning with standard data, a standard model for fish species discrimination is generated prior to the generation of the individual models. Information indicating user modifications to the fish species identification result by the standard model is obtained from the underwater detection device as feedback information to generate the individual model. A server characterized by the following features.

5. In the server according to claim 1 or 2, Information indicating user modifications to the fish species identification result by the individual model is obtained as feedback information, and the individual model is updated. A server characterized by the following features.

6. A server according to claim 1 or 2, The underwater detection device comprises, A fish species identification system characterized by the following features.

7. In the fish species discrimination system described in claim 6, The underwater detection device is Display unit and Input section, The system includes a control unit that displays the fish species identification result based on the individual model on the display unit, accepts corrections to the identification result via the input unit, and transmits the corrections to the server as feedback information. A fish species identification system characterized by the following features.

8. In the fish species discrimination system described in claim 7, The control unit receives customization information, including changes to the output conditions of the individual model, via the input unit, and transmits the input customization information to the server. The server modifies the output conditions for the individual model of the underwater detection device based on the received customization information. A fish species identification system characterized by the following features.

9. Feedback information, which associates echo data with fish species, is acquired from multiple underwater detection devices. The acquired feedback information is stored in association with attributes including at least the fishing method in which the underwater detection device is used. By machine learning using multiple pieces of feedback information with the same attribute, an individual model for fish species discrimination is generated for each attribute. The system receives customized information, including changes to the output conditions of each individual model, which are input by the user of each of the underwater detection devices. Based on the received customization information, the output conditions for the individual model for the underwater detection device are changed. The modification of the output conditions includes, even if the prediction probability of the individual model for a specific fish species specified by the user is lower than that of other fish species, if the prediction probability of the specific fish species is above a predetermined threshold, the specific fish species will be given priority in the output of the discrimination result. A method for identifying fish species characterized by the following features.

10. Feedback information that associates echo data with fish species is acquired from multiple underwater detection devices, The acquired feedback information is stored in association with attributes including at least the fishing method in which the underwater detection device is used. By machine learning using multiple pieces of feedback information with the same attribute, an individual model for fish species discrimination is generated for each attribute. The system receives customized information, including changes to the output conditions of each individual model, which are input by the user of each of the underwater detection devices. Based on the received customization information, the output conditions for the individual model for the underwater detection device are changed. The aforementioned change in output conditions includes changing the lower limit of the predicted probability for outputting the discrimination result for a specific fish species designated by the user. A method for identifying fish species characterized by the following features.

11. The system has a function to acquire feedback information that associates echo data with fish species from multiple underwater detection devices, A function to store acquired feedback information in a memory unit, linked to at least attributes including the fishing method in which the underwater detection device is used, A function to generate individual models for fish species discrimination for each attribute by machine learning using multiple pieces of feedback information having the same attribute, A function to receive customized information, including changes to the output conditions of the individual model, entered by the user of each of the underwater detection devices, Based on the received customization information, the computer is instructed to perform a function that changes the output conditions for the individual model of the underwater detection device. The program, which modifies the output conditions, includes prioritizing the output of the specified fish species as the discrimination result, even if the prediction probability of the individual model for the specified fish species specified by the user is lower than that of other fish species, as long as the prediction probability of the specified fish species is above a predetermined threshold.

12. A function to acquire feedback information that associates echo data with fish species from multiple underwater detection devices, A function to store acquired feedback information in a memory unit, linked to at least attributes including the fishing method in which the underwater detection device is used, A function to generate individual models for fish species discrimination for each attribute by machine learning using multiple pieces of feedback information having the same attribute, A function to receive customized information, including changes to the output conditions of the individual model, entered by the user of each of the underwater detection devices, Based on the received customization information, the computer is instructed to perform a function that changes the output conditions for the individual model of the underwater detection device. The program includes modifying the output conditions by changing the lower limit of the predicted probability for outputting a discrimination result for a specific fish species designated by the user.

Images