Fish quality estimation system

Abstract

To provide a fish quality estimation system which can estimate quality of a fish.SOLUTION: The fish quality estimation system estimates quality of a fish from a cross-sectional image of a tail of the fish and comprises: an image acquisition unit which acquires the cross-sectional image of the tail of the fish; an extraction unit which extracts a feature quantity characteristic of the cross-sectional image of the tail of the fish from the image acquired by the image acquisition unit; a machine learning unit which generates a first estimation model by performing machine learning by taking quality information of the fish and the feature quantity extracted by the extraction unit as learning data; and an estimation unit which uses the first estimation model generated by the machine learning unit to estimate the quality of the fish from the image acquired by the image acquisition unit.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a system for estimating the quality of fish (particularly tuna). [Background technology]

[0002] Techniques for evaluating the freshness of seafood have been proposed (see, for example, Patent Document 1). Furthermore, the quality of tuna has traditionally been determined in fish wholesale markets and the like by examining the cross section of the tuna's tail. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2010-286262 Summary of the Invention [Problem to be solved by the invention]

[0004] Determining the quality of fish (especially tuna) from the cross section of its tail requires many years of experience, and it has been difficult for anyone other than a highly skilled person to determine the quality of tuna. Therefore, there has been a demand for the development of a system that allows anyone (even non-experts) to easily determine the quality of fish.

[0005] However, even for the same fish, different evaluators (users) may evaluate the quality of the fish differently, which may lead to dissatisfaction with the results of fish quality evaluation by the system.

[0006] The present invention has been made in consideration of the above-mentioned problems, and aims to provide a fish quality estimation system capable of estimating the quality of fish. [Means for solving the problem]

[0007] [1] A fish quality estimation system according to one embodiment includes: A fish quality estimation system that estimates the quality of a fish from a cross-sectional image of a fish's tail, comprising: an image acquisition unit that acquires a cross-sectional image of the fish's tail; an extraction unit that extracts a feature quantity that is characteristic of a cross-sectional image of a fish tail from the image acquired by the image acquisition unit; a machine learning unit that generates a first estimation model by performing machine learning using the fish quality information and the feature values ​​extracted by the extraction unit as learning data; The apparatus further includes an estimation unit that estimates the quality of fish from the image acquired by the image acquisition unit using the first estimation model generated by the machine learning unit. [2] In one aspect, the fish quality estimation system according to the above [1] further comprises: a feedback acquisition unit that acquires a first feedback on the estimation result by the estimation unit; a model generation unit that generates a second estimation model by updating the first estimation model using the first feedback acquired by the feedback acquisition unit; Equipped with.

[0008] [3] In one aspect, the fish quality estimation system according to [2] above comprises: The estimation unit uses the second estimation model generated by the model generation unit to estimate the quality of the fish from the image acquired by the image acquisition unit.

[0009] [4] In one embodiment, the fish quality estimation system is the above-mentioned [2] or [3], the feedback acquisition unit further acquires second feedback on the estimation result estimated by the estimation unit using the second estimation model; The fish quality estimation system according to claim 2 , further comprising a model update unit that updates the second estimation model using the second feedback acquired by the feedback acquisition unit.

[0010] [5] A fish quality estimation system according to one embodiment is any of the above [2] to [4], 4. The fish quality estimation system according to claim 3, wherein the extraction unit extracts color information as a feature characteristic of the cross-sectional image of the fish.

[0011] [6] In one embodiment, the method comprises: A method executed by a fish quality estimation system for estimating fish quality from a cross-sectional image of a fish tail, comprising: acquiring a cross-sectional image of the fish's tail; extracting a feature quantity characteristic of a cross-sectional image of a fish tail from the acquired image; generating a first estimation model by performing machine learning using the fish quality information and the extracted features as learning data; estimating the quality of the fish from the acquired image using the first estimation model; It has.

[0012] [7] A program according to one aspect includes: A program for causing a computer to execute a method for estimating the quality of a fish from a cross-sectional image of the fish's tail, the method comprising: acquiring a cross-sectional image of the fish's tail; extracting a feature quantity characteristic of a cross-sectional image of a fish tail from the acquired image; generating a first estimation model by performing machine learning using the fish quality information and the extracted features as learning data; estimating the quality of the fish from the acquired image using the first estimation model; It has.

[0013] [8] A storage medium according to one aspect includes: A computer-readable storage medium storing the program described in [7] above.

[0014] [9] An apparatus according to one embodiment comprises: A fish quality estimation device that estimates the quality of a fish from a cross-sectional image of a fish's tail, comprising: an image acquisition unit that acquires a cross-sectional image of the fish's tail; an extraction unit that extracts a feature quantity that is characteristic of a cross-sectional image of a fish tail from the image acquired by the image acquisition unit; a machine learning unit that generates a first estimation model by performing machine learning using the fish quality information and the feature values ​​extracted by the extraction unit as learning data; an estimation unit that estimates the quality of fish from the image acquired by the image acquisition unit using the first estimation model generated by the machine learning unit; Equipped with. [Effects of the Invention]

[0015] According to the present invention, the quality of fish can be estimated. [Brief explanation of the drawings]

[0016] [Figure 1] FIG. 1 is a block diagram of a quality estimation system according to an embodiment of the present invention. [Figure 2] 10 is a flowchart showing an example of a feedback storage process in the fish quality estimation system of the present embodiment. [Figure 3] 10 is a flowchart showing an example of a process for generating a learning model (estimation model) for a specific user based on a basic model in the fish quality estimation system of this embodiment. [Figure 4] 10 is a flowchart showing an example of a fish quality estimation process using a learning model (estimation model) for a specific user in the fish quality estimation system of this embodiment. [Figure 5] 10 is a flowchart showing an example of a process for updating a learning model (estimation model) for a specific user in the fish quality estimation system of this embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0017] Hereinafter, a fish quality estimation system according to an embodiment of the present invention will be described with reference to the drawings. In this embodiment, a system for estimating the quality of tuna will be described as an example.

[0018] The configuration of a quality estimation system according to an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing the configuration of the quality estimation system according to this embodiment. As shown in FIG. 1, the quality estimation system 1 includes a server device 2 and a user device 3. The server device 2 and the user device 3 are connected to each other via a network 4 so that they can communicate with each other. For example, the server device 2 is a cloud server or the like owned by a provider of a quality assessment service, and the user device 3 is a smartphone or the like owned by a user of the quality assessment service.

[0019] As shown in FIG. 1, the server device 2 includes a machine learning unit 201, an input unit (image acquisition unit) 202, an estimation unit 203, a feedback acquisition unit 204, a model generation unit 205, a model update unit 206, a memory unit 207, and an extraction unit 208.

[0020] The machine learning unit 201 analyzes, by machine learning, the relationship between the feature amount extracted by the extraction unit 208 (described later) from image data of a cross section of a fish tail and the quality of the fish in the image data (for example, a ranking value). The relationship analyzed by the machine learning unit 201 is stored in the storage unit 207 as a learning model (estimation model). That is, the machine learning unit 201 generates, by machine learning, an estimation model for estimating the quality of the fish. Any method such as deep learning using a neural network is used for this machine learning.

[0021] For example, a neural network is configured so that image data of a cross-section of a tuna tail is input to an input layer, and quality data of the fish corresponding to that image data (e.g., a ranking value) is output from an output layer. Then, weighting coefficients between neurons in the neural network are optimized through supervised learning using analytical data (teaching data) that links the data input to the input layer with the data output from the output layer. For example, the teaching data for the quality data corresponding to the image data of the cross-section of the tuna tail can be the results of an expert's evaluation of the quality of the tuna (e.g., a ranking value).

[0022] Image data of the cross section of the tail of the tuna to be estimated is input to the input unit (image acquisition unit) 202. The image data of the cross section of the tail of the tuna to be estimated is acquired, for example, by photographing the cross section of the tuna tail at fisheries factories in various locations.

[0023] Estimation unit 203 estimates and outputs the quality of the tuna to be estimated based on the relationship (estimation model) analyzed by machine learning unit 201, using as input image data of the cross section of the tail of the tuna to be estimated that is input by input unit 202. For example, in the case of the above-mentioned neural network, the quality of the tuna to be estimated is estimated by inputting image data of the cross section of the tail of the tuna to be estimated into the input layer, estimating the quality of the tuna to be estimated, and outputting the estimation result from the output layer.

[0024] The feedback acquisition unit 204 acquires information (first feedback / second feedback) related to the user's feedback on the estimation result by the estimation unit 203. For example, when the estimation unit 203 outputs the quality of the fish as a rank "B" as the estimation result, the estimation unit 203 accepts a rank "A" as the user's subjective evaluation (feedback). The information related to the feedback acquired by the feedback acquisition unit 204 is stored in the storage unit 207.

[0025] The model generation unit 205 updates the basic model (first estimation model) using the feedback (first feedback) acquired by the feedback acquisition unit 204, thereby generating an estimation model (second estimation model) specialized for the user who input the feedback. Here, the basic model is a general-purpose estimation model (learning model) that is not specialized for a specific user. For example, the model generation unit 205 generates the second estimation model by updating various parameters constituting the first estimation model so as to reduce the loss, regarding the difference between the estimation result by the first estimation model and the user feedback as a loss.

[0026] The model update unit 206 updates an estimation model (second estimation model) specialized for a specific user using the feedback (second feedback) acquired by the feedback acquisition unit 204. For example, the model generation unit 205 updates the second estimation model by updating various parameters constituting the second estimation model so as to reduce a loss, regarding the difference between the estimation result by the second estimation model and the user feedback as a loss.

[0027] The memory unit 207 stores image data acquired from the user device 3, an estimation model corresponding to the relationship analyzed by the machine learning unit 201, the fish quality estimation results by the estimation unit 203, information regarding feedback acquired by the feedback acquisition unit 204, the estimation model generated by the model generation unit 205, the estimation model after update by the model update unit 206, etc. The extraction unit 208 extracts feature quantities characteristic of the cross-sectional image of the fish tail from the image acquired by the image acquisition unit. The feature quantities extracted by the extraction unit 208 can be, for example, color information of the cross-sectional image of the fish tail, or brightness data at a specific wavelength (e.g., near-infrared light) (e.g., brightness data at the wavelength of the lipid absorption peak, brightness data at the wavelength of the water absorption peak, etc.).

[0028] Next, we will explain the user device 3. As shown in Fig. 1, the user device 3 includes an image capturing unit 301, a data input unit 302, a display unit 303, and a storage unit 304.

[0029] The photographing unit 301 is realized by the camera function of the user device 3, and generates image data of the cross section of the tuna tail by photographing the cross section of the tuna tail. Note that the generated image data of the cross section of the tuna tail may be linked to data on the date of photographing.

[0030] The data input unit 302 has a function for inputting various data. Information about the tuna that is the subject of estimation (for example, identification information of the fish that is the subject of estimation, its weight, fishing ground, name of the fishing boat, etc.), user identification information (user ID), etc. are input from the data input unit 302.

[0031] The display unit 303 has a function of displaying various data. Information about the quality of the fish estimated by the estimation unit 203 is displayed on the display unit 303. If the display unit 303 has a touch panel function, the display unit 303 can also serve as the data input unit 302.

[0032] The storage unit 304 stores image data of the tuna photographed by the photographing unit 301 and information input via the data input unit 302. The storage unit 304 also stores data on the quality of the fish received from the server device 2.

[0033] Next, the operation of the quality estimation system 1 will be described.

[0034] (Estimation using a basic model) 2, in the quality estimation system 1 of this embodiment, first, the machine learning unit 201 of the server device 2 analyzes in advance by machine learning the relationship between image data of a cross section of a tuna tail and the quality of the fish corresponding to that image data (S1). The relationship analyzed by this machine learning is stored as an estimation model (learning model) in the storage unit 207. The estimation model in step S1 is a general-purpose estimation model (basic model / first estimation model) that is suitable to some extent for any user.

[0035] Next, the user of the user device 3 inputs user information (identification information such as a user ID and password) via the data input unit 302 (S2).

[0036] The user information input by the user is transmitted from the user device 3 to the server device 2 via the network 4 (S3).

[0037] The server device 2 receives the user information from the user device 3 and identifies the user based on the received user information (S4).

[0038] Next, the user of the user device 3 photographs a cross section of the fish's tail using the photographing unit 301 (S5).

[0039] Image data of the cross section of the fish tail photographed by the user using the photographing unit 301 is transmitted from the user device 3 to the server device 2 (S6).

[0040] The estimation unit 203 of the server device 2 receives the image data of the cross section of the fish tail from the user device 3 and estimates the quality of the fish using the estimation model (basic model / first estimation model) obtained in step S1 (S7).

[0041] Then, the server device 2 transmits the fish quality estimation result (for example, quality rank "B") by the estimation unit 203 to the user device 3 (S8).

[0042] The user device 3 receives the estimation result from the server device 2 and displays the estimation result on the display unit 303 (S9).

[0043] After checking the estimation result displayed on the display unit 303 of the user device 3, the user inputs feedback (user's subjective evaluation) on the estimation result (S10). For example, if the user subjectively judges the quality of the fish to be rank "A," the user inputs rank "A" via the data input unit 302.

[0044] The feedback input by the user (for example, a quality rank "A") is transmitted from the user device 3 to the server device 2 (S11).

[0045] The server device 2 receives the feedback from the user device 3 and stores information about the feedback in the storage unit 207 in association with the user information (S12).

[0046] (Generating estimation models for specific users) Next, a flow of a process for generating an estimation model for a specific user will be described. Fig. 3 is a flowchart showing an example of a process for generating an estimation model for a specific user (second estimation model) using information related to feedback obtained in the process shown in Fig. 2.

[0047] As shown in FIG. 3, first, the user of the user device 3 inputs user information via the data input unit 302 (S10).

[0048] The user information input by the user is transmitted from the user device 3 to the server device 2 via the network 4 (S11).

[0049] The server device 2 receives the user information from the user device 3 and identifies the user based on the received user information (S12).

[0050] Next, the user of the user device 3 performs a "model generation" operation via the data input unit 302 or by operating the GUI displayed on the display unit 303 (S13). For example, model generation is performed by selecting a "model generation" button displayed on the display unit 303. Here, model generation refers to generating an estimation model specialized for the user of the user device 3.

[0051] When a command to "create a model" is issued based on an operation by the user of the user device 3, information instructing execution of "create a model" is transmitted from the user device 3 to the server device 2 (S14).

[0052] When the server device 2 receives information instructing execution of "model generation" from the user device 3, it reads the basic model (first estimated model) from the storage unit 207 and copies it (S15).

[0053] Next, the server device 2 reads out information relating to feedback associated with the user information from the storage unit 207 (S16).

[0054] Then, the model generation unit 205 of the server device 2 updates the basic model (first estimation model) based on the information related to the read feedback, thereby generating an estimation model (second estimation model) for the specific user (S17).

[0055] (Estimation using estimation models for specific users) Next, a description will be given of an estimation process using the estimation model (second estimation model) generated by the process shown in Fig. 3. Fig. 4 is a flowchart showing an example of the estimation process using the second estimation model.

[0056] As shown in FIG. 4, first, the user of the user device 3 inputs user information via the data input unit 302 (S20).

[0057] The user information input by the user is transmitted from the user device 3 to the server device 2 via the network 4 (S21).

[0058] The server device 2 receives the user information from the user device 3 and identifies the user (for example, user A) based on the user information (S22).

[0059] Next, the user of the user device 3 photographs a cross section of the fish's tail using the photographing unit 301 (S23).

[0060] Image data of the cross section of the fish tail photographed by the user using the photographing unit 301 is transmitted from the user device 3 to the server device 2 (S24).

[0061] The server device 2 selects an estimation model (second estimation model) corresponding to the user identified in step S22, and reads out the estimation model (second estimation model) from the storage unit 207 (S25).

[0062] Next, the estimation unit 203 of the server device 2 estimates the quality of the fish using the read estimation model (second estimation model) (S26).

[0063] Then, the server device 2 transmits the fish quality estimation result (for example, quality rank "B") by the estimation unit 203 to the user device 3 (S27).

[0064] The user device 3 receives the estimation result from the server device 2 and displays the estimation result on the display unit 303 (S28).

[0065] The user confirms the estimation result displayed on the display unit 303 of the user device 3 and inputs feedback (user's subjective evaluation) on the estimation result (S29). For example, if the user subjectively judges the quality of the fish to be rank "C," the user inputs rank "C" via the data input unit 302.

[0066] The feedback input by the user (for example, a quality rank "C") is transmitted from the user device 3 to the server device 2 (S30).

[0067] The server device 2 receives the feedback from the user device 3 and stores information about the feedback in the storage unit 207 in association with the user information (S31).

[0068] (Update estimation model for specific users) Next, the update process of the estimation model for a specific user (second estimation model) will be described with reference to the flowchart of FIG.

[0069] As shown in FIG. 5, first, the user of the user device 3 inputs user information via the data input unit 302 (S40).

[0070] The user information input by the user is transmitted from the user device 3 to the server device 2 via the network 4 (S41).

[0071] The server device 2 receives the user information from the user device 3 and identifies the user (for example, user A) based on the user information (S42).

[0072] Next, the user of the user device 3 performs a "model update" operation via the data input unit 302 or by operating the GUI displayed on the display unit 303 (S43). For example, the model update is performed by selecting the "model update" button displayed on the display unit 303. The model update here refers to updating the estimation model (second estimation model) specialized for the user of the user device 3.

[0073] When a "model update" instruction is given based on an operation by the user of the user device 3, information instructing execution of the "model update" is transmitted from the user device 3 to the server device 2 (S44).

[0074] When the server device 2 receives information instructing execution of "model update" from the user device 3, it selects a second estimation model corresponding to the user information and reads the second estimation model from the storage unit 207 (S55).

[0075] Next, the server device 2 reads out information about feedback corresponding to the user information from the storage unit 207 (S46).

[0076] Then, the model update unit 206 of the server device 2 updates the second estimation model based on the information related to the read feedback (S47).

[0077] As such, the quality estimation system 1 of this embodiment includes an image acquisition unit 202 that acquires a cross-sectional image of the fish's tail, an extraction unit that extracts features that are characteristic of the cross-sectional image of the fish's tail from the image acquired by the image acquisition unit 202, a machine learning unit that generates a first estimation model by machine learning the fish's quality information and the features extracted by the extraction unit as learning data, and an estimation unit that estimates the quality of the fish from the image acquired by the image acquisition unit using the first estimation model generated by the machine learning unit. With this configuration, even an unskilled person can easily estimate the quality of the fish. In addition, the quality estimation system 1 of this embodiment further includes a feedback acquisition unit that acquires first feedback on the estimation result by the estimation unit, and a model generation unit that generates a second estimation model by updating the first estimation model using the first feedback acquired by the feedback acquisition unit. According to this configuration, it is possible to estimate the quality of fish specifically for a specific user.

[0078] Although the present embodiment has been described above, any part or all of the functional units described in this specification may be realized by a program. The program mentioned in this specification may be distributed by being non-temporarily recorded on a computer-readable recording medium, or may be distributed via a communication line (including wireless communication) such as the Internet, or may be distributed in a state where it is installed on any terminal.

[0079] Based on the above description, a person skilled in the art may be able to conceive additional effects and various modifications of the present invention, but the aspects of the present invention are not limited to the individual embodiments described above. Various additions, modifications, and partial deletions are possible within the scope of the conceptual idea and spirit of the present invention, which is derived from the content defined in the claims and their equivalents.

[0080] For example, what is described herein as a single device (or component, the same applies hereinafter) (including what is depicted as a single device in the drawings) may be realized by multiple devices. Conversely, what is described herein as multiple devices (including what is depicted as multiple devices in the drawings) may be realized by a single device. Alternatively, some or all of the means and functions included in a certain device (e.g., a server) may be included in another device (e.g., a user terminal).

[0081] Furthermore, not all of the features described in this specification are essential requirements. In particular, features described in this specification but not included in the claims can be considered optional additional features.

[0082] It should be noted that the applicant is merely aware of the inventions disclosed in the documents listed in the "Prior Art Documents" section of this specification, and the present invention does not necessarily aim to solve the problems of the disclosed inventions. The problem that the present invention aims to solve should be determined by taking into consideration the entire specification. For example, if this specification states that a specific configuration achieves a certain effect, it can also be said that the present invention solves a problem that is the reverse of that effect. However, it is not necessarily intended that such a specific configuration be an essential requirement. [Industrial Applicability]

[0083] As described above, the fish quality estimation system according to the present invention has the effect of being able to estimate the quality of fish, and is useful as a fish quality estimation system, etc. [Explanation of symbols]

[0084] 1 Quality estimation system 2. Server device 201 Machine Learning Department 202 Input unit (image acquisition unit) 203 Estimation Department 204 Feedback Acquisition Department 205 Model Generation Unit 206 Model Update Department 207 Memory section 208 Extraction part 3 User Device 301 Photography Department 302 Data Entry Section 303 Display section 304 Storage section 4 Network

Claims

1. A fish quality estimation system that estimates the quality of a fish from a cross-sectional image of a fish's tail, comprising: an image acquisition unit that acquires a cross-sectional image of the fish's tail; an extraction unit that extracts a feature quantity that is characteristic of a cross-sectional image of a fish tail from the image acquired by the image acquisition unit; a machine learning unit that generates a first estimation model by performing machine learning using the fish quality information and the feature values ​​extracted by the extraction unit as learning data; an estimation unit that estimates the quality of fish from the image acquired by the image acquisition unit using the first estimation model generated by the machine learning unit; A fish quality estimation system comprising:

2. a feedback acquisition unit that acquires a first feedback on an estimation result by the estimation unit; a model generation unit that generates a second estimation model by updating the first estimation model using the first feedback acquired by the feedback acquisition unit; The fish quality estimation system according to claim 1 , comprising:

3. The fish quality estimation system according to claim 2 , wherein the estimation unit estimates the quality of the fish from the image acquired by the image acquisition unit using the second estimation model generated by the model generation unit.

4. the feedback acquisition unit further acquires second feedback on the estimation result estimated by the estimation unit using the second estimation model; The fish quality estimation system according to claim 3 , further comprising a model update unit that updates the second estimation model using the second feedback acquired by the feedback acquisition unit.

5. The fish quality estimation system according to claim 3 , wherein the extraction unit extracts color information as a feature characteristic of the cross-sectional image of the fish.

6. A method executed by a fish quality estimation system for estimating fish quality from a cross-sectional image of a fish tail, comprising: acquiring a cross-sectional image of the fish's tail; extracting a feature quantity characteristic of a cross-sectional image of a fish tail from the acquired image; generating a first estimation model by performing machine learning using the fish quality information and the extracted features as learning data; estimating the quality of the fish from the acquired image using the first estimation model; A method comprising:

7. A program for causing a computer to execute a method for estimating the quality of a fish from a cross-sectional image of the fish's tail, the method comprising: acquiring a cross-sectional image of the fish's tail; extracting a feature quantity characteristic of a cross-sectional image of a fish tail from the acquired image; generating a first estimation model by performing machine learning using the fish quality information and the extracted features as learning data; estimating the quality of the fish from the acquired image using the first estimation model; A program having:

8. A computer-readable storage medium storing the program according to claim 7.

9. A fish quality estimation device that estimates the quality of a fish from a cross-sectional image of a fish's tail, comprising: an image acquisition unit that acquires a cross-sectional image of the fish's tail; an extraction unit that extracts a feature quantity that is characteristic of a cross-sectional image of a fish tail from the image acquired by the image acquisition unit; a machine learning unit that generates a first estimation model by performing machine learning using the fish quality information and the feature values ​​extracted by the extraction unit as learning data; an estimation unit that estimates the quality of fish from the image acquired by the image acquisition unit using the first estimation model generated by the machine learning unit; A fish quality estimation device comprising:

Images