Information processing apparatus, information processing system, information processing method, and program

Abstract

To identify a type of an object with high accuracy.SOLUTION: An information processing apparatus includes: first identification means which inputs a first image obtained by capturing an object in a first photographic condition, to a first model to identify a photographic condition appropriate to identifying a type of the object; and second identification means which inputs a second image obtained by capturing the object in a second photographic condition identified by the first identification means, to a second model to identify the type of the object.SELECTED DRAWING: Figure 4

Description

Technical Field

[0001] The present invention relates to object identification.

Background Art

[0002] Japan's catch has been decreasing year by year. One of the reasons is the decline in fishery resources. In order to stop the decline in catch, the country is trying to manage the catch and strengthen fishery resource management. To grasp the catch, the country requires fishermen to report detailed catch records, but currently fishermen do not have the spare capacity to expend effort on catch surveys. Therefore, an image analysis system for catch monitoring is required. To realize an image analysis system for catch monitoring, a model for classifying fish species is necessary. Patent Document 1 discloses a method for discriminating fish species using deep learning. Also, Patent Document 2 discloses a method of applying an appropriate recognition method according to shooting conditions for recognition.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Patent Document 2

Non-Patent Documents

[0004]

Non-Patent Document 1

Non-Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, since the lighting condition of the fish body and the color of the fish body vary from fish species to fish species, it may be difficult to identify the fish species from an image taken under fixed shooting conditions. For example, blue fish such as horse mackerel and saury are likely to reflect light, and when illuminated from the front, the fish body will shine and correct identification cannot be made. Also, if the color of the platform on which the fish is placed is the same as the color of the fish body, it is difficult to distinguish between the fish body and the identification of the fish species becomes difficult.

[0006] The present invention has been made in view of such problems, and an object thereof is to accurately identify the type of an object.

Means for Solving the Problems

[0007] The information processing apparatus according to the present invention includes: a first identifying means for identifying shooting conditions suitable for identifying the type of an object by inputting a first image obtained by imaging the object under first shooting conditions into a first model; and a second identifying means for identifying the type by inputting a second image obtained by imaging the object under second shooting conditions identified by the first identifying means into a second model. wherein the second model is prepared for each shooting condition, and the second identification means uses the second model associated with the second shooting condition among the plurality of second models It is characterized by this.

Effects of the Invention

[0008] According to the present invention, the type of an object can be accurately identified.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Embodiments for Carrying Out the Invention

[0010] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. Note that the configurations shown in the following embodiments are merely examples, and the present invention is not limited to the illustrated configurations.

[0011] FIG. 1 shows an overall configuration example of the information processing system according to the present embodiment. The information processing system includes a belt conveyor 104 that conveys objects 105 and 106, imaging devices 101 and 102 that image the objects 105 and 106, an information processing device 100 that performs identification, and an automatic sorting machine 107 that sorts the objects 105 and 106. The imaging devices 101 and 102 image the objects 105 and 106 on the belt surface of the belt conveyor 104 and transmit the images to the information processing device 100. The imaging device 102 is disposed on the downstream side of the imaging device 101 with respect to the conveyance direction of the belt conveyor 104. Further, as lighting equipment for the imaging device 102, a lighting device 103 is provided. The lighting device 103 includes a light source that irradiates the object on the belt surface from the front at the imaging position of the imaging device 102, and a backlight incorporated in the belt conveyor 104 so as to irradiate the object on the belt surface from the back. Note that the belt surface has translucency. The automatic sorting machine 107 is disposed on the downstream side of the imaging device 102 with respect to the conveyance direction of the belt conveyor 104, and sorts the objects 105 and 106 that have flowed through the belt conveyor 104 into any one of a plurality of sorting bins 108 to 110 according to the identification result. The information processing device 100, the imaging devices 101 and 102, and the lighting device 103 are communicably connected to each other via a network 211 (FIG. 2). Further, the information processing device 100 outputs the identification result to the automatic sorting machine 107 via an output I / F 206 (FIG. 2). Hereinafter, the description will be made assuming that the object is a fish, but the fish is merely an example of an object as the object.

[0012] The information processing apparatus 100 receives, from the imaging device 101, an image captured under predetermined imaging conditions at the imaging position of the imaging device 101, and detects a region of a target fish from the received image. Then, by inputting the image of the fish region into a pre-trained first model, it identifies imaging conditions suitable for identifying the type of the target fish. Also, it specifies a second model associated with the identified imaging conditions. The information processing apparatus 100 controls so as to change the camera settings of the imaging device 102 and the lighting settings of the lighting device 103 according to the identified imaging conditions. Thereafter, the information processing apparatus 100 receives, from the imaging device 102, an image captured at the timing when the target fish flows on the belt conveyor 104 and reaches the imaging position of the imaging device 102. Note that the information processing apparatus 100 predicts the timing at which the same fish as the fish detected from the image of the imaging device 101 reaches the imaging position of the imaging device 102, based on the conveyance speed of the belt conveyor 104 and the distance between the imaging device 101 and the imaging device 102. The information processing apparatus 100 may receive a moving image from the imaging device 102 and acquire a frame image at the predicted timing from the received moving image. The information processing apparatus 100 detects a fish region from the image received from the imaging device 102, and estimates the fish species of the target fish by inputting the image of the fish region into a pre-trained second model. The information processing apparatus 100 controls the automatic sorting machine 107 to sort the target fish into the sorting bins 108 to 110 according to the estimated fish species.

[0013] Here, for the fish as the object according to the present embodiment, characteristics such as the glossiness of the surface, the color of the surface, the smoothness of the surface, the shape, and the size differ for each fish species. Therefore, it is difficult to accurately identify all fish species from an image captured under fixed imaging conditions. For example, in the case of blue fish such as horse mackerel and saury, since they easily reflect light, when photographed from the front with a light source from the front, the fish body will shine. In such a case, by adjusting the installation angles of the lighting device and the imaging device so that the reflection of the illumination light does not directly enter the imaging device, it becomes easier to identify the fish species. Also, when the color of the fish body surface is the same color system as the color of the belt surface of the belt conveyor, it is difficult to distinguish from the fish body. In such a case, by using the backlight of the belt conveyor and changing the color of the backlight according to the color of the fish body surface, it becomes easier to identify the fish species. Also, when the surface of the fish body is not smooth like a sea bream, by adjusting to cast light obliquely so that a shadow appears, or in the case of a flat fish, by adjusting to cast light from the front so that the entire fish body is illuminated, it becomes easier to identify the fish species. As described above, it is considered that the identification performance of fish species is improved by using an image captured under imaging conditions suitable for identifying fish species. Therefore, in the present embodiment, a first model is used to identify imaging conditions suitable for identifying the type of target fish.

[0014] Next, with reference to FIG. 5, an overview of the imaging conditions to be identified in the first model will be described. In the present embodiment, as shown in FIG. 5, for each of a plurality of imaging conditions, settings of various imaging parameters are determined. Note that FIG. 5 shows imaging conditions A to D, but imaging conditions A to D are merely examples, and there may be additional imaging conditions. The settings of the imaging parameters include the camera settings of the imaging device 102 and the illumination settings of the illumination device 103. The camera settings include, for example, settings related to the imaging position such as whether to image from the front or at an angle of 30 degrees obliquely, and settings related to camera parameters such as the shutter speed. The illumination settings include, for example, settings related to the illumination position such as whether to irradiate from the front or at an angle of 45 degrees obliquely, settings related to the light amount, and settings related to backlight (ON / OFF, light amount, color). Note that the settings of the imaging parameters are not limited to these. For example, in addition to these, there may be settings such as the type and color temperature of the light source. Also, for example, there may be settings for the aperture and focus. Further, there may be a setting for the conveyance speed of the belt conveyor 104. Also, there are combinations of setting contents that are not used among the imaging parameters. For example, a combination of a weak light amount and a fast shutter speed is not used because it results in insufficient exposure. Also, the camera settings include a setting for the zoom magnification, such as zooming in on the target fish when the range of the fish area in the image is less than a predetermined size. Among the plurality of imaging conditions A to D as shown in FIG. 5, which imaging condition is suitable for fish species identification is acquired through learning. The learning method will be described later.

[0015] In this embodiment, as shown below, identification is performed in two stages. In the first-stage identification, identification of imaging conditions suitable for fish species identification is performed. In the second-stage identification, fish species identification is performed using an image captured under the imaging conditions identified in the first stage. In the first-stage identification, a first model, which is a classification model for performing the first-stage identification, is used to identify imaging conditions suitable for identifying the type of fish captured in an image (first image) captured under fixed imaging conditions (first imaging conditions). The second model is a classification model for performing the second-stage identification, and is learned for each imaging condition and stored in association with the imaging condition. In this embodiment, since there are multiple imaging conditions, there are also multiple second models. In the second-stage identification, a second model associated with the imaging conditions identified in the first-stage identification is used to identify the type of fish captured in an image (second image) captured under the imaging conditions (second imaging conditions) determined in the first-stage identification. As described above, in the first-stage identification, identification of imaging conditions suitable for fish species identification is performed. In the second-stage identification, fish species identification is performed using an image captured under imaging conditions suitable for each fish species. This can improve the accuracy of fish species identification. Furthermore, in the first-stage identification, identification of groups of fish species is performed based on imaging conditions suitable for fish species identification. In the second-stage identification, identification is performed within a group of fish species with the same suitable imaging conditions. Therefore, the number of classes to be classified can be reduced, and the time required for fish species identification can be shortened. The learning methods of the first model and the second model will be described later.

[0016] Next, the hardware configuration of the information processing system according to this embodiment will be described. FIG. 2 shows an example of the hardware configuration of the information processing system according to this embodiment. The information processing apparatus 100 is composed of a CPU 201, a ROM 202, a RAM 203, an external storage device 204, an input I / F 205, an output I / F 206, and a communication I / F 207. These components are communicably connected via a system bus 208.

[0017] The CPU (Central Processing Unit) 201 controls the entire information processing apparatus 100. The ROM (Read Only Memory) 202 stores programs and parameters that do not require modification. The RAM (Random Access Memory) 203 temporarily stores data supplied from external devices and the like. The external storage device 204 is a storage device such as a hard disk or a memory card fixedly installed in the information processing apparatus 100. Note that the external storage device 204 may include a flexible disk (FD), an optical disk such as a compact disk (CD), a magnetic or optical card, an IC card, a memory card, etc., which are detachable from the information processing apparatus 100. Note that the functions and processes of the information processing apparatus 100 described later are realized by the CPU 201 reading out the programs stored in the ROM 202 and the like and executing these programs.

[0018] The input I / F 205 is an interface with an input device 209 such as a pointing device or a keyboard that receives the user's operation and inputs data. The output device I / F 206 is an interface with a monitor 210 for displaying the data held by the information processing apparatus 100 or the supplied data, and an automatic sorting machine 107 that automatically sorts fish based on the identification result. The communication I / F 207 connects to the network 211. The CPU 201 transmits and receives data to and from the imaging devices 101, 102, and the lighting device 103 via the network 211. The imaging devices 101, 102, and the lighting device 103 have a communication function and are connected to the network 211. Note that the belt conveyor 104 may also be connected to the network 211. In this case, the information processing apparatus 100 may control the start / end of conveyance and the conveyance speed of the belt conveyor 104.

[0019] Figure 3 shows an example of the data flow used in the information processing system according to this embodiment. In Figure 3, the information processing apparatus 100 functions as a detection unit 301 that detects a fish region from an image, and an identification unit 302 that performs identification using the image of the fish region. In the external storage device 204 of the information processing apparatus 100, a first model and a second model used by the identification unit 302 are stored. The second model is prepared for each of a plurality of shooting conditions. Hereinafter, the description will be made along the data flow.

[0020] First, the information processing apparatus 100 acquires a captured image 303 from the imaging apparatuses 101 and 102 via the network 211 and the communication I / F 207. The detection unit 301 analyzes the acquired captured image 303 and detects the region of the fish shown in the image. The detection unit 301 uses, for example, an object detection method using CNN (Convolutional Neural Networks) or Faster R-CNN described in Non-Patent Document 1. In Faster R-CNN, multi-class objects such as people and cars can be detected. When using Faster R-CNN, learning is performed in advance using a large number of collected fish images so that fish can be detected.

[0021] The identification unit 302 identifies the fish region extracted by the detection unit 301. When the target captured image 303 is acquired from the imaging apparatus 101, the identification unit 302 reads out the first model 304 from the external storage device 204 and identifies the shooting conditions using the first model 304. Further, the identification unit 302 specifies the second model 307 associated with the identified shooting condition 306 among the plurality of second models stored in the external storage device 204. Thereafter, the information processing apparatus 100 transmits the identified shooting condition 306 to the imaging apparatus 102 and the lighting apparatus 103 via the communication I / F 207. The imaging apparatus 102 and the lighting apparatus 103 perform imaging by adjusting various settings based on the received shooting condition 306.

[0022] On the other hand, when the target captured image 303 is acquired from the imaging device 102, the identification unit 302 reads out the second model 307 specified above from the external storage device 204, identifies the fish species using the second model 307, and obtains the identification result 305. Thereafter, the information processing device 100 notifies the automatic sorting machine 107 of the identification result 305 via the output I / F 206. The automatic sorting machine 107 performs sorting according to the identification result 305. Note that the identification unit 302 may be configured to acquire the shooting conditions used at the time of imaging from the imaging device 102 together with the captured image 303 and use the second model associated with the acquired shooting conditions.

[0023] Subsequently, with reference to FIG. 4, the identification process executed by the information processing device 100 according to the present embodiment will be described. FIG. 4 is a flowchart showing the flow of the identification process. The flowchart in FIG. 4 is realized by the CPU 201 expanding and executing a program stored in the ROM 202 or the like in the RAM 203. The flowchart in FIG. 4 starts when the belt conveyor 104 is operated by an operation of the input device 209 and a fish is placed on the belt conveyor 104. Hereinafter, each step (step) of the flowchart will be described with an S (step) added at the beginning of those reference numerals.

[0024] First, in S401, the CPU 201 acquires a first image from the imaging device 101 via the communication I / F 207. Next, in S402, the CPU 201 detects a fish region from the first image acquired in S401. Here, the flow when one fish region is detected will be described. When a plurality of fish regions are detected, the processes from S403 to S409 hereinafter may be repeated, and the description thereof will be omitted here. Next, at S403, the CPU 201 inputs the image of the fish area detected at S402 into the first model 304, and identifies the shooting conditions 306 suitable for fish species identification and the second model 307. The first model 304 is a model that identifies which shooting conditions are suitable when identifying the type of fish detected at S402. The CPU 201 uses the first model 304 to identify the class corresponding to the shooting conditions suitable for identifying the fish species of the input image. Specifically, class classification such as shooting conditions A to D shown in FIG. 5 is performed. The second model 307 is a model that identifies the fish species to be identified under each shooting condition, and one second model is associated with each shooting condition.

[0025] Next, at S404, the CPU 201 compares the shooting conditions 306 specified at S403 with the shooting conditions used when the first image was captured, and determines whether they are the same shooting conditions. If the CPU 201 determines that the shooting conditions are the same, since the first image can be used as the second image as it is, S405 to S407 are skipped, and the process proceeds to S408. On the other hand, if the CPU 201 determines that the shooting conditions are different, the process proceeds to S405. At S405, the CPU 201 transmits the shooting conditions 306 specified at S403 to the imaging device 102 and the lighting device 103 via the communication I / F 207. The imaging device 102 and the lighting device 103 change their settings according to the received shooting conditions 306. Next, at S406, the CPU 201 acquires, via the communication I / F 207, the second image captured under the shooting conditions 306 from the imaging device 102 at the timing when the same fish as the fish detected at S402 reaches the shooting position of the imaging device 102. Next, at S407, the CPU 201 detects the fish area from the second image acquired at S406. Next, at S408, the CPU 201 inputs the image of the fish area detected at S407 into the second model 307 specified at S403 to identify the fish species. When the shooting conditions 306 specified at S403 are the same as the shooting conditions used when the first image was captured, the image of the fish area detected at S402 is input. Next, in S409, the CPU 201 notifies the identified fish species to the automatic sorting machine 107 via the output I / F 206. The automatic sorting machine 107 sorts the fish detected in S402 according to the notified fish species. Thereafter, the series of identification processes shown in the flowchart ends.

[0026] According to the identification process according to the present embodiment as described above, since the images captured under the shooting conditions suitable for each fish species are used, the identification accuracy of the fish species can be improved. Further, since the suitable shooting conditions are for identification within the same fish species group, the number of classes to be classified can be reduced. Therefore, the time required for fish species identification can be shortened.

[0027] Subsequently, with reference to FIG. 6, the learning process in the learning phase of the information processing apparatus 100 according to the present embodiment will be described. FIG. 6 is a flowchart showing the flow of the learning process. The flowchart of FIG. 6 is realized by the CPU 201 expanding and executing the program stored in the ROM 202 or the like in the RAM 203. FIG. 7 shows the data generated by the learning process of FIG. 6. First, as learning data for learning, a plurality of types of fish are imaged for each different shooting condition using the imaging devices 101 and 102 according to the present embodiment, thereby preparing a plurality of captured images. Each captured image is labeled with the shooting conditions and fish species used at the time of imaging as correct data. First, in S601, the CPU 201 performs learning of the second model using a DNN (Deep Neural Network). Details of the learning method will be described later. This learning is performed for each shooting condition in FIG. 5. Then, the CPU 201 performs learning so as to be in separate classes for each fish species, and obtains the identification accuracy for each fish species. When the learning for all shooting conditions is completed, the CPU 201 aggregates the identification accuracy for each shooting condition for each fish species. Thereafter, the CPU 201 determines the shooting conditions suitable for identification for each fish species based on the aggregation result.

[0028] Figure 7(a) shows an example of the result of aggregating the discrimination accuracies under each imaging condition for each fish species. In Figure 7(a), for horse mackerel, mackerel, squid, sea bream, red sea bream, and porgy, the discrimination accuracies under imaging conditions A to D are shown respectively. Note that the imaging conditions A to D in Figures 7(a) to 7(c) are the same as the imaging conditions A to D shown in Figure 5. Here, the CPU 201 determines the imaging condition suitable for discrimination as the imaging condition with the highest discrimination accuracy. Note that it is not limited to the case where the discrimination accuracy is the highest as long as it is higher than others. The location marked with a star (★) in Figure 7(a) indicates the imaging condition with the highest discrimination accuracy. For example, in the case of horse mackerel, since the discrimination accuracies under imaging conditions A to D are 32%, 47%, 66%, and 95%, it is determined as imaging condition D. Also, for example, in the case of mackerel, since the discrimination accuracies under imaging conditions A to D are 18%, 33%, 52%, and 94%, it is determined as imaging condition D. Also, for example, in the case of squid, since the discrimination accuracies under imaging conditions A to D are 99%, 75%, 83%, and 67%, it is determined as imaging condition A. Similarly, for sea bream, red sea bream, and porgy, the imaging condition suitable for discrimination is determined according to the discrimination accuracy under each imaging condition. Figure 7(b) shows an example of a list of fish species suitable for discrimination under each imaging condition A to D. As shown in Figure 7(b), fish species can be grouped according to the imaging conditions suitable for the discrimination of fish species. Thereby, in S403 of Figure 4, fish can be classified into the same number of groups as the number of imaging conditions. In the example of Figure 7(b), as fish species suitable for discrimination under imaging condition D, horse mackerel, mackerel, and Pacific saury are classified into the same group.

[0029] Next, in S602, the CPU 201 performs learning of the first model using the DNN. This learning is performed using the learning data of one of the plurality of shooting conditions (for example, shooting condition A). This learning is performed such that fish species of the same group with the same shooting condition are classified into the same class, and fish species of different groups with different shooting conditions are classified into different classes. The model learned in this step is used as the first model in the identification process of FIG. 4. Also, the shooting condition of the first image acquired in S401 uses the shooting condition (here, shooting condition A) used for the learning in this step. In the above manner, a first model for identifying a shooting condition suitable for fish species identification is generated from the image captured under shooting condition A. Next, in S603, the CPU 201 performs re-learning of the second model learned in S601. The learning in S601 was performed to identify all fish species. In this step, for each shooting condition, learning is performed to identify fish species belonging to the same shooting condition. For example, using the learning data of shooting condition A, learning is performed such that the fish determined to belong to shooting condition A in S601 are in separate classes for each fish species. The model learned in this step is used as the second model associated with shooting condition A when shooting condition A is identified in S403 in the identification process of FIG. 4. Re-learning of the second model is also performed in the same manner for shooting conditions B to D. The fish species to be identified by the second model are different for each associated shooting condition. In the above manner, a second model for each shooting condition is generated.

[0030] In this embodiment, the shooting conditions for the first image acquired in S401 are set to use shooting condition A. Here, the CPU 201 may perform learning on which shooting conditions are suitable for capturing the first image. As a method, for example, using the learning data of shooting conditions A to D, learning in S602 is performed for each shooting condition. Then, the discrimination accuracy is obtained for each shooting condition. When the learning for all shooting conditions is completed, the CPU 201 determines the shooting conditions suitable for discrimination based on the discrimination accuracy for each shooting condition. FIG. 7(c) shows an example of the discrimination accuracy for each shooting condition. Here, the CPU 201 determines the shooting conditions suitable for discrimination as the shooting conditions with the highest discrimination accuracy. Note that it is not limited to the case where the discrimination accuracy is the highest as long as it is higher than others. The location marked with a star (★) in FIG. 7(c) indicates the shooting conditions with the highest discrimination accuracy. In the example of FIG. 7(c), it is shooting condition C. Thus, it can be seen that it is optimal to use the image captured under shooting condition C for learning as the first model. In this case, in the discrimination process of FIG. 4, in S401, the first image captured under shooting condition C is acquired, and in S403, the first model learned using the image captured under shooting condition C is used.

[0031] FIG. 8 shows a configuration example of the learning system used when performing learning. Hereinafter, a learning method when the learning system is configured by a DNN (Deep Neural Network) will be described. In this embodiment, as the neural network, for example, LeNet described in Non-Patent Document 2 is used.

[0032] In FIG. 8, an image 81 is input to the DNN 82, and a vector 83 having the number of fish species to be discriminated in the cases of S601 and S603 or the number of shooting conditions in the case of S602 as the number of dimensions is output. The vector 83 is a vector in which each element is a probability from 0 to 1 and the total value is 1. Then, using the correct answer vector of the image 81, a loss function 84 is calculated. The correct answer vector is a vector in which only the elements of the correct answer among each element are 1 and the other elements are 0. As the loss function 84, the cross-entropy error H(p, q) of the following formula (1) is used.

[0033] [Number]

[0034] In the above formula (1), p is the correct vector of image 81, and q is the output vector 83 of DNN 82. x is the number of dimensions, that is, the number of fish species or the number of shooting conditions. Using the calculated value of the loss function 84, learning is performed by updating the weights of DNN 82 by the error propagation method. Using the learning system as described above, the first model and the second model are learned.

[0035] According to the information processing system according to the present embodiment as described above, shooting conditions suitable for identifying the fish species are determined in the first-stage identification, and the fish species are identified in the second-stage identification using the image taken under the shooting conditions. In the second-stage identification, since the identification is performed using the images captured under the shooting conditions suitable for each fish species, the identification accuracy of the fish species can be improved as compared with the case of using the images captured under fixed shooting conditions. Further, in the second-stage identification, since the suitable shooting conditions are for identification within the same fish species group, the number of classes to be classified can be reduced. Therefore, the time required for identifying the fish species can be shortened.

[0036] In the present embodiment, an example applied to the identification of fish species has been described. However, it can be applied to the identification of various other objects, such as the identification of the types of parts on the belt conveyor of the assembly line in the factory. Depending on the parts, features such as the presence or absence of gloss, unevenness, shape, and dimensions are different. In the first-stage identification, the shooting conditions suitable for the features of the parts as described above are identified, and in the second-stage identification, the types of parts are identified using the images captured under the identified shooting conditions. This makes it possible to accurately identify the types of parts.

[0037] (Other Embodiments) The present invention can also be implemented by supplying a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and causing one or more processors in a computer of the system or apparatus to read and execute the program. It can also be implemented by a circuit (for example, ASIC) that realizes one or more functions.

[0038] As described above, the present invention has been described together with the embodiments. However, the above embodiments are merely examples of concretization in implementing the present invention, and the technical scope of the present invention should not be construed in a limited manner by these. That is, the present invention can be implemented in various forms without departing from its technical idea or its main features.

[0039] Hereinafter, modified examples of the above embodiments will be described. As a first modified example, instead of transporting the objects 105 and 106 by the belt conveyor 104 and performing imaging for first-stage identification and imaging for second-stage identification at different locations, imaging for identification may be performed at the same location with different imaging conditions. In this case, it is not necessary to use the belt conveyor 104. As a second modified example, imaging may be performed at a plurality of locations on the belt conveyor 104 under imaging conditions for several minutes, and an image captured under imaging conditions suitable for identification may be selected from those images to perform second-stage identification.

[0040] As a third modified example, the objects 105 and 106 may be arranged in an aligned state on the belt conveyor 104 so that the number of objects captured in the first image and the second image is limited. As a fourth modified example, the learning process shown in FIG. 6 may be executed by another apparatus that performs learning. In that case, a configuration may be adopted in which a first model and a second model used for the identification process are acquired from the other apparatus.

Explanation of Reference Numerals

[0041] 100: Information processing apparatus, 101, 102: Imaging devices, 103: Lighting device, 104: Belt conveyor, 105, 106: Objects, 107: Automatic sorting machine

Claims

1. First identification means for identifying imaging conditions suitable for identifying the type of the object by inputting a first image obtained by imaging the object under first imaging conditions into a first model; Second identification means for identifying the type by inputting a second image obtained by imaging the object under second imaging conditions identified by the first identification means into a second model; characterized by comprising: The second model is prepared for each imaging condition, and the second identification means uses the second model associated with the second imaging condition among a plurality of the second models. An information processing apparatus.

2. The information processing apparatus according to claim 1, wherein the type to be identified is different for each of the second models.

3. The imaging conditions suitable for identifying the type are determined for each type based on the result of aggregating the identification accuracy of the second model for each of a plurality of imaging conditions. The information processing apparatus according to claim 1 or 2.

4. The information processing apparatus according to any one of claims 1 to 3, further comprising first learning means for learning the first model so that images obtained by imaging the object under the first imaging conditions are classified for each type having the same imaging conditions suitable for identifying the type.

5. The information processing apparatus according to any one of claims 1 to 4, further comprising second learning means for learning the second model so that images obtained by imaging the object under imaging conditions suitable for identifying the type are classified for each type.

6. The information processing apparatus according to claim 4, wherein the first learning means determines imaging conditions to be used as the first imaging conditions based on the result of aggregating the identification accuracy of the first model for each of a plurality of imaging conditions.

7. The information processing apparatus according to any one of claims 1 to 6, wherein the second identification means inputs the first image into the second model when the second imaging conditions are the same as the first imaging conditions.

8. The information processing apparatus according to any one of claims 1 to 7, further comprising detection means for detecting an area of the object from at least one of the first image and the second image.

9. The information processing apparatus according to any one of claims 1 to 8, wherein the imaging conditions include at least one setting among settings related to the imaging position and camera parameters.

10. The information processing apparatus according to any one of claims 1 to 9, wherein the imaging conditions include at least one setting among settings related to the illumination position, the amount of light of the illumination, and the color of the illumination.

11. The information processing apparatus further includes control means for controlling an imaging device that images the object, wherein the control means controls to change the settings of the imaging device to the second imaging conditions, The information processing apparatus according to any one of claims 1 to 10, wherein the second identification means acquires the second image from the imaging device.

12. The information processing apparatus according to claim 11, wherein the control means further controls an illumination device of the imaging device and controls to change the settings of the illumination device to the second imaging conditions.

13. The object is being conveyed on a belt conveyor, wherein the control means controls a first imaging device and a second imaging device provided downstream of the first imaging device with respect to the conveying direction of the belt conveyor, wherein the first identification means acquires the first image from the first imaging device, The information processing apparatus according to claim 11 or 12, wherein the second identification means acquires the second image from the second imaging device at a timing predicted that the same object imaged by the first imaging device reaches the imaging position of the second imaging device based on the distance between the first imaging device and the second imaging device and the conveying speed of the belt conveyor.

14. The information processing apparatus according to any one of claims 1 to 13, wherein the second identification means selects an image captured under the second imaging conditions from among images captured under a plurality of imaging conditions.

15. The information processing apparatus according to any one of claims 1 to 14, wherein the object is a fish.

16. The information processing apparatus according to any one of claims 1 to 15, wherein at least one of the first model and the second model is configured by a neural network.

17. An information processing system including an imaging device that images an object and an information processing device that identifies the object, wherein the information processing device has first acquisition means for acquiring a first image captured by the imaging device under first imaging conditions, first identification means for identifying imaging conditions suitable for identifying the type of the object by inputting the first image into a first model, control means for controlling to change the settings of the imaging device to second imaging conditions, second acquisition means for acquiring a second image captured by the imaging device under the second imaging conditions, second identification means for identifying the type by inputting the second image into a second model, and the second model is prepared for each imaging condition, and the second identification means uses the second model associated with the second imaging condition among a plurality of the second models. An information processing system characterized by this.

18. A first identification step of identifying imaging conditions suitable for identifying the type of the object by inputting a first image obtained by imaging the object under first imaging conditions into a first model, a second identification step of identifying the type by inputting a second image obtained by imaging the object under the second imaging conditions identified in the first identification step into a second model, including the second model is prepared for each imaging condition, and in the second identification step, the second model associated with the second imaging condition among a plurality of the second models is used. An information processing method characterized by this.

19. A program for causing a computer to function as each means of the information processing device according to any one of Claims 1 to 16.

Images